The ease of introduction was key to success. Efforts to automate operations for a change in awareness within the company, with an aim to make the organization more powerful and sustainable

Ecbeing Inc. is an e-commerce specialist company that has been providing e-commerce building packages since around 2000, with the main business focus being e-commerce website creation.

While e-commerce itself has evolved in the last 20 years, Ecbeing is supporting and developing solutions not only for e-commerce website creation but also for e-commerce related services such as data analysis and marketing.

The Web Solution Development Management Department plays a central role in promoting the company-wide adoption of Autify. We interviewed Mr. Shinichiro Abe, the Executive Officer Mr. Seishi Suzuki, the deputy general manager, and Mr. Koji Takaishi, an architect, about the background and effects of introducing Autify and their future outlook.

(From the left: Mr. Shinichiro Abe, Corporate Executive Officer, Mr. Koji Takaishi, Architect at the Web Solution Development Management Department, Mr. Seishi Suzuki, Deputy General Manager of Web Solution Development Management Department)

— What kind of testing systems did you have before introducing Autify?

Shinichiro: For many years, people handled implementation. This had been an issue for many years.

Our business is essentially development contracting and many individual projects run simultaneously. We would introduce test automation tools for individual projects, but while the program side of the service evolved, the testing couldn’t keep up. It just didn’t last long even if we liked it at first. On the end, it reverted back to manual testing.

We start the project on the premise that we have customers (companies) who place orders. All the workhours required for testing are reflected in the development cost.

Due to the nature of e-commerce, even if it is initially a small site/project, it can gradually become a large project. As the project grows, the test workhours and regression test workhours would increase so even if one person could initially handle testing, it would gradually increase to two, three, four people, and so on and as a result, the staff cost would increase. The increased staff cost for testing means an increased cost to the customer. I wanted to reduce this somehow.

Also, if a staff had to work on testing, it would mean there’s one less staff for application development. It was a vicious cycle.

— The workload would certainly increase as the number of functions increases. What kind of test automation tools did you try?

Shinichiro: I was trying to promote automation by using Selenium and other paid tools. Honestly, there was only so much I could do to catch up with these tools.

— This is when you came across Autify.

Shinichiro: Yes, I learned about Autify when I saw the Forbes JAPAN article around March 2019.

I was able to talk with CEO Mr. Chikazawa around the summer of 2019, and we introduced it around October 2019.

— You’ve been interested in it since it was offered as a closed beta. We’re really thankful for that.

The key to success and the introduction of the test automation tool. The deciding factor was the ease of introduction.

— You have previously tried test automation tools. What did you find appealing about Autify?

Seishi: The biggest appeal was that it was incredibly easy to introduce. It’s a web-based service, and you can use it by simply clicking on the browser. You can quickly create a scenario just by watching a 20-30 minute video tutorial. This was not the case with tools I had tried before.

One of the tools I had experience with is Selenium. You have to read the created code with Selenium, so if a team member cannot code, they can’t use it.

Some testers can’t code, so they won’t be able to get involved from that point on.

Meanwhile, Autify shows screenshots too, which is helpful for everyone to understand visually. It was incredibly easy to introduce.

— The main issue that we are working to solve is the fact that only engineers can work on testing, so I’m happy to hear this.
Did you have any special ideas at the time of introduction?

Seishi: It was Mr. Takaishi who took initiative. “Let’s just try and run it” was our stance and we began by targeting actual applications. I think being able to “just try it” was one of the keys to success.

We didn’t do anything special about how we should proceed. We would just run it, try it, and continued with trial and error. I think it was great that we can and wanted to continue with the trial and error method. Again, I feel that the ease of introduction was what allowed us to do this. By continuing to trial and error, we have been able to gradually expand the projects.

Meanwhile, Autify shows screenshots too, which is helpful for everyone to understand visually. It was incredibly easy to introduce.

Integration with CI/CD and the realization of DevTestOps.

— Did you have any creative ideas after you started operation?

Koji: Since we use it in various projects, we have defined very clear rules for how to name scenarios.

At the same time, we are building an automatic deployment system in the development environment using Autify’s API. This is an effort to further reduce the need for manual deployment.

I’m thinking of making it available for production environments in the near future, too.

We only have a small number of staff, so we can’t make things too complicated, and it’s difficult to oversee the entire platform. Slight changes in specifications can cause unexpected bugs, but we automated tests with Autify to prevent that.

— You mean that you have achieved CI/CD?

Koji: That’s right. When we introduced Autify, we were beginning to introduce Azure DevOps as well. We wanted to automate manual tasks in older projects. The aim was to reduce cost as much as possible by incorporating Autify as well.

Shinichiro: The timing and combination were very good. Aside from testing, we were faced with staff costs and troubles when releasing. When we were talking about resolving these issues, the topic of DevOps came up, and at the same time, we were trialling Autify. So, we decided to do it all together. I think this was perfect.

“Creating an easy environment for everyone” was important when introducing new tools.

— Who engages and operates Autify?

Seishi: A tester creates test scenarios. They delegate any manual tasks during the operation check workflow to Autify.

Even with Autify, we still have to think about how to test and what the test design should be like.

If you ask everyone to do this, testing would be incredibly difficult. Instead, we are focusing on creating an environment that is easy for everyone involved.

For example, Ecbeing has a process of applying a design so we run a test to confirm the design. For this, we create specific examples such as “You can check this with Autify’s capture base” or “Let’s use it like this for regression testing.” Then, we assign it to the person in charge of each project.

By showing the usage applications for each pattern and each phase, the manual operation check can be streamlined by the programmer when deploying, and tests are possible on the tester side as well.

— What about the cost?

Seishi: Each project bears Autify’s fees. It was originally coming from our team’s budget, but now we are trying to make it so that it would be the project’s waste if it’s not used.

It’s a little bit forced (laughs), but we will pick up on each team’s issues, solve them, and standardize them. Then, we will gradually make it so that each project becomes independent.

Autify resulted in a heightened awareness for regression testing.
Separating automated tests and manual tests lead to an improvement in efficiency of the entire testing and a reduction of workhours.

– What are your plans for the future?

Seishi: I think there will be an increase in the awareness of regression tests and degradation. Of course, we were aware of them in the past, but there was an impression that manual testing would take up too much time and that it’s tedious. So, it often did not happen. Automation can handle this, so we started to have a more positive impression about regression testing, and there’s an increased awareness about the importance of regression testing. I think that has been a meaningful change.

Koji: There was a decrease in workhours required for testing in the example of the most recent project. Autify handles degradation tests and standard tests, leaving testers with more difficult and complicated ones and special cases.

— Can you give me an example of a test case that testers do manually?

Koji: Here’s an example. We asked them to test cases such as special cancellation after ordering, and cases where there was telephone correspondence before ordering.

A wider range of tests can be covered as a result.

— We also believe that people should do what only people can do, and everything else should be delegated to Autify.
Have there been any other benefits?

Koji: We made some test cases with Selenium before introducing Autify, but the maintenance cost was very high. The test scenario we made couldn’t be applied to other settings. It was only ad hoc, and we would just stop doing it.

With Autify it’s very easy to apply tests to different settings. For example, a test case used in one version could be used in another version.

I started to recognize these cases, so I started to actively duplicate the scenario created in Project A and expand it to Projects B and C.

Seishi: Before I started using Autify, I was curious how much Autify’s Maintenance AI could do.

In an Ecbeing package, we apply a design to each project from scratch.

With Autify, we were able to run the same test scenario on projects with and without a design applied, and it was able to track changes well. I realized it could be applied to a wider range of settings.

The word “AI” was being overused so I was actually a little bit skeptical about it (laughs). However, I was convinced that Autify’s AI was reliable after seeing this result.

— I’m very happy to hear that you feel that way! Autify’s AI is evolving everyday, so I hope it will become even more useful for you.
We think that the AI is directly linked to the reduction of maintenance workhours for test scenarios during operation. Has this been the case at Ecbeing?

Seishi: We are working to share within the company what kind of test was done in what process, and how workhours for testing has reduced and how quality has improved.

One thing that we think is possible is to monitor continuously.

There are important features in e-commerce websites. For example, you get points if you show your membership card at the POS cash register of a store. However, we can’t confirm whether this function is continuously working at the moment.

Of course, we monitor server services, but in the future, we would like to check whether specific functions and parts controlled by JavaScript are working as well.

— Even if you can monitor the site, monitoring the function is difficult, isn’t it?

Seishi: We’re able to notice if a function isn’t working if it’s straight after a release. However, e-commerce sites like the one we provide add content on the user side. We don’t know the timing or content, so we have no control over it. We are planning on improving the overall quality by properly confirming these changes made by the user and confirming the operation as a service function.

I would like to focus mainly on raising our customers’ and retailers’ satisfaction. I think this will make Ecbeing a more powerful organization.

Outlook

Seishi: As I mentioned earlier, Ecbeing has many different types of services. There is a team for individual development, a team that provides services like SaaS, a team that makes products, and a team that manages site operation and analysis.

Currently, the solution department is taking initiative in promoting Autify, but we will be developing a company-wide automated test project to create a system that is beneficial to the company as a whole.

Do you have any advice for those who are planning on working on test automation?

Seishi: When departments and projects are separate, they sometimes have their own methods. Even if the package is the same, it may be specialized, or the way of thinking may be different.

In such cases, I think we sometimes need to specify the method and almost make it compulsory to use that method and show how to use it with concrete examples.

It’s easy to share information about new tools in day to day work, but that would be the end of it, and it wouldn’t be used in a meaningful way. There are teams that tell us they are interested and want to use such tools. I think showing them and monitoring and reporting the usage will lead to meaningful use.

Shinichiro: There is resistance even within the company. Even when we tried to encourage using it in the company and showed people what it does, people didn’t find test automation itself to be an attractive concept, and there’s a deep-rooted belief that we are managing just fine without it.

Someone even said, “I don’t know what’s so good about AI tools.” Ironically, the department that said that has not managed to use it.

In order to stop sticking to tradition, we need people to imagine what things could be like in the future and prove to them that it’s possible by showing them examples and results.

Projects that could be more efficient tend to be the ones which are insistent on staying the same. Not just with testing but also how documents are managed. I want them in particular to adopt new ways, and I think it would be worthwhile.

Currently, I am planning on gradually building up results with new projects and nudge the company culture towards something better.

By that, I mean being able to concentrate on the necessary work and improving work efficiency. I believe that we will have a positive effect by working on improving productivity.

I would like to have a strong will and keep striving in order to make the organization stronger in the current social climate.