How Did Asoview Automate 200 Scenarios? ‘It Really Is a No-Code Testing Solution for Everyone!’

With the COVID-19 pandemic, we’re all encouraged to stay at home. Although some leisure facilities had no choice but to temporarily close, many others remained open with extensive measures to prevent the spread of the virus, such as taking reservations, restricting the number of people in the facility, and setting time limits.

Asoview!, a ticket sales website for leisure facilities, has been updating its system to meet these changing demands. In these unusual circumstances, how are they maintaining quality?

In this interview, we spoke to two people who worked on test automation at Asoview, Inc.: Mr. Daisuke Takeuchi, a development manager, and Mr. Masaya Aoyagi, a QA engineer. They talked about the challenges they faced before automating testing and how they are currently using Autify, our test automation platform.

(Left: Mr. Masaya Aoyagi, a QA engineer Right: Mr. Daisuke Takeuchi, a development manager) Photo courtesy of Asoview, Inc.

The pandemic changed demands in the leisure industry

– Please tell us about you and your company.

Daisuke: Asoview, Inc. is primarily a system where users can buy tickets for outdoor/indoor activities online. In addition to leisure facilities such as amusement parks and aquariums, we also sell tickets for activities such as pottery classes and craft experiences. In addition, we offer business solutions for companies that want to sell tickets directly to consumers online. I’m the development manager of our ticket sales system.

Masaya: I’m in charge of QA for e-tickets, mainly working on product testing and quality improvement.

– Did the COVID-19 pandemic change the way we spend our free time?

Daisuke: Yes, because of coronavirus, people go out nearly as much. The key for leisure facilities to succeed in the pandemic is to manage the number of people per site area. We responded to this change by adding new features. More companies seem to be taking this opportunity to create a system where there used to be none. The pandemic has definitely changed what people need.

The system became complex, and test cases grew

– What challenges did you face before implementing Autify?

Daisuke: We are continuing to add features to our online ticketing system. Last year, in response to the pandemic, we developed tickets that allow users to select conditions like capacity restrictions and pick the date/time. As a result, the number of product variations increased, inflating the amount of testing we had to do. We realized that we have to run tests efficiently while maintaining quality.

– In addition to capacity restriction, it must have been challenging to make sure the number of slots was correct after cancellations.

Daisuke: That’s right. A complex system is more prone to issues. So, we inevitably spend more time testing, causing the release cycle to slow down. We only have a certain number of QA staff, so testing puts a massive strain on us.

– Masaya, you are the only person responsible for QA. Did testing become overwhelming?

Masaya: We were adding more features to respond to the pandemic. This means there were more and more test conditions I needed to account for. For instance, when I created a matrix, the number of test cases ballooned to thousands or tens of thousands. We used to do all of that manually, but I quickly realized that I was reaching a limit. I couldn’t handle all these tests by myself, so I often had to ask the support team and other team members to give me a hand.

Ease of maintenance is key

– I’m sure you considered other test automation platforms other than Autify. How did you end up picking Autify?

Masaya: About a year or two ago, we invited a QA engineer who was good with Selenium to automate testing for specific areas in our platform. However, maintenance didn’t go well. We ended up not using the test scenarios we created.

After we were faced with the issues mentioned earlier, a fellow team member told me about Autify. I looked into it. It seemed easy enough, so we decided to try it out. Once we started using it, we realized how useful it is and decided to implement it fully.

– You mentioned that maintenance didn’t go well with Selenium. What exactly was the issue?

Masaya: In our current QA system, there isn’t anyone who can program automated tests. Features are being fixed or added all the time. We simply couldn’t keep up with fixing the test code.

– And you thought that no-code tools like Autify would be more straightforward.

Masaya: During Autify’s trial, I tried creating test scenarios and test plans. I was able to use it intuitively and found it easy to use. It could also save maintenance costs, so I requested that we implement Autify.

Clarify ownership

– How did you go about implementing Autify after that?

Masaya: Since the test targets were web tickets, the first thing we decided to automate with Autify were existing regression tests for web tickets. We have a release checklist, so the first step was to create Test Scenarios on Autify according to the list.

Replacing the release check with Autify went smoothly. However, test automation stagnated for a while because we started the process without deciding who would do what and by when. So, we created a system for how we were going to use Autify.

– It was unclear who was responsible for making Test Scenarios. That does seem to happen a lot. How did you overcome it?

Masaya: Daisuke and I recognized the issue and devised a strategy. First, we prepared a test case and clarified the Test Plan. We also chose a person in charge of implementation and worked with them to create test cases on Autify from the top down.

As the number of test cases increased, it became difficult to know the purpose of each test case. So, we set up a naming convention so that the aim of test cases was easy to understand.

As we replaced release tests, I added links to Autify so that we could run tests, even if it’s just partially. We also used the Step Group feature.

– How did the implementation manager react?

Masaya: We had a base in Vietnam, so we ran a tutorial on using Autify for the development members there and had them create Test Scenarios according to Test Cases. As we started using Autify, we followed up on any issues or questions, but there was no feedback that the platform was hard to use. In fact, we got a lot of positive responses.

– For companies that already have a QA team, the key is to think about getting the development team involved. How did you get them engaged once you strategized?

Masaya: I assigned tasks for the Autify implementation project. There were no significant problems or pushback, and implementation went smoothly. My supervisor, Takeuchi, also understood that automation is essential for ensuring quality. This empowered me to be proactive with getting the development team members involved. Instead of leading the way, we were able to work on automation as a team. This was possible because our supervisors were understanding.

– Daisuke, as a development leader, what made you aware that test automation is important?

Masaya: For a while, we always encountered some kind of bug or an issue every time we released. That’s why I began thinking about how we can ensure quality.

Automated tests for 18 items with 200 scenarios

– Of all the items on the checklist, how many were you able to automate with Autify?

Masaya: There were 18 main items on the checklist, and among them, we’ve already automated all regression tests for our ticket system.

When users purchase tickets, the conditions can get complicated. So, we have 18 items from which to branch out, and we have about 200 test scenarios in total. If you were to run them all manually, it would take about two to three hours. We’ve saved a lot of time by automating those tests with Autify.

It allows us to automatically test on both PC and smartphone environments, freeing up two to three hours. Also, there are some variations in accuracy and speed when we used to do this manually. Since transitioning to Autify, quality has been stable too.

– It’s incredible that you can run 200 scenarios automatically.

Masaya: We’ve linked Autify with Slack, so we are instantly notified of test results. We get to check results and catch any issues quickly.

Create scenarios efficiently with the Step Group feature

– Do you use Autify in any creative ways?

Masaya: We’ve prepared a scenario that we run daily. This is helpful because we can check for issues in the standard features every morning. Other than that, we’ve put together a test plan that we run before release.

Currently, we use Autify for pre-release regression testing. I hope to expand its use in CI (Continuous Integration) testing, but we haven’t worked out the specifics for those scenarios. That’ll be our next step.

– What other Autify features do you utilize?

Masaya: The Step Group feature has been invaluable. In our ticket system, there are so many combinations after the user accesses asoview.com and selects a ticket. How many adult tickets would they like? What about tickets for children? Will they use points or coupons? Which payment method will they use on the next page? There are so many possible combinations.

We’ve grouped each selection into Step Groups and used them when creating scenarios. I use it so much that I don’t know how I would have managed it without it.

Even when making changes, all I have to do is change the Step Group once, and it’s applied to all scenarios, which has reduced maintenance costs a lot.

– I’m glad the Step Group feature is helpful when making scenarios with different combinations of steps.

Masaya: Also, as someone who does the final check, I like that the test results are visually shown using color and execution time. I also like that it takes screenshots. It helps us start solving issues when an error occurs.

Test automation allows you to focus on development

– You’ve managed to automate all 18 items that you used to test manually. Would you say that the biggest impact with Autify has been the time savings?

Masaya: Yes, Autify has drastically saved time. Another thing is that anyone can now run regression tests before release. All you have to do is hit Run. It is no longer dependent on one person. No one has to pause what they’re doing because you can run the test yourself

Since introducing Autify and automating pre-release tests of ticket sales, other departments have shown interest. So, we’ve been getting them to try Autify for their products’ pre-release checks.

Daisuke: Development staff used to do some QA work. However, that’s hardly ever the case these days because Autify handles testing. This has allowed developers to focus more on development.

Masaya: When I talk to other members about Autify, people seem very keen. Because of Autify, there’s more awareness and understanding towards test automation.

Autify is easy to implement and will get you excited

– What’s next?

Masaya: I plan to use Autify to automate regression testing in other teams and eliminate as much manual testing as possible. Humans have to make test cases. That’s not something we can get rid of completely. As I continue working on this, I will keep in mind that it’s all about the balance between automated tests and manual tests.

In our case, there are huge test scenarios that check the entire flow from posting tickets on the website until selling them. I’d like to automate it with Autify as well. I’d like to use Autify as one of the ways to continually monitor quality.

– What would you say to someone who wants to automate testing?

Masaya: Autify claims that anyone can easily create tests. We found that this is absolutely true. Implementing Autify is incredibly easy. As the person who runs tests, Autify makes life much easier because results come with screenshots, allowing you to understand easily.

Autify is something that gets more exciting the more you use it. I recommend using Autify as one means to automate testing.

Daisuke: We started Autify by automating small tests. Out of the 18 items, we began with a few cases to see if Autify could really be used for regression testing. It’s an easy platform to start on a small scale. As Masaya mentioned earlier, the UI is easy to use, so it’s straightforward to begin.

– Finally, do you have any announcements?

Masaya: On Asoview, you can make bookings for a wide variety of leisure facilities all over Japan. Please take a look when planning your weekends or when you go on holiday.

We are also hiring engineers. We provide our customers with exciting activities all over Japan. If you’re interested, please take a look at our website and get in touch. We look forward to hearing from you!

(Interviewer: Ryo Chikazawa, CEO & Co-founder of Autify, Inc.)