How Test Automation Creates an Upward Spiral: MiiTel's Vision for Global Expansion

In today’s society, increasing productivity is on everyone’s mind. However, it’s not easy to improve efficiency in business negotiations and customer service, where communication skills are critical. Oral communication, especially telesales, directly impacts business performance, so while old-school, it’s effective.

Meanwhile, a new service that digitally transforms telesales has emerged. MiiTel is a VoIP (Voice over IP) service that utilizes AI to analyze telemarketers’ speech. Calls are automatically recorded, transcribed by AI, and content analyzed and visualized. The CRM integration feature allows users to remotely manage customer service by overseeing staff by keeping an eye on when someone said what to whom.

In this interview, we spoke to Mr. Takekatsu Hiramura (CTO of RevComm, Inc., which develops MiiTel), Mr. Ken Ikehata, and Ms. Kanako Fukuda (In charge of QA projects) about how they approach QA in development.

The number of clients rapidly increased, and QA needed to be faster

– Please introduce yourselves.

Takekatsu, RevComm: I’m the CTO at RevComm, Inc. I’m responsible for organizational development and management, as well as planning cloud services, building service specifications, designing system architecture, and developing speech recognition technology and speech communication infrastructure using machine learning and neural networks.

Ken, RevComm: I am a server-side engineer at RevComm Inc. Besides server-side operations, I’m responsible for QA projects and implementing E2E testing.

Kanako, RevComm: I work in customer support. Like Ken, I’m also involved in QA projects and test automation.

– What led you to implement Autify?

Takekatsu: What prompted us was because the number of clients rapidly increased, and some users were using our platform in a way that we hadn’t anticipated. At the same time, we began releasing more frequently, which meant that QA had to be faster.

– Before Autify, were you testing manually for the most part?

Ken: Yes, customer support would test manually, and sometimes engineers would run tests on their side as well. With a rapid release cycle, we had to find a way to identify unexpected issues and expand test coverage.

– Before implementing Autify, did you consider other testing tools?

Ken: We sometimes ran tests that we developed using Selenium, but the code and scenarios had to be changed for each product, making it expensive to operate and manage. On Autify, we can reuse a scenario on other products by simply duplicating an existing scenario and tweaking some values. It’s helped us save time managing test scenarios by a long shot. After considering the time savings, we decided to use Autify to improve quality and release faster.

The deciding factors were that Autify supports WebRTC and sends test result notifications

– What was the process of implementing Autify like?

Ken: We began trialing Autify in March 2021 to test what kind of scenarios we can create. MiiTel Phone has a feature that allows users to make and receive phone calls, and the technology uses WebRTC and WebSockets. We wanted to see if Autify could handle them.

Along with the CTO, tech lead, and the infrastructure lead, we consulted Autify’s support team. The whole engineering team at Autify helped us out. We were able to confirm that Autify works with WebRTC, so we concluded that we’d be able to run QA tests sufficiently and decided to implement it.

Kanako: Making phone calls is one of MiiTel Phone’s basic features, and not being able to do so would be a critical issue. If we can automate testing it, we can ensure quality naturally. Autify automatically records tests and allows everyone to review them. If there are any errors, it notifies you on Slack. Sharing test results with team members is important to us, and I think this feature encouraged us to go ahead and implement it.

– How often do you release at RevComm?

Ken: It depends on the product. For example, MiiTel Analytics releases three times a week. We work on multiple products at the same time, so something gets deployed every day.

– What were the deciding factors for choosing Autify?

Ken: There are other services like Autify. What makes Autify different is that Japanese is fully supported in its UI. This makes it easy for anyone to use the platform, so we had one less barrier to implementing it.

As I mentioned earlier, it was important for us to have a test automation platform that supports WebRTC. We turned to Autify’s support team for advice, and they suggested specific scenarios, and the response was quick. Those were major deciding factors.

– Thank you. WebRTC isn’t easy in E2E testing, is it?

Ken: Once we contacted you about this, we found that someone at Autify had added a sentence to an FAQ article: “Please don’t set up BASIC Authentication when enabling WebRTC.” You acted fast!

Scrutinize scenarios and set rules to avoid unnecessary test runs

– You used to run tests manually before Autify. How did you migrate test cases to Autify?

Ken: When I first used Autify, I found the UI intuitive and simple. We began by re-evaluating MiiTel’s test specifications. Each unit within the scenarios was made into a Step Group. We created Step Groups for each service and perfected scenarios through trial and error.

– Who’s in your QA team? How did you go about automating testing?

Takekatsu: In our company, the support team has a QA person, and the development team is also involved in QA from a development point of view. Quite a few members use Autify. The support and development teams worked together as a task force to automate tests.

– How did you go about it? Were there any tips and tricks when implementing and operating Autify?

Ken: We had an internal discussion about when to run tests. We can run 400 tests per month on Autify, and while it gives us plenty of value for money, we’ve agreed not to waste it. We scrutinize existing scenarios and run regression testing in staging/production environments. When other companies’ services are involved, external factors can cause failures, so we’ve decided to run those tests every day since last month. We’ve gotten into the habit of running tests on Autify when languages and frameworks are upgraded. It gives us peace of mind that quality is ensured at a bare minimum.

When an error occurs, Autify lets us know why it went wrong, but the cause isn’t as clear compared to when it’s written in a programming language. However, everyone is starting to get used to Autify, so we can figure it out. It’s great that we are gradually getting better at utilizing Autify.

– It’s common for companies to successfully implement our platform in one team or product but struggle to do so company-wide. Why do you think it was successful in your company?

Ken: It wasn’t easy at first. However, we increased the number of people who can use Autify by having new members join us in developing scenarios, thereby transferring knowledge. On one occasion, Autify alerted us on a staging environment, which was a concrete example that Autify helped prevent an issue. Gradually, everyone began to realize how beneficial it is. After that point, we started making more and more scenarios on Autify. We also made it a rule to clarify which test case had been automated on Autify.

– You expanded its use by passing on knowledge, making rules, and communicating the results.

Sharing test results to refine scenarios further

– Are there any Autify features that you find particularly useful?

Ken: We use Step Groups the most. It’s also helpful that we can share scenarios with other teams. For operation tests, being able to add test email addresses helps a lot.

We love the Slack notifications too. It allows us to check test results and have team discussions on how to improve, making the scenarios more and more refined. It’s leading us to have a virtuous cycle. Slack notifications are a must-have feature, so I think everyone will love it if this functionality is strengthened.

– We released a feature that allows you to receive Slack notifications for each test plan. Each test plan’s notifications can be posted on different channels. We hope you find it useful. Have you integrated Autify with external tools?

Ken: Yes, we’ve integrated it with Salesforce. Recently, Salesforce made it mandatory to enable two-factor authentication for login, and we are developing MiiTel in response to this change. In Autify’s Help Center, there was documentation on handling MFA in a login test (access the server and get an OTP). We tried it and were able to solve this issue, so we hope to create even more scenarios.

Reducing operation costs by having Autify tests ready

– What effects did you see after implementing Autify?

Ken: We were always worried if everything worked correctly around the time of release, but now that we have automated regression tests with Autify, we have a sense of security. Having less stress helps us move on to the next task. It makes me glad that we implemented Autify.

We prepare scenarios on Autify before developing new services and features, which keeps operational costs low. Developers work together and utilize JS Steps to create scenarios. We are also working on creating a shared repository for JS Steps.

Kanako: We re-examine the reference manual when automating, so the overall accuracy of the test scenarios has improved.

Takekatsu: Often, we run manual tests after releasing at night during maintenance hours. However, we wanted to avoid this as much as possible to respect everyone’s work-life balance. I’m happy to say that Autify has helped with this issue.

Taking on the global market by expanding functionality and supporting other languages

– What’s next?

Ken: One of our goals is to cover more ground with automated tests. We would also like to consistently review scenarios, create a workflow where scenarios for all products are automated using Autify, and reduce test time.

Kanako: We want to keep providing services that customers can rely on. It is important to develop products that are even more convenient while ensuring quality. We will be utilizing Autify to create great products with confidence.

Takekatsu: RevComm’s development principle is to develop products faster and make them more convenient. We plan to expand features other than phone calls and gather information. We are currently growing our business in Japan and Indonesia, with further global expansion on the horizon. We would also like to support other languages and offer integration with other products such as SFDC and HubSpot.

Automate tests early to achieve product growth

– Do you have any advice for those planning on automating tests?

Ken: As the SaaS business model is becoming more widespread, how we think about cost is key. If it’s going to save your time, automating is worth it, even if it costs money. I believe services like Autify will be essential in the future. Rather than considering Autify after the product has grown, it’s better to use it from the get-go so that you can grow the product without ever compromising quality.

Kanako: I’m sure there are tests that you can’t automate and errors that only humans can find, but honestly, running the same manual test over and over is a massive pain. When it’s a time-consuming test, you will lose concentration. Please ease the burden for testers by implementing Autify.

Takekatsu: Hiring testers is pretty challenging. If you automate tests with Autify, you can achieve your objectives at a low cost. It’s well worth considering.

– Do you have any announcements you’d like to make?

Takekatsu: MiiTel Phone is a valuable service, so we hope you try it out. We are hiring development team members, test engineers, and QA engineers. We are looking for people who want to make the world a better place through speech, cloud, and AI. Please apply if you’re interested.

– Thank you very much!

(Interviewer: Ryo Chikazawa, CEO, Autify Inc.)