Automate Testing With Autify Before Service Expansion: How NearMe Solidified Quality by Automating Mobile and Web App Testing

Mobility as a Service (MaaS), a next-generation transportation service that utilizes ICT to make transportation more efficient and convenient, is attracting attention globally. Even in Japan, MaaS is gaining traction with the rise of car sharing.

NearMe, Inc. offers nearMe.Airport, a car-sharing service that provides transportation from the user’s home to the airport. The idea is to reduce empty seats by matching users with available seats in round-trip transportation services between cities and airports, such as taxis.

Since the Ministry of Land, Infrastructure, Transport, and Tourism lifted the restriction on sharing taxi rides in November 2021, the company has expanded its service offering with the launch of nearMe.Town, which allows users to share transportation around town.

We interviewed Mr. Yuichi Komori and Mr. Yuta Okamoto, both responsible for service development, operation, and support at NearMe, about the challenges they faced with QA and their test automation efforts.

As the service offering expanded, testing got increasingly complicated. We considered automation at an early stage.

– What do you do at NearMe?

Yuichi: NearMe is a transportation service, so we partner with taxi and car rental companies. I’m in charge of supporting those partner companies with the implementation and operation of our platform. We call Partner Success (PS). I also help with day-to-day operations. As a part of that, I also use Autify for QA.

Yuta: I’m responsible for engineering, which calls airport transfer routes, and QA. In terms of QA, I’m also in charge of running regression tests on our web- and mobile-platform, reproducing and checking bugs, and manual testing when there are changes in the platform.

– Sounds like you both have a lot on your plate. What kind of QA challenges did you face before implementing Autify?

Yuichi: We ran tests manually and released if there were no issues about a year after launching our airport transfer service, nearMe.Airport. As we expanded our service offering, testing became increasingly complex, so we began looking into test automation. We decided to create a QA process where we automate as much as possible and manually check complicated areas.

Autify had a booth next to us at a TechCrunch event, which is how we heard about your automation platform. We implemented Autify a year later.

– I’m glad we attended TechCrunch! Was there something specific that made you go for test automation?

Yuichi: We didn’t really think too deeply about how we should go about testing. At the time, we hadn’t experienced any major bugs or crises just before a release.

However, we were expanding our transportation services for businesses, such as airport transfers and golf transfers, and the number of landing pages was also increasing. That’s when we thought we needed to have some sort of preventative measure. We realized that manual testing would eventually reach a limit.

Daily testing in development and production environments

– How many services do you offer, and how often do you release?

Yuichi: Currently, we release every two weeks, but we do minor releases several times a month. In our case, each of our five services has pages for administrators and drivers. This means that when the frontend changes, the backend also has to be tested.

We also launched a mobile app for users a year ago, and a native app for drivers is in the making. The scope of testing will keep expanding. I tried Autify with former staff and found it surprisingly easy. I handed over testing to Yuta, who joined the company as a new graduate this April.

Yuta: I test once or twice every day. I run tests in the development environment once or twice a day for essential areas, twice a week on Tuesdays and Thursdays for areas that are less important but we still want to check. We also have a daily and weekly schedule for production environment testing to keep an eye on it.

– After implementing Autify for Web, you’ve launched a mobile app for users. How did you test the mobile app?

Yuichi: When we first launched the mobile app for users, the download count was low, so manual testing was enough. But we decided to strengthen the mobile app last year.

Testing mobile apps requires more time and effort than web apps because you need to prepare test environments with real devices. We have a number of different terminals, but manual testing couldn’t cover them all. Around this time, Autify released the mobile app test automation platform, Autify for Mobile, which was perfect timing.

Yuta: I like that Autify for Mobile allows us to select an iOS version and run tests on each.

Run positive tests more frequently and maintain quality

– How did you go about implementing Autify and automating tests?

Yuichi: Out of our five services, we designed the tests for the landing page of our main service, nearMe.Airport, so that we can check the user flow exhaustively. At first, we created a simple screen transition diagram to create positive test scenarios on Autify, which test whether we get the expected results during normal use.

Yuta: Similarly, we test the mobile app’s basic user flow from login, making a booking, to canceling. It would be tedious to test manually in the morning and afternoon every time the code changes, so I use Autify as much as possible to get it done automatically.

Yuichi: Errors that usually don’t appear can show up when unusual data is entered. For example, when the user is matched in our carpool matching feature, an issue may occur, resulting in the next screen not being displayed. Even problems that go unnoticed in regular testing can be caught by running tests more frequently. We find Autify very useful in those areas.

Some issues are hard to find if you only test once every day at a fixed time. For example, say you run a test in the morning and the afternoon. If the results are different, you can deduce that the cause is related to timezone. The biggest advantage of automation is that you can repeat the test any time, finding bugs that might be missed the first time.

– Your company has been working on test automation since the early stages of launching. Why did you invest in testing from the get-go instead of putting automation on the back burner?

Yuichi: We are a startup, so if the quality isn’t there and the service doesn’t work as it should, we wouldn’t be seen as reliable. If key features fail and we don’t know why, we can’t just say, “let’s test manually every time we release.” It would be too difficult.

Thankfully, we’ve only had minor bugs, such as fixing a small portion of the screen after release. The reason major issues haven’t occurred is that we’ve tested regularly and dealt with issues as we find them. Before we implemented Autify, the scale of our services was still pretty small, so it wasn’t an issue. But if we had kept going without taking any measures, we probably would have had several major problems.

Utilizing the memo feature, Step Groups, and the email feature

– Do you use Autify in any creative ways?

Yuta: I use the memo feature to leave notes to share things to look out for with other members. I try to make handover as easy as possible when the QA system eventually changes or when new members join us. I also make maintenance easier by using the Step Group feature and creating a list of commands.

Yuichi: How did it go when you took over this role?

Yuta: At first, I didn’t know what QA even was. But using Autify was intuitive, and I was able to create test scenarios by interacting with the screen.

I like that Autify automatically saves screenshots when errors occur. Also, Autify’s support team is quick to respond. Even when I asked very basic questions, they were very helpful. They’ve helped me many times.

– You’ll soon be able to insert Step groups in scenarios, which will make it even more useful.

Yuta: That would be great. I’m looking forward to it. I have also been using the email feature, and it’s extremely useful.

– Thank you. I’m glad our effort in developing the email feature was worth it.

Advantages of using both the mobile version and the web version of Autify

– You use Autify for Web and Autify for Mobile. What is the advantage of using both?

Yuta: Creating and running test scenarios, Slack integration, and API integration are all done the same way on both the web and mobile versions. Once I used the web version, getting started with the mobile version was straightforward.

Yuichi: Around the end of last year, we found that recording on the mobile version was very slow, so we actually considered using another tool for mobile testing. However, the fact that you can use Autify for Mobile and Autify for Web in the exact same way was a major advantage.

Yuta: I began taking over in March and started dabbling with the mobile version around the beginning of April. I think _Autify for Mobile _has gotten a lot faster in April. I also like that we get to see the cute bee at startup!

– I’m glad! We are working hard on performance improvements, so it will keep getting better. Startup will also get much faster. We will continue to focus on improving performance and stability, so please send us any feedback you may have.

No more post-release issues and high quality is being maintained

– After implementing Autify, what kind of changes and benefits have there been?

Yuichi: The most significant benefit is that we haven’t had any major post-release issues. Yuta is our main tester, and others get assigned when there are a lot of test items. Thanks to automation, we’ve been able to test without being under pressure. It’s as if someone called Autify is working extra hard for us!

Yuta: In terms of quality assurance, Autify allows us to keep a close eye on things through daily testing. When there is a problem, the screenshots make it easy to figure out what went wrong and fix it quickly. Since we generally work remotely, having visual information makes communicating on Slack more smooth. That’s one of the things I find useful.

Autify supports parallel execution, and I like that we can multiple test cases in a short time. By delegating to Autify as much as possible, we can spend time manually checking other areas.

– Which areas were hard to automate?

Yuichi: Some data patterns, such as timezones and location information, can cause issues. For example, there could be no issues when a booking is made in the Japanese timezone, but the booking time can be off when it’s made in a US timezone. This type of issue is rare and is only found if a user gets in touch with us, so the need to update the feature is pretty low. That’s not to say we don’t care about rare issues! I think we can improve quality even more if we can automate negative testing.

The psychological benefits, not just quantitative benefits, are significant.

– What’s next for you in terms of test automation and in general?

Yuta: I would like to standardize our overall QA process. The way we use Autify has already been standardized; we use the memo feature, so anyone can use it. I’d like to use the time that Autify has freed up to create a standard procedure in other areas.

In terms of testing, I’d like to expand test coverage by having more negative testing scenarios. I would also like to work on CI/CD integration down the line.

Yuichi: We hope to make it so that the platform can accept more bookings with the same amount of resources, even when the number of users increases. Positive testing is still lacking, so I hope to expand test coverage there as well. I would also like to make it so that we can run tests using data similar to that in the production environment.

– Do you have any advice for those planning to work on automating tests?

Yuta: Autify will not only save you time, but it will also give you peace of mind. Test automation may seem like a daunting task, but once you’ve made scenarios on Autify, all you have to do is press a button. If you’re considering Autify, I think it’s worth thinking about the qualitative benefits, not just the quantitative.

– Finally, do you have any announcements?

Yuichi: We are hiring various engineers, including web engineers, frontend/backend engineers, mobile engineers, etc. Please apply if you’re interested. We look forward to hearing from you!

– Thank you very much!

(Interviewer: Ryo Chikazawa, CEO, Autify Inc.)