Repaying Technical Debt to Create Value and Speed up Development: Why is READYFOR Prioritizing Test Automation?

With the vision to create a world where anyone can achieve their dreams, READYFOR provides a crowdfunding platform for areas where money does not flow easily through existing financial services. This includes NPOs, medical care providers, research institutions, and regional revitalization programs. So far, approximately 20,000 projects have raised about 20 billion yen on the platform.

Last year, they established and operated the COVID-19 relief fund with The Tokyo Metropolitan Community Foundation, raising about 870 million yen, making it the largest crowdfunding project in Japan. Funds were distributed to aid social activities. As a company that makes more money flow from people who care, they strive to ignite change that goes beyond crowdfunding. What challenges do developers face every day to realize this grand mission, and how are they trying to create new value? We spoke with Mr. Hiroshi Ito and Ms. Reiko Ishii, who are in charge of development and QA at READYFOR, Inc.

With the vision to create a world where anyone can achieve their dreams, READYFOR provides a crowdfunding platform for areas where money does not flow easily through existing financial services. This includes NPOs, medical care providers, research institutions, and regional revitalization programs. So far, approximately 20,000 projects have raised about 20 billion yen on the platform.

Last year, they established and operated the COVID-19 relief fund with The Tokyo Metropolitan Community Foundation, raising about 870 million yen, making it the largest crowdfunding project in Japan. Funds were distributed to aid social activities. As a company that makes more money flow from people who care, they strive to ignite change that goes beyond crowdfunding.

What challenges do developers face every day to realize this grand mission, and how are they trying to create new value? We spoke with Mr. Hiroshi Ito and Ms. Reiko Ishii, who are in charge of development and QA at READYFOR, Inc.

Reinforce QA if you want to scale up business

– What do you do at READYFOR?

Hiroshi: I’m the VP of Engineering (Vice President of Engineering). The team was pretty small when I joined the company in October 2019, with only ten engineers, including myself. However, the team has grown to have 25 members in the last two years, and QA has been one of our priorities.

Reiko: I joined the company as an engineer as a side job. To scale up our business, we wanted to create a development lifecycle that doesn’t compromise quality or slow down. That’s why QA has been one of our focuses.

Technical debt was slowing down development

– What challenges did you face before implementing Autify?

Hiroshi: READYFOR is a service that started in 2011, and I joined in its eighth year. In those eight years, READYFOR consisted of a small group of people working to create more value as a startup. Meanwhile, technical debt was slowly accumulating.

When we started aiming for non-linear growth by adding value to our product, not just through crowdfunding, we realized that we needed to properly deal with the technical debt.

We had to maintain quality, create new value, and repay the technical debt simultaneously. With only ten engineers, this is asking a lot.

So, to repay the technical debt, we thought we needed to develop while keeping regression as low as possible.

Until then, engineers and product managers would spend hours nervously going through test scenarios manually when Rails upgraded or when there was a major refactoring.

Working like this was time-consuming and made it hard for us to focus on creating value. In that sense, we were aware that we needed to implement an E2E testing tool.

– Do you think your company is highly aware of quality and cost because your service deals with money?

Hiroshi: That’s part of it, but the main reason was that the technical debt that we’d accumulated was already putting pressure on the entire company in a way that we couldn’t see. The engineering team kept the company up to date about our technical debt, which may be why we became increasingly aware of the issue.

– It must have been tough to communicate with management about technical debt. How did you go about it?

Hiroshi: I used a balance sheet as a metaphor to explain technical debt, not just to management but also to the whole company. People understand what a balance sheet is, and the word “technical debt” is very similar to financial debt. What I presented at the Object-Oriented Conference was similar to that.

Within our total asset, which is our software, there is a certain percentage of technical debt. In my explanation, I called the remaining part ‘technical net asset’.

The higher the technical net asset to total asset ratio is, the faster you create value. In other words, when you have a high net worth ratio, the software’s total asset will grow more quickly.

Conversely, a high level of technical debt slows us down. Having debt is easy, and you can use it to increase assets during the early stages. I explained that we can create value faster by gradually increasing our technical net asset ratio.

– That’s an intuitive way of explaining it. Of course, you have to keep increasing your net asset, but even if it stays the same, you can develop faster by reducing debt. Is it also important to increase test coverage and make regression testing more efficient to reduce technical debt?

Hiroshi: Yes, it is. We don’t want to create issues by releasing without checking that all critical parts of our software work fine.

– Did you consider Selenium for test automation?

Hiroshi: Yes, we did. However, we don’t have many engineers on our team, so we decided that pushing through using Selenium would be counterproductive and costly. When we came across Autify, it seemed like a life-saver.

I’ve been an engineer for a long time, and speaking from experience, I know that E2E tests fail quite often. If something breaks frequently, it has to be easy to fix and maintain.

If something breaks a lot and is hard to maintain, you’re better off not using it. In that case, I would have thought that it might even be better to carry on testing manually. Autify is designed to make test scenarios easy to fix, so naturally, I had to use it.

– How did you get to implementation?

Hiroshi: We were already aware that READYFOR needed to improve its QA processes, and Autify came up when we consulted Reiko about it.

Autify was on our mind as we carried on with development. However, with only a few engineers and product managers, the initial cost of implementing Autify was an issue. It was hard to take the plunge.

While we were waiting for the right timing, I heard Reiko mention that her main job was using Autify quite heavily, so I asked whether she was interested in helping us out! So we got Reiko on board and started implementing Autify.

Reiko: I didn’t think it would be such a big responsibility (laughs).

We talked about maintenance costs earlier, and that’s what we focus on most when automating. Anyone can implement an automation tool, but the question is who will use it and how. If specifications change, the appearance of the screens will also change. We need to make sure we can test those changes quickly.

Rather than having a team of designated automation engineers, it would be great if management, or anyone in the company for that matter, could readily fix any broken parts in the Test Scenario when they have time to spare.

Creating Test Scenarios started small

– Reiko, you said you joined the company part-time. How exactly did you participate in the project?

Reiko: Since there wasn’t a dedicated QA manager, that’s what I did and focused on testing first. Although I had limited time as a part-timer, I could bring a lot of value if I could use Autify.

After discussing how we can sustainably automate testing, we decided on Autify. Then, I discussed the type of tests and scenarios we wanted to have with the product manager and began creating what we needed at the bare minimum.

I was able to see how the existing test environment worked and asked what they wanted to test. I made a list of how users would interact with the platform, how it would behave, which screen elements needed to be checked, and which user journey we needed to test.

There are cases where users have to enter different content depending on their selection, so we discussed whether to create a test scenario for each user journey or stick to the main journey.

– Did you pick and choose what could be automated after that?

Reiko: It’s rather difficult to list everything that needs testing and then pick out the ones that can be automated. So when I surveyed the situation, I focused on what Autify can automate.

Some tests are quicker done manually, and some can be done by machines. So, I studied what parts are repetitive and concentrated on automating them.

– It sounds like your previous experience with Autify helped.

Reiko: You’re right. If I hadn’t used Autify before, I don’t think I would have been able to tell what can and can’t be automated.

– You mentioned earlier that you wanted to make it so that anyone can test with no-code, even non-engineers. Some people say that even with no-code testing tools, you can’t test without QA experience. What do you think?

Reiko: You absolutely need to know about QA and have some QA experience when creating complicated test scenarios. However, I also think that if the base scenario is ready, people can do it correctly with some thought.

Rather than thinking alone, you can get valuable advice by asking in morning meetings and getting others involved. It doesn’t have to be perfect, and you only create a new scenario once. Most of the time, you maintain/modify existing scenarios or create a new one by combining or reusing existing ones. I think anyone can do it if you work on it like a puzzle.

– Do you think a successful technique would be for someone with QA experience to pave the way in the initial stage and then hand it over so that everyone else can use it?

Reiko: Yes, I think so. Someone smarter might say there’s an entirely different way that’s a lot better, though!

The concept of “time” is a challenge that's unique to crowdfunding

– What was your biggest struggle when implementing Autify?

Reiko: I often struggled with ****making sure tests were repeatable because READYFOR’s state changes over time. I could solve some issues by creating and running scenarios myself, but I sometimes had to get advice from engineers to fix them.

– Did you have trouble with making sure the E2E tests yielded the same result every time?

Reiko: Yes. It wouldn’t be good if the second test was executed in a different state than the first one!

– Hiroshi, how did you respond to this issue as an engineer?

Hiroshi: It’s still one of the things we find challenging. We deal with it on a case-by-case basis. One of the biggest hurdles, which is unique to crowdfunding platforms, is time.

Each crowdfunding campaign has a fixed duration for when people can make contributions. After the deadline, people can no longer fund the campaign, and you’ll know whether the campaign succeeded or not. In an E2E environment, the deadline for a crowdfunding campaign will eventually pass. In other words, the state changes.

At the moment, Reiko deals with this issue by creating a new project after the deadline passes, but this is a costly way of doing maintenance (Editor’s note: this issue has been resolved by automatically extending the project deadline of the test target by batch processing). With testing, the concept of time is a complicated issue.

Utilizing features such as Step Groups and regular execution

– Do you use Autify in creative ways?

Reiko: We use the data testing feature to improve efficiency. This is useful because you can create a scenario template and just change the user details, crowdfunding URL, and project name without modifying the scenario itself. The data testing feature was something we intended to use when we implemented Autify.

Hiroshi: We’re able to use it because of Reiko. I doubt things would have gone smoothly without her!

Reiko: We use most Autify features as is. For example, we’ve used the Step Group feature by creating a ‘login Step Group,’ which is applied to the beginning of each scenario.

– Do you run tests at regular intervals? Have you set up Autify in your CI flow?

Reiko: Yes, we run several tests regularly. Depending on the Test Plan, the test frequency is set to either once or twice a week. We run low-priority tests once a week. Higher priority tests are run twice a week on Mondays/Thursdays or Tuesdays/Fridays, for example. In total, there are about 10-15 tests that we run at fixed intervals.

Aside from those periodic executions, development engineers also run regression tests just before release, so we probably run more tests than that.

We haven’t linked Autify with our CI system yet because the configuration needs to be hardcoded again when test content changes.

– It’s impressive that you run 10-15 tests regularly. How do you decide where to create different Test Plans?

Reiko: On crowdfunding platforms, the user journey depends on whether they are making donations or receiving them. So, there’s a Test Plan which runs scenarios that check everything is working fine on the supporters’ side. Another Test Plan checks that the flow is OK for users receiving funds, in other words, the project owner’s side. We also run separate tests at different times and days of the week.

Technical debt is getting paid off, and quality has significantly improved

– How much time does Autify save?

Hiroshi: Now that Autify is embedded into the development cycle, it’s hard to say how much time we would have spent on a given task without Autify… I can’t even imagine where we would be without it. I think it’s given us a tremendous amount of value.

– You mentioned earlier that you could not speed up development because technical debt was increasing and tests took too long. How did implementing Autify change all that?

Hiroshi: It’s hard to quantify how much money we’ve saved Autify in terms of reducing costs. However, it’s safe to say that we can quickly check for issues when we have a new release. We spend significantly less time manually checking things.

– How has the balance between technical debt and net asset changed?

Hiroshi: We are paying back technical debt in a drastic way. When I joined the company, the ratio of technical debt and net worth was around 8:2, but currently, it’s improved to about 5:5.

For example, our entire service was a giant Rails monolith until two years ago. However, the frontend and backend had already been separated. Most of the configuration has changed to one where the frontend, written using Next.js/React/TypeScript, and Rails APIs communicate using APIs.

This has allowed the frontend to develop more flexibly. Even during this major refactoring process, we were able to keep going because Autify enabled us to confirm that none of the existing operations aren’t broken.

– With Autify ready to run automated tests, you are revamping areas that users can’t see.

Hiroshi: Yes, that’s right. That was one of the primary purposes of having Autify. If it weren’t for Autify, we wouldn’t have undertaken such an endeavor! It’s been very beneficial.

– Is the release cycle getting faster?

Hiroshi: We deployed quite frequently, even before Autify. But now, the quality of what we deploy has improved considerably, and we’re able to provide more value more often. I think the value has increased fivefold.

At the same time, each release has significantly more content. So while the frequency hasn’t changed much, I think the value per release is increasing.

Articulating required quality characteristics and creating new value

– What’s next for you?

Reiko: We want to cover more ground with Autify while solidifying the quality of what we already have.

QA tends to be a defensive measure, but I hope to figure out the best way to automate tests for new features and pages that we add.

Reiko: Now that we have a good grip on paying back technical debt, I want to start focusing more on increasing the value of our products and creating new value. Where can we create value? How can we change the architecture so that we can provide value quickly? We are building a system so that the entire engineering team can work on those points.

We are also learning about domain-driven design and incorporating it. The most important thing here is to clarify the core domain, verbalize its domain model, and make an architecture accordingly.

It’s also vital to methodically improve the quality of the value we create. However, we haven’t managed to establish a QA system yet. I want to articulate what quality characteristics our domain needs and improve it through selection and concentration.

Test automation broadens horizons and reveals what you want to achieve

– Do you have any advice for those who are planning to automate testing?

Reiko: I think test automation has a lot of depth to it. And Autify is a breath of fresh air.

One way of going about it is to think Test first or Autify first. In other words, start developing with the assumption that you’re going to use Autify. It could help create something new. I hope to promote test automation along with the QA community.

Reiko: When you try to automate what you do manually, you’ll have to learn a new tool and get used to it. You also have to negotiate the cost with management. But pushing through with manual testing may overburden the person in charge.

If you accept that anything that can be automated will be automated, you can broaden your horizons, look ahead, and think about what you want to achieve.

– Finally, do you have any announcements?

Hiroshi: I mentioned that we are still in the middle of building our QA system. READYFOR is recruiting a QA manager who can analyze quality practices and instill them in the entire organization.

If our latest article or this interview has piqued your interest, please apply. We look forward to hearing from you.

(Interviewer: Autify, Inc., CEO & Co-Founder Ryo Chikazawa)