Talent Palette is a talent management system that allows users to manage employee information, talent training, skill, evaluation, and recruitment - all in one place. As a means for data-driven talent utilization and for HR digitization, it has been implemented by over 3,000 organizations as a talent management system that maximizes employee performance. (1,380 organizations are signed up to the service as of September 30, 2023)
Since the system handles data that is critical for companies, a high level of QA (Quality Assurance) is a must.
In the service’s development and QA processes, the company saw the benefits of implementing Autify not only in quality assurance and time reduction but also in improving the Quality Control team’s development literacy.
We visited Plus Alpha Consulting Co., Ltd., which provides Talent Palette, and spoke with Mr. Morita, Group Manager of the New Innovation Quality Control Group and Mr. Meguro, Engineer of the same group.
QA team was created to reduce incidents to near zero
Morita: I’m the manager of Talent Palette’s Quality Control (QC) Group. I also help out with QC for Plus Alpha Consulting services as a whole.
Meguro: I’m a member of Talent Palette’s QC Group, responsible for using and promoting Autify.
Morita: Within our company, we have about seven members who are in charge of QC for Talent Palette and other services. We do agile development, and testing is done on a task-by-task basis.
Morita: We didn’t have a dedicated QC team before. Our development used to prioritize speed, but in order to focus on quality as well, we officially created this team about two years ago.
Currently, our mission is to build a testing system and to reduce the number of incidents to almost zero. Another goal is to expand the testing expertise we’ve gained from Talent Palette, which has been our main focus, to other products.
Morita: We didn’t have a specific problem, but it was the nature of the service that supports human resources, recruitment, and labor management. The service has to work in order for our clients to be able to perform their daily tasks. That’s why high quality is a requirement.
As the company grew in size, so did the size of our clients. The need for QC grew, and that's when we set up the team.
Tackling the growing amount of regression
Morita: The issue was the excessive workload for impact analysis - testing the effect on features other than what was implemented. We have a somewhat unique style of agile development. Each user story to be implemented is assigned to one developer, and that developer implements the feature on both the front-end and back-end. It was also up to that developer to determine the scope of testing. This made it difficult for third parties, such as reviewers and other members of the QA team, to know the scope of impact and which parts the developer tested. Checking how features other than those implemented was challenging.
In terms of product development, in order to prevent bugs from slipping through the cracks, the QA team had to review the results of tests performed by developers and manually test areas not covered by those tests. Again, however, time was an issue. We therefore decided that we needed to improve the QA team’s efficiency and automate certain tasks so that the team would have the resources to quickly implement features we wanted our customers to use. That is when we started considering test automation.
Accessibility was the major deciding factor for choosing Autify
Morita: The first thing on our list of priorities was accessibility - whether everyone on the QA team could use it. We had a large number of test scenarios that we wanted to automate, and to cover them all, we needed a platform that could be used by a large number of people.
We were looking for something that could capture UI actions and make them available for use in test scenarios. At the time, Autify was the only service that met that requirement, and that’s why we went with it. Autify is still evolving rapidly, and it’s only gotten even more versatile with new feature releases, such as the self-healing feature where the AI fixes scenarios.
Morita: At that time, we had a lot of testers with various backgrounds in terms of hardware/software types, but there were only a few who had engineering experience.
Autify significantly reduced man-hours, increased development knowledge, and prevented bug recurrence
Morita: The total number of scenarios is now close to 1,000, and it’s increasing considerably. We use Autify at every version upgrade and have been able to find issues that need fixing. Autify has paid off in the sense that we now have a testing system in place. We want to continue expanding coverage to include more scenarios that verify in-depth actions that are even closer to user scenarios.
Meguro: I’d like to share the quantitative and qualitative results we’ve achieved with Autify.
Let's start with the quantitative results. For example, regression tests for on-screen actions take 3-5 minutes per screen when done manually, and we have over 100 of these tests. We automated them with Autify, and based on a simple calculation, we’ve saved about 50-83 hours (man-hours) per month.
In addition, when reviewing test results, screenshots are automatically captured in the Autify UI, making it easy to see what errors are occurring. Also considering the verification process for for every update, I really feel that we’ve been able to save a lot of man-hours.
One of the qualitative results is that when a system issue is detected, we can easily create scenarios as a preventative measure, and it has actually prevented recurrences.
Another is that I can see that QA members, who aren’t developers, are becoming more knowledgeable about development. In some cases, they use Autify's JS steps and CSS selectors to create scenarios.
I'm an active developer, and when I share some of my knowledge with testers, they refer to Autify’s official snippets and documentation, builds a scenario, and ask me if I think it’ll work. The documentation is easy to understand and covers a wide range of topics. Team members are gaining knowledge and are able to create slightly complicated test scenarios without me having to give detailed instructions. Thanks to Autify, members are able to gather development knowledge proactively.
Managing and sharing QA knowledge and using Autify for testing other products
Meguro: Currently, we are focusing on Talent Palette in terms of knowledge management. For each case, we compile information in spreadsheets and Notion. The QA team asks me when they have any questions, and I add the insight to the document.
Morita: A member who used Autify for Talent Palette is now in charge of QA for Yorisoar (management service for educational institutions), and the expertise is being utilized. Our company has few barriers between services, so we can ask other teams when we run into any problems, which makes horizontal development easier.
The company values the First Penguin mindset - their office is filled with penguin items!
Morita: For Talent Palette, the QA team’s mission is to reduce incidents. We want to focus on that and use Autify more and more. Recently, we’ve been making some tiny UI changes at the pixel level, so we’ve started to work on this issue using the visual regression feature.
Visualizing results to drive internal use. What the team expects from Autify
Meguro: There are many things! First, it would be great if it supported batch actions for scenarios, such as “add label” and “delete”. We have a growing number of scenarios and batch actions would make it easier to manage them.
Second, the visual regression feature could be even more powerful. It’s already a great feature and we’ve been using it lately, but the Talent Palette UI changes frequently so I’d love to see it strengthened even more.
The third is reports. I use them to see how many scenarios we have and to check test values. It would be even better if the reports could be narrowed down by scenario pass rate for each test plan and test coverage percentage using labels. It would also be nice if it could show graphs to give me an overview.
If there was a graph to show how the pass rate, failure rate, and coverage have changed over time, we could share the dashboard internally. And since it would show results visually, it would help promote the use of Autify within the company.
Message for those who want to start QA: comprehensive support and community are Autify’s strengths
Morita: Autify can be used by basically any team. Even companies without a dedicated QA team and non-developers can use it.
I think a lot of companies separate their development and QA teams for faster development. The advantage of Autify is that the QA team can build scenarios from the user's perspective.
Autify’s standout point is its robust support and that they’re putting a lot of effort into building community. QA is a job that tends to be quite isolating, even within the company, but the team needs to have a strong presence working with engineers from a position that’s closer to the users. That’s why I think it’s important to have a community where QA members can ask each other how they work and learn more about using Autify. I would recommend Autify to anyone who feels a bit disconnected in the office.
Meguro: As Morita mentioned, I would recommend this tool to people who want to get into QA. Anyone can use it even if they are not developers. If they are developers, they can make use of JS Steps and other helpful features to get into the nitty-gritty.
Many people won’t know where to start at first. My suggestion is to start with the bare minimum - test the parts that are absolutely critical. Share stories about how you’re successfully doing QA with Autify with people in your company; it will help spread its use horizontally. The ease of use is exceptional, so if you’re just starting out with QA, I would recommend Autify.
.jpg)
.jpg)
