The future of software testing: This no-code software testing tool delivered 3X faster results and 85% test coverage

Recently, we spoke with Chastain Marchildon who is an automation engineer for Q4 Inc., an investor relations software firm. Q4 works with enterprise companies like Apple, Netflix, Facebook, Amazon, and more. One of their largest software products, Virtual Events, relies on Autify for automated testing. In this case study, learn how our no-code test automation tool helped Q4 cut quality assurance time and costs by one-third.

In the world of software testing, most developers and quality assurance testers have become accustomed to “writing code” in tools like Selenium or Cypress in order to run test scripts. As the software grows and/or changes, maintaining a large number of scripts becomes quite challenging. This increases development time and thus countermands the principles of DevOps- shortening the development life cycle with continuous delivery.

QA team leads are in constant search of an easy-to-use tool that doesn’t require such a high learning curve. This is the dilemma Chastain faced. Furthermore, he was against a fast timeline to implement a test automation plan in less than 3 months and achieve at least 85% test coverage. Learn how Chastain achieved it in our interview with him…

Autify was chosen for its low hurdle and speed of implementation.

– What is the Virtual Events product?

Virtual Events is our virtual shareholder meetings platform. Similar to giant Zoom calls but more secure for financial information.

– What was the need for the product?

Virtual Events has taken off like a rocket in the era of Covid. Our company saw the need to connect shareholders and deliver it quickly and with quality.

– Were there plans to launch something like this before Covid?

No. We had no intention for a virtual platform. However, we felt there was no real competitor to handle large-scale yet secure financial meetings in the way we could.

– How often do you deploy?

Weekly. We want to get to continuous deployment, however, there are a lot of challenges currently.

– How QAs are on your team?

We have six QAs on the events platform (and a total of 15 in the company.)

Test automation that is quick and easy to learn

– What were some challenges you faced before using Autify?

To set the stage… We didn’t have a solid automation workflow. We use 5-6 products and each had a different automation tool.

We used Java, Selenium, Nightwatch, Cypress, and other tools. Our QA team has various skill levels; some coders and some non-coders. We needed something easy to learn, quick to implement, and in such a short time. Wanted 85% test automation coverage initially.

Initially, we started using Cypress for this product. We even hired someone for it but it fell through the cracks. We were one month in and virtually no automation was done.

Autify’s Director of Sales had reached out to our VPoE about this idea of a no-code testing tool. I was tasked with creating a really quick proof of concept. “We wanted something all the QAs can pick up, that we didn’t have to build around, and it would be really quick to implement. It checked all those boxes. That’s how Autify came into play!”

– After Cypress, why didn’t you try another coding automation tool?

Didn’t have the time. We wanted something super quick everyone can use.

So we made the POC (proof of concept) from Autify’s record-and-playback tool. Historically, recommending record-and-playback tools got pushback. The team was skeptical and fair enough they were problematic in the past.

However, we did a POC and it worked out! All QAs can pick it up quickly and be relatively good at it. There was no special training necessary. Nor was developer support needed.

– Was QA convinced of the product?

Yes, everyone was on board. From the VP of Tech all the way down to product leads, developers, and QAs.

Everyone on the team found it easy to implement

– How did you introduce Autify in such a short time crunch?

The biggest hurdle was finding space to get those automation tasks into a sprint. There was a lot of coordination between the product manager, the development lead, the QA manager, and myself. We also brought in more people to cover the extra work while the QA’s got used to the product and Autify. And once they were ready to take on the automation themselves. The workload went back to normal. So the good thing, with Autify there’s such a quick ramp-up time that it was a sprint or two before they were ready to automate things themselves. That was an easy sell!

– How are other code-first projects going? Will you migrate those to Autify in the future?

Currently, we only use Autify on our Virtual Events software but the end goal is to migrate most projects. There are some which are data-intensive projects, they will stay on Cypress, however. Pivoting the future of software testing

– Any tips on how to integrate Autify?

The best tip is to utilize Autify’s support team. They are knowledgeable and provide fast feedback. We honestly received a lot of support from them in developing the POC before even becoming a paying customer.

One of our DevOps guys got creative and rigged it to be continuous integration, too. Meaning, every time they deploy, Autify runs on its own with no problem. This takes it from being not very hands-on to pretty much hands-free. Once I’m done with my scenario, I barely have to touch it anymore. Plus, it’s hooked to Slack, so I peek in now and again.

Continuous integration made it great, especially with off-peak deployment hours, and this made it easy on me.

My other tip is to “use Step Groups, they save a lot of time when used properly. Step Groups are probably one of the most underrated features of the tool from our perspective.”

Previously, we had a lot of QAs manually inputting every single login step- which was a ton of wasted time. Step Groups can solve this repetitive task.

– How are you testing on production?

We conduct a pretty robust Smoke test on production almost comparable to regression testing. We spare no expense in testing in production as it adds peace of mind. Furthermore, Autify makes it easy to move from one scenario to another by pointing it at a different URL.

Autify helped save one-third of the time versus using Cypress or Selenium

– What was the impact of using Autify?

“It really speeds things up an extreme amount. There’s not a lot of bottlenecks that we’ve found with Autify. Our events platform requires multi-tab support in order to simulate user interaction. There’s a workaround in Cypress, but we didn’t have time to figure it out. In Autify, being able to open a new tab was easier. I can’t imagine using anything else for it now.”

The turnaround time of general tickets was streamlined. For example, just one QA can automate an entire feature in a day.

Using Nightwatch or Selenium was pretty antiquated for this product. And maintenance is the biggest issue! With Autify, you can delete a step and add a new one. There’s no Github stuff, it’s just a couple of clicks and you’re done.

There were a few times where the UI changed. With Autify, it gets a review flag and it is accepted. “That’s it… That is the maintenance!”

For example, one time we moved a button inside of a menu, Autify flagged it and the test failed. I added a step and ran it again and it passed. That’s the maintenance… not rewriting code. There are no locating things with XPath anymore- which is my biggest frustration!

“I can’t stress that point enough… the maintenance is minimal [with Autify].”

Test automation in a fraction of the time.

– Approximately how much time was saved using Autify?

Yuji: I’d say for the time saved, or at least the amount of work accomplished in the time that we had was probably three times the amount, maybe more of what we would have been able to accomplish with something like Cypress or Selenium. So because we were pretty hard-locked at three months’ worth of work before our first major release. And we really wanted to get 85% automation coverage for our test suites. We did manage to get that before three months, whereas I did not see that at all even closely possible with Cypress or Selenium or anything like that.

Essentially, Autify helped save one-third of the time versus using Cypress or Selenium.

(Interviewer: Autify, Inc., CEO & Co-Founder Ryo Chikazawa)