Why you shouldn’t use ids in E2E testing

January 10, 2020

Hello. My name is Takuya Suemura (@tsueeemura), and I am a SET (Software Engineer in Test) for Autify.

Are you performing E2E testing? It used to be the case that Selenium was the only game in town, but now we have many frameworks like Puppeteer, Cypress, and TestCafe — there are so many choices, it almost makes it difficult to choose!

Regardless of what framework you decide to use, there are certain key factors to keep in mind when writing any form of E2E test.

The first is locators. These are keys used to target specific elements needed for manipulating or validating the page. Both CSS selectors and XPath can be used for locators. Generally speaking, you use attributes like id, class, and name.

Today I’d like to talk a bit more about locators.

Locators explained

Provided you can uniquely target an element, the locator can be anything. However, in terms of maintainability, it should satisfy the conditions below.

Must always be unique
Should be unlikely to change

For example, in the below code snippet, an element is specified using a class called btn-primary.

<button class="btn-primary">Submit</button

However, given that the class is closely related to the styling of the button, there may be multiple instances on the page, and it is highly likely to change in the future. Therefore, using id is generally considered to be a best practice instead of using other attributes such as class.

driver.getElementsByclassName("btn-primary") // target the element with the class btn-primaryCOPY

driver.getElementsByclassName("btn-primary") // target the element with the class btn-primary

<!-- Added the submit id -->
<button id="submit" class="btn-primary">Submit</button> <!--Change to targeting by id instead of class-->
driver.getElementByid("submit")

Why not to use id

That being said, when developing complex applications like those seen today, referencing an id from test code is problematic.

One reason for this is that it makes modifying your application more difficult when you are changing the id. id is ordinarily treated as internal values, so referencing them through external code, such as test code, makes the production code less maintainable.

For the sake of argument, suppose that an update to the JavaScript framework assigns a specific prefix to all id and these are now controlled by the framework, causing them to be changed each time you build.

When these destructive changes occur, E2E tests are used to verify that the underlying behavior remains the same. However, if you depend on id for your locators, even if the underlying behavior has not changed, you will need to refactor all of your locator definitions for the test code to remain compliant.

There are also issues uniquely associated with E2E testing, such as slow processing speed and limited times when tests can be run. In other words, if a developer refactors their code and changes id, it only becomes clear to what extent this affects the test code and application as a whole when that phase of development has concluded and the code has been deployed, and then the app is launched in a web browser and run through E2E testing. This makes it difficult to refactor code in situ when an id changes.

(As an aside, I believe one of the reasons E2E tests are criticized as prone to become obsolete is for this reason above.)

Focusing on meaning and behavior

If we assume that id is not immutable, but rather prone to change by the vagaries of the developer’s needs, just what should be used as a suitable alternative for E2E testing? Let’s take a look at some of the remarks I made in the previous section.

When these destructive changes occur, E2E tests are used to verify that the underlying behavior remains the same. However, if you depend on id for your locators, even if the underlying behavior has not changed, you will need to refactor all of your locator definitions in the test code to remain compliant.

In other words, in the context of E2E testing, behavior is, in essence, your definition — the test code should adapt precisely when the underlying behavioral changes. Therefore, rather than depending on internal elements like id or class, the most natural approach is to locate content based on the meaning of the elements and behaviors therein.

Locating by text

One method of locating content by its meaning or behavior is to focus on the text content.

Did you know there is an article in the Cypress documentation introducing best practices in selecting elements? Best Practices – Selecting Elements This article describes how you can use text content instead of id and class.

Don’t target elements based on CSS attributes such as: id, class, tag
Don’t target elements that may change their textContent
Add data-* attributes to make it easier to target elements.

The use of data-* attributes is recommended (`data-` is discussed in a later section), rather than text content, but there is also the line below:

If the content of the element changed would you want the test to fail?

If the answer is yes: then use cy.contains()
If the answer is no: then use a data attribute.

cy.contains() is a Cypress method whereby you can locate elements if they contain a certain string of text. Selenium also exposes a Find Element By Link Text method, but it only targets text within tags, while cy.contains() targets all textContent.

We can use cy.contains() to locate text content within the code snippet from before, searching for the string Submit:

<button type="submit" data-qa="submit">Submit</button>
cy.contains('Submit')

One benefit of using text content as a locator is that, as mentioned above, the test will fail if the text within the element changes. Put differently, this lets you cover external element behavior. What this means is that you can verify whether the submit button contains an illegal string such as null simply by looking for the text, rather than having to assert like assert(button.text === ‘Submit’).

Another benefit of using text content for your locators is that it is decoupled from internal element behavior. What developers look for in E2E layer tests is verifying that changes to the production code do not affect the application’s functionality as a whole. Therefore, strictly speaking, you should not use internal values for elements like, id.

Adopting text content for your locators means that even wholesale changes like the JavaScript framework being replaced would (theoretically speaking) not necessitate refactoring the test code, provided the flow of the application does not change.

Maintain element uniqueness

While using text content as a means of locating is highly effective, one downside is that ensuring the uniqueness of elements is difficult. For example, suppose we have a UI where after clicking the submit button, a confirmation dialog appears.

<button type="button" class="btn btn-primary" data-toggle="modal" data-target="#exampleModal">  Submit</button>
<div class="modal" tabindex="-1" role="dialog">  
	<div class="modal-dialog" role="document">    
    	<div class="modal-content">      
        	<div class="modal-header">        
            	<h5 class="modal-title">Confirm</h5>        
                	<button type="button" class="close" data-dismiss="modal" aria-label="Close">          <span aria-hidden="true">&times;</span>        
                    </button>      
                    	</div>      
     <div class="modal-body">        
     	<p>Really submit?</p>     
        	</div>      
        <div class="modal-footer">        
        	<button type="button" class="btn btn-primary">Submit</button>        
            <button type="button" class="btn btn-secondary" data-dismiss="modal">Cancel</button>      
            	</div>   
                	</div> 
                    	</div>
                        	</div>

In this case, there are two Submit buttons within the DOM, so you must clarify which button is being clicked. To restrict the test to the Submit button within the modal, you would locate the modal and then the button.

Cypress contains syntax for handling these patterns using the within method. Suppose that we are looking for a submit button within the modal tags.

cy.get('.modal').within(() => {  
cy.contains('Submit’).click()})

Using cy.get(‘.modal’),/span> restricts our query to within elements containing the modal class, and then we use cy.contains(‘Submit’) to search for the submit button. This syntax ensures that we get the right hit even if there are multiple matching strings, as well as ensures high readability by describing the action in the same way a human would find the button: clicking the submit button within the confirmation modal.

data-* attributes

Incidentally, do you recall the mention of data-* attributes in the Cypress document earlier?

Don’t target elements based on CSS attributes such as: id, class, tag
Don’t target elements that may change their textContent
Add data-* attributes to make it easier to target elements

This describes how you should use the data-* attribute for locators, instead of CSS-native attributes like class and tag or textContent within elements that may change.

The data-* attribute, broadly speaking, allows developers to create proprietary attributes within tags. Suppose we define a custom attribute called data-qa within an element that must be tested.

<input type="text" name="first_name" data-qa="first_name">
<input type="text" name="last_name" data-qa="last_name">
<button type="submit" data-qa="submit">Submit</button>

We can then use the locator below to find the element.

// cy.get() is a method used to find an element by CSS selector

.cy.get('[data-qa=first_name]')
cy.get('[data-qa=last_name]')cy.get('[data-qa=submit]')

Even if the naming convention for the name attribute were refactored in the production code, moving from the use of snake case (first_name) to camelCase (firstName), as long as the data-qa attribute is not changed, the test code can continue being run as-is.

Issues with using data-* attributes

While this method may at first glance seem perfect, it greatly increases the cost of maintaining and managing the code. The biggest downside to using the data-* attribute is that these custom attributes must be maintained in the production code.

For example, the developer must take pains to ensure that the data-qa attributes onscreen always retain unique values. In the next example, we create a new form called input example, and the duplicate values for first_name and other entries cause the test to fail.

<!-- Existing test fails when a new form called “Input example” is created -->
<span>Input example:</span>
<input data-qa="first_name" disabled value="Takuya">
<input data-qa="last_name" disabled value="Suemura">
<input type="text" name="first_name" data-qa="first_name">
<input type="text" name="last_name" data-qa="last_name">
<button type="submit" data-qa="submit">Submit</button>

The data-* attribute is effective insofar as it functions as a substitute to id that does not affect upkeep of production code, but this means developers have to keep track of both id</span. and data-* attributes, making it more difficult to maintain compatibility between the application and your test code.

When to use the data-* attribute

As described above, I am personally opposed to using the data-* attribute as you would id, as it has steep upkeep costs, but it is effective when used in a supplementary capacity, such as adding a meaningful attribute to a UI component.

Suppose we have the following sample code used to maintain uniqueness when locating text content.

cy.get('.modal').within(() => {  cy.contains('Submit').click()})

As you can see, this code is inadvertently referencing a class called .modal. Whether or not the modal has a class called .modal is not particularly meaningful in terms of the behavior of these elements, so this syntax has low maintainability and may necessitate refactoring the test code.

Therefore, assigning a data-* attribute to these larger components increases the maintainability of the test code without adding steep management costs to your production code.

<button type="button" class="btn btn-primary" data-toggle="modal" data-target="#exampleModal">  Submit</button>
<div class="modal" tabindex="-1" role="dialog" data-qa="modal">  
	<div class="modal-dialog" role="document">    
    	<div class="modal-content">      
        	<div class="modal-header">       
            	<h5 class="modal-title">Confirm</h5>        
                	<button type="button" class="close" data-dismiss="modal" aria-label="Close">          		<span aria-hidden="true">&times;</span>        
                    </button>     
                    	</div>      
           <div class="modal-body">        
           	<p>Really submit?</p>      
           </div>      
    <div class="modal-footer">        
    	<button type="button" class="btn btn-primary">Submit</button>        
        <button type="button" class="btn btn-secondary" data-dismiss="modal">Cancel</button>      
    </div>    
    	</div>
        	</div>
            	</div>

You can update the test code as follows.

cy.get('[data-qa=modal]').within(() => {  cy.contains('Submit').click()})

It goes without saying, but unless new data-* attributes are shared within your team, they may be inadvertently deleted when refactoring, so this has to be properly documented.

Conclusion

Thus far, we have covered the following points:

What should be immutable in an application is not its id or class, but its underlying behavior.
When the behavior changes, the E2E test must detect this.
E2E tests should not lock anything other than external behavior.

Given the above, what I consider to be the best-locating strategy is as follows. It is similar to the Best Practices espoused by Cypress, but it is more text-centric.

As a rule, locate using elements’ text content
*The `data-` attribute should be used on a limited basis for the abstraction of UI components, etc.

Using this strategy brings the following advantages:

It does not employ internal id, class, or name values, ensuring maintainability of the production code
Compared to the heavy use of the data-* attribute, it has fewer production code maintenance costs
No need for additional assertions for text content changes, making the test more robust to changes in the app’s behavior

Addendum 1: What testing framework should you use?

Given that you can use XPath to locate elements by text content, theoretically speaking, it can be implemented using any testing framework. However, I recommend using a solution that explicitly supports text-based locating, such as Cypress.

I strongly recommend CodeceptJS. That is because of the declarative and understandable syntax of Locator Builder and that using Semantic Locator make it easy to write tests targeting text content alone, provided the site uses a standard DOM configuration.

// Locator Builder example
// Simple notation, avoiding the complexities of targeting CSS selectors and XPath addresseslocate('a')  .withAttr({ href: '#' })  .inside(locate('label').withText('Hello'));
// Semantic Locator example
// Any site with a standard DOM layout can be manipulated by targeting text content//
// Click the "Sign in" button// Enter “tsuemura” in the "Username" fieldI.click('Sign in');I.fillField('Username', 'tsuemura');

CodeceptJS also exposes the aforementioned within method, making it ideal for use in the methodology discussed in this article. I encourage you to give it a try.

Addendum 2: Locators in Autify

Autify provides a Record & Playback style solution like the Selenium IDE, so the user does not write locators directly, but it does employ some of the methodologies discussed in this article.

When recording a test scenario in Autify, you poll the various attributes held by an element. In addition to id, class, name, and tag metadata, this also polls text content and coordinates — i.e., information seen by the end-user. When running the test code, it measures to what extent the onscreen elements correspond to these attributes and then grabs the best match.

This methodology is generally used for automatic correction (“self-healing”) of locators by AI, but one byproduct of this is ensuring maintainability of the production code. This goes beyond the text content and data-* attribute location methods discussed in this article in that it does not lock your production code to a specific format, both in terms of id and class and in terms of text content or other locators.

Autify is generally described as a no-code solution and is considered a product geared at non-engineers, but it also neatly handles problems associated with traditional forms of test automation and seeks to contribute to greater productivity of development projects worldwide.

// Locator Builder example
// Simple notation, avoiding the complexities of targeting CSS selectors and XPath 
	addresseslocate('a')  
    .withAttr({ href: '#' })  
    .inside(locate('label').withText('Hello'));
// Semantic Locator example
// Any site with a standard DOM layout can be manipulated by targeting text content//
// Click the "Sign in" button
// Enter “tsuemura” in the "Username" fieldI.click('Sign in');I.fillField('Username', 'tsuemura');