Can the test runner be too quick? Can the tests start before the application is ready to handle the test steps? It definitely can happen. This blog post shows a little trick one can use to detect when a web application really starts, so that the test runner never has to wait longer than necessary.
A slow application
I will start with a tiny “web app”. It asks the user to enter a name, and then greets the user with the entered name. Here is the body of the HTML page.
<body>
What's your name? <input id="name" type="text">
<div id="answer"></div>
<script>
document.getElementById('name').addEventListener('change', (e) => {
const answer = document.getElementById('answer')
answer.innerText = `Hello, ${e.target.value}`
})
</script>
</body>
We can write a Cypress test to confirm that the greeting really appears.
it('greets', () => {
cy.visit('app.html')
cy.get('#name').type('Cypress{enter}')
cy.contains('#answer', 'Cypress')
})
The test passes.
Life on the web is full of unpredictable delays, so what if our application takes its time showing the greeting? Let’s add a delay between receiving the user name and showing the greeting.
document.getElementById('name').addEventListener('change', (e) => {
setTimeout(() => {
const answer = document.getElementById('answer')
answer.innerText = `Hello, ${e.target.value}`
}, 2000)
})
The test still passes unchanged - this is because Cypress retries an assertion for 4 seconds by default. Because the expected greeting appears after 2 seconds, the test passes.
But what if there is a delay in attaching the change event listener to the <input id="name"> element? Sometimes the web application does not start right away; maybe it is still loading, or bootstrapping or has not hydrated the HTML. We will simulate the delay between the HTML appearing and the app starting by using a setTimeout in the code fragment.
setTimeout(() => {
// attach "change" event listener after 2 seconds
document.getElementById('name').addEventListener('change', (e) => {
const answer = document.getElementById('answer')
answer.innerText = `Hello, ${e.target.value}`
})
}, 2000)
This time the test fails because the greeting never appears on the page.
The test runner starts typing into the input box before the application has started listening for the change event. Our test runner is too quick.
Constant delay
Usually, to solve this problem, developers just add a delay to their test suite. A few seconds here and there cannot hurt anyone, right? Soon the tests that used to fly barely crawl because of all the delays sprinkled throughout the test code. Back to my test:
it('greets', () => {
cy.visit('app.html')
cy.wait(2500) // wait 2.5 seconds
cy.get('#name').type('Cypress{enter}')
cy.contains('#answer', 'Cypress')
})
Just to be safe, I have added a wait of 2.5 seconds using cy.wait(2500). The test passes.
We can guess the maximum delay our test runner should wait before continuing. Of course, this slows down every test and is unreliable. What if only the very first server start is slow, and the other times the web application start much faster? This constant delay makes everything slow, but without it the tests are flaky. Luckily there is a better way. Cypress runs very close to your code - it controls the browser. Anything you can do in the DevTools, you can do directly from your tests. In our example, let’s not wait longer than necessary - let us detect when the change event listener is attached to the <input> element - that is a good signal for the test runner to start the test.
Spying on the DOM API
Let us temporarily overwrite addEventListener. When the application loads and calls element.addEventListener we will know that it has started and can continue with the test. We can setup this overwrite using the onBeforeLoad option in cy.visit() .
let appHasStarted
function spyOnAddEventListener (win) {
// win = window object in our application
const addListener = win.EventTarget.prototype.addEventListener
win.EventTarget.prototype.addEventListener = function (name) {
if (name === 'change') {
// web app added an event listener to the input box -
// that means the web application has started
appHasStarted = true
// restore the original event listener
win.EventTarget.prototype.addEventListener = addListener
}
return addListener.apply(this, arguments)
}
}
it('greets', () => {
cy.visit('app.html', {
onBeforeLoad: spyOnAddEventListener
})
// more test commands
})
By overwriting the EventTarget.prototype.addEventListener in the application window, Cypress can detect when the application calls document.getElementById('name').addEventListener('change', (e) ...). Then the test sets a variable appHasStarted = true and restores the original addEventListener method. But how do we pause the test runner until the appHasStarted variable has the value true? More JavaScript to the rescue! We can return a Promise that will only resolve when the variable appHasStarted becomes true.
function waitForAppStart() {
// keeps rechecking "appHasStarted" variable
return new Cypress.Promise((resolve, reject) => {
const isReady = () => {
if (appHasStarted) {
return resolve()
}
setTimeout(isReady, 0)
}
isReady()
})
}
it('greets', () => {
cy.visit('app.html', {
onBeforeLoad: spyOnAddEventListener
}).then(waitForAppStart)
// all other assertion will run only when
// the application has started
cy.get('#name').type('Cypress{enter}')
cy.contains('#answer', 'Cypress')
})
A combination of cy.visit() with .then(waitForAppStart) guarantees that the test runner only starts typing into the input element when the application is ready to process the change event. And the test runner is NOT waiting longer than necessary. For example, if our application starts after 500ms, the test runner also executes quickly.
In the above GIF the test runner executes the test commands after detecting the change event listener attachment. Thus the tests never wait longer than necessary, and are also flake-free.
If our application takes longer than the default command timeout in Cypress, we can wait longer for the promise to resolve.
cy.visit('app.html', {
onBeforeLoad: spyOnAddEventListener
}).then({ timeout: 10000 }, waitForAppStart)
See .then() for more examples.
Conclusion
Using a static delay like cy.wait(2500) might seems like a simple and straightforward solution to the slow loading web application problem. But spying on the DOM API is a more robust and correct solution for web applications because it observes what the browser is actually doing. You are not limited to the examples of spying on elements shown above. You can also observe localStorage, sessionStorage and caches objects and anything else in the browser accessible from JavaScript. If there are changes or calls, then the web application has started - and that is when the test can start too.