In this blog post we will test a typical web application that uses a central data store to manage state. The data store communicates with the backend that stores the data long term. You can find patterns in React + Redux applications, Angular + MobX and countless other combinations.
This example looks at a Vue.js web application with Vuex data store. The backend server implements a simple REST API to store, modify and delete data. You can find the source code in cypress-io/cypress-example-recipes repository.
UI testing
Testing the user interface is perhaps what our tool, Cypress.io, does best. The application we are testing is a “TodoMVC” clone. Users can enter text, press “Enter” and add a new incomplete todo to the list. In the screenshot below I annotated the main CSS classes we can use to interact with the application from our end-to-end tests.
The simplest test we can write is the one that loads the application and checks if the application element is visible.
// ui-spec.js
it('loads the app', () => {
cy.visit('/')
cy.get('.todoapp').should('be.visible')
})
The test passes: the page is loading, the element with the selector .todoapp is found and is visible.
Resetting the data
Before each test, it would be nice to set the list of todos to some initial state. For example, our tests can always start with an empty list for simplicity. Because the Cypress Test Runner has full access to the operating system, we could write an NPM command to execute a data reset command and trigger the command before each test:
const resetDatabase = () =>
cy.exec('npm run reset:database')
describe('UI', () => {
beforeEach(resetDatabase)
it('starts with zero items', () => {
cy.visit('/')
cy.get('.todo-list')
.find('li')
.should('have.length', 0)
})
})
In our simple application, the backend is a json-server running against a regular JSON file named data.json. After a couple of items have been added, the file looks like this:
{
"todos": [
{
"title": "first item",
"completed": false,
"id": "4973171049"
},
{
"title": "second item",
"completed": false,
"id": "7205378173"
}
]
}
In this simple case, instead of writing an NPM command we can overwrite this file directly from the test using the cy.writeFile() command.
export const resetDatabase = () => {
// if needed we could have more complex initial state
const data = {
todos: []
}
const str = JSON.stringify(data)
cy.writeFile('./data.json', str)
// short delay gives json-server a chance to reload
// when watching the file
cy.wait(1000)
}
We can even move the cy.visit('/') into its own utility function to run before each test. This way the code is going to be shorter and more maintainable in the long run.
const visit = () => cy.visit('/')
describe('UI', () => {
beforeEach(resetDatabase)
beforeEach(visit)
context('basic features', () => {
it('starts with zero items', () => {
cy.get('.todo-list')
.find('li')
.should('have.length', 0)
})
})
})
The Test Runner shows the test steps executed before and during the test. The arrow below highlights the step during the test when the test runner recorded the XHR request from the web application. The application executes GET /todos to load the initial list of todos from the server. We can just inspect the call by clicking on it in the Command Log on the left; this prints the XHR information in the DevTools console.
update: file watching on CI can be flaky. TravisCI did not restart the json-server a lot of times when we saved the file, causing intermittent test failures. To properly reset json-server I wrote json-server-reset that removed all flake and made the tests much faster. All the test had to do was to request POST /reset.
// cypress/support/utils.js
export const resetDatabase = () => {
console.log('resetDatabase')
cy.request({
method: 'POST',
url: '/reset',
body: {
todos: []
}
})
}
Page functions
One question Cypress users ask frequently is how to use the “page object pattern” in their E2E tests. While you can create custom commands or have an entire hierarchy of page object classes, this is not what we recommend. Remember, the Cypress Test Runner has direct access to the DOM elements in your app, and it is running modern JavaScript right in the browser next to your application. We recommend moving all common code into a utility file. For example, we can move several functions we already have written into a utils.js file; you can use CommonJS or ES6 module syntax.
// utils.js
export const visit = () => cy.visit('/')
export const getTodoApp = () => cy.get('.todoapp')
export const resetDatabase = () => {
// reset database function
}
Notice that in most cases we return the cy... object, because we expect to chain more commands from our utility functions. For example, to get the items in the list, we could write another small utility function and reuse getTodoApp.
// utils.js
export const getTodoApp = () => cy.get('.todoapp')
export const getTodoItems = () => getTodoApp().find('.todo-list').find('li')
// ui-spec.js
import { resetDatabase, visit, getTodoItems } from './utils'
describe('UI', () => {
beforeEach(resetDatabase)
beforeEach(visit)
it('starts with zero items', () => {
getTodoItems()
.should('have.length', 0)
})
})
Using small functions is not only simpler than making up a separate testing “api” of page objects; it is extremely DRY and helps code handle future changes. For example, if the web application has replaced class="todo-list" with the class="all-todos" attribute, our tests are going to break. How fast can we detect the problem? With Cypress, it is clear what went wrong. The Test Runner points out exactly where the problem is with the tests because it shows each find(<selector>) step separately.
When we see a failing find(<selector>) test step, there are two steps to figure out why this failed.
First, look at the previous successful command. The Test Runner found the .todoapp element.
Next, click on the failing step while the DevTools Console is open. It will print the relevant information for the failure. In this case it prints the yielded element fromt he previous get() command. We can expand the element and immediately see that the list element is <ul class="all-todos"> and not .todo-list like our test expected to find. The chained selectors cy.get('.todoapp').find('.todo-list') thus are preferable to compound selector .todoapp .todo-list that would not be as helpful.
Adding todos
Let us add a quick test to confirm that we can add todo items. We will create a utility method to enter a given text so we can use it from multiple tests. Let us add 2 items and confirm both are added to the list.
// utils.js
export const enterTodo = (text = 'example todo') =>
getTodoApp()
.find('.new-todo')
.type(`${text}{enter}`)
// ui-spec.js
it('adds two items', () => {
enterTodo('first item')
enterTodo('second item')
getTodoItems().should('have.length', 2)
})
What if we want to delete a todo? Let us delete the first todo and confirm that the second one remains in the list. We need to add two items, delete the first one, then assert the remaining list has only a single item - the one we have not deleted. I love using the cy.contains(text|regex) method to select an item by text.
it('adds two and deletes first', () => {
enterTodo('first item')
enterTodo('second item')
getTodoItems()
.contains('first item')
.parent() // go up from label to li
.find('.destroy') // then down to ".destroy" element
// because it only becomes visible on hover,
// we disable visibility checking w/ force: true
.click({ force: true })
cy.contains('first item').should('not.exist')
cy.contains('second item').should('exist')
getTodoItems().should('have.length', 1)
})
You can find all these tests in the cypress/integration/ui-spec.js file. If you want to see a complete set of TodoMVC end to end tests, check out cypress-example-todomvc repository or watch our tutorial series building a TodoMVC app in React.
By the way, by changing baseUrl you can point the same tests to any TodoMVC web application to see if it implements all features correctly.
API testing
We have driven the web application completely via its DOM interface. This is the most common and simplest way to test it, and simulates how a typical user would interact with the application.
But we have no idea what is going on behind the scenes. Is the application really sending each new item to the server immediately? Or is it buffering and sending several at once? Maybe the web application is storing the items in localStorage? We have no idea if the application really implements features according to its specs, or if it only has the correct looking user interface.
Let us confirm that the REST backend is called correctly; and we are going to do this in several different ways.
Call server API directly
Cypress is not just an UI test runner - it can easily make HTTP requests also, exercising the server API. I have described this feature in detail in the blog post Add GUI to your E2E API tests. For our simple API we can write just a few tests to add, fetch and delete todo items.
// api-spec.js
describe('via API', () => {
beforeEach(resetDatabase)
// used to create predictable ids
let counter = 1
beforeEach(() => {
counter = 1
})
const addTodo = title =>
cy.request('POST', '/todos', {
title,
completed: false,
id: String(counter++)
})
const fetchTodos = () => cy.request('/todos').its('body')
const deleteTodo = (id) => cy.request('DELETE', `/todos/${id}`)
it('adds todo', () => {
addTodo('first todo')
addTodo('second todo')
fetchTodos().should('have.length', 2)
})
it('adds todo deep', () => {
addTodo('first todo')
addTodo('second todo')
fetchTodos().should('deep.equal', [
{
title: 'first todo',
completed: false,
id: '1'
},
{
title: 'second todo',
completed: false,
id: '2'
}
])
})
it('adds and deletes a todo', () => {
addTodo('first todo') // id "1"
addTodo('second todo') // id "2"
deleteTodo('2')
fetchTodos().should('deep.equal', [
{
title: 'first todo',
completed: false,
id: '1'
}
])
})
})
API tests are useful because they can confirm that the server does the right thing for edge cases that are not easy to trigger through the UI. For example, let us confirm that the server does not crash and burn when we are trying to delete a non-existent item.
it('does not delete non-existent item', () => {
cy
.request({
method: 'DELETE',
url: 'todos/aaa111bbb',
failOnStatusCode: false
})
.its('status')
.should('equal', 404)
})
While these tests certainly work in general, the Cypress UI is kind of boring. There is no web application preview in the Test Runner, only the Comman Log is showing the test steps.
Let us make the tests a little more interesting and a lot more useful.
Stubbing API calls
Cypress comes with built-in spying and stubbing of server XHR calls using cy.server() and cy.route() commands. Let us take advantage of these commands to confirm that the initial list shows todos fetched from the server.
Let us intercept the API call GET /todos we expect the application to perform on startup (we have seen this XHR request reported by the Test Runner). We are going to mock or stub the response with our own data. For example we can return two todo items, but set one to be completed, and assert that the UI really checks it off.
it('initial todos', () => {
// setup XHR interception
cy.server()
cy.route('/todos', [{
title: 'mock first',
completed: false,
id: '1'
}, {
title: 'mock second',
completed: true,
id: '2'
}])
// visit the page
visit()
getTodoItems()
.should('have.length', 2)
.contains('li', 'mock second')
.find('.toggle')
.should('be.checked')
})
We are combining mocking server responses with confirming the right user interface rendering. Let us perform the opposite test - drive the application through the user interface and confirm that the API calls we expect are indeed performed.
Spying on API calls
In the tests below we are going to spy on the API calls, but instead of responding with mock data, we are just going to confirm the arguments to the calls are what we expect them to be.
it('is adding todo item', () => {
cy.server()
cy.route({
method: 'POST',
url: '/todos'
})
.as('postTodo')
// go through the UI
enterTodo('first item') // id "1"
cy.wait('@postTodo')
.its('request.body')
.should('deep.equal', {
title: 'first item',
completed: false,
id: '1'
})
})
it('is deleting a todo item', () => {
cy.server()
cy.route({
method: 'DELETE',
url: '/todos/1' // note exact URL we are expecting
})
.as('deleteTodo')
// go through the UI
enterTodo('first item') // id "1"
getTodoItems()
.first()
.find('.destroy')
.click({ force: true })
cy.wait('@deleteTodo')
})
Note that the cy.wait() command forces Cypress to actually wait for the API call we are spying on to return with a successful HTTP response code, thus confirming that our application is making calls to the server we expect it to make.
You can find all API tests for this TodoMVC application in the file cypress/integration/api-spec.js.
We have tested the user interface, and we have tested API calls by themselves and in combination with the UI. But what about the middle component of the application - the central state Vuex store?
Store testing
Vuex state stores sits right in the middle of our application. The Vue component dispatches actions, updating the state object inside the store. The store forwards updates to the server (via XHR calls we have tested). And the store exposes observable “getters” like “newTodo” and “todos” for the Vue component to render the DOM elements from.
// Vuex store from our application
const store = new Vuex.Store({
state: {
loading: true,
todos: [],
newTodo: ''
},
getters: {
newTodo: state => state.newTodo,
todos: state => state.todos
}
// mutations
// actions
})
Because the Vue component and the Vuex store are staying in sync thanks to the Vue’s reactivity model, our first tests drive the user interface and check the store object.
UI to store
If we control the application via its user interface, we should see the data changes in the store.
Before we write a test, we need to decide how to get to the store reference. To allow testing and controlling the application through the Vue instance I prefer to keep a reference to the component on the window object. In the app.js, set window.app for testability.
const app = new Vue({
store,
el: '.todoapp'
//
})
window.app = app
If you are worried about having publicly accessible window.app variable, expose it only during tests
if (window.Cypress) {
// only available during E2E tests
window.app = app
}
In our tests, we first visit the page, which loads the application, then grab the window.app object reference - this is the same app object set above.
beforeEach(() => {
cy.visit('/')
})
it('loads', () => {
cy.window()
.its('app')
.then(app => ...)
})
From the end-to-end tests, we need to get to the window.app reference after cy.visit() finishes. Because Cypress code runs in the “main” window context, and the web application under test runs in its own iframe, we need to use the cy.window() command to get the application’s window reference. Then we can get the store reference. Here is our first test that confirms the properties of the state object inside the store. And while we are at it, it is easy to check the entire state object.
// cy.its('app.$store') is equivalent to
// cy.its('app').its('$store')
// see https://on.cypress.io/its
const getStore = () => cy.window().its('app.$store')
it('has loading, newTodo and todos properties', () => {
getStore().its('state').should('have.keys', ['loading', 'newTodo', 'todos'])
})
it('starts empty', () => {
getStore().its('state').should('deep.equal', {
loading: true, // initially the store is loading data
todos: [],
newTodo: ''
})
})
When we type into the input element text, the value gets sent to the store on “change” event. Let us test this.
it('can enter new todo text', () => {
const text = 'learn how to test with Cypress.io'
cy.get('.todoapp').find('.new-todo').type(text).trigger('change')
getStore().its('state.newTodo').should('equal', text)
})
Excellent, let us add a few todos via the UI and confirm that the state has been updated.
it('stores todos in the store', () => {
enterTodo('first todo')
enterTodo('second todo')
getStore().its('state.todos').should('deep.equal', [
{
title: 'first todo',
completed: false,
id: 'hmm, what should this be?'
},
{
title: 'second todo',
completed: false,
id: 'we do not know yet'
}
])
})
Hmm, we have a problem. Each Todo item gets assigned a random id, thus failing our deep equality assertion.
We have two solutions; first we will do the simplest one. Just strip the id property from each Todo item before running the assertion.
it('stores todos in the store', () => {
enterTodo('first todo')
enterTodo('second todo')
const removeIds = list => list.map(todo => Cypress._.omit(todo, 'id'))
getStore().its('state.todos').then(removeIds).should('deep.equal', [
{
title: 'first todo',
completed: false
},
{
title: 'second todo',
completed: false
}
])
})
I am using the Lodash.omit method to quickly remove a property from each element in the list. Lodash is bundled with Cypress under Cypress._ by the way, no need to require it separately.
The second way to solve this problem is a little bit more involved. We can look at the ID generator used to assign new ids to the todo items and mock it. In our application, a random id is generated using the following internal function inside the app.js file.
function randomId () {
return Math.random()
.toString()
.substr(2, 10)
}
We cannot overwrite this function from our E2E tests, it is inaccessible from outside code. But we can stub the Math.random method. Again, like the application reference, the Math object used by the application code is attached to the window context inside the application under test, not to the Test Runner’s window. Let us stub it to produce a predictable series of strings for our tests to “predict”.
const stubMathRandom = () => {
// first two digits are disregarded, so our "random" sequence of ids
// should be '1', '2', '3', ...
let counter = 101
cy.window().then(win => {
// inside testing iframe
cy.stub(win.Math, 'random').callsFake(() => counter++)
})
}
it('stores todos in the store (with ids)', () => {
stubMathRandom()
enterTodo('first todo')
enterTodo('second todo')
getStore().its('state.todos').should('deep.equal', [
{
title: 'first todo',
completed: false,
id: '1'
},
{
title: 'second todo',
completed: false,
id: '2'
}
])
})
We are using the cy.stub() method that is implemented using the powerful Sinon.js library. Now we get an increasing series of random ids, which allows us to make assertions over the entire list of todos. Note: Cypress automatically cleans up all mocked methods after each test, so we do not have to restore them manually.
Store to UI and server API
Our tests can dispatch actions to the store like app.$store.dispatch('NEW_TODO', 'new title') and it should trigger both REST calls to the server and DOM updates. Really, try it from the DevTools console - if you can execute a command from the browser’s console, then you can perform the same operation from the Cypress E2E test file. Let us write tests that drive the application via Vuex store actions, while asserting that the UI updates correctly.
We reset the database, load the web application and stub Math.random method like before. To update the store we are going to dispatch actions after getting the store reference. Here is a test that forces the store to create a new todo, then enters text for the next one, but does not add it yet.
describe('Store actions', () => {
const getStore = () => cy.window().its('app.$store')
beforeEach(resetDatabase)
beforeEach(visit)
beforeEach(stubMathRandom)
it('changes the state', () => {
getStore().then(store => {
store.dispatch('setNewTodo', 'a new todo')
store.dispatch('addTodo')
store.dispatch('clearNewTodo')
})
getStore().its('state').should('deep.equal', {
loading: false,
todos: [
{
title: 'a new todo',
completed: false,
id: '1'
}
],
newTodo: ''
})
})
})
Note a small but important detail. Store actions are not synchronous - the action addTodo is executing an Ajax call to the server before updating the internal state object. Here is its code.
// app.js store addTodo action
const addTodo = ({ commit, state }) => {
const todo = {
title: state.newTodo,
completed: false,
id: randomId()
}
axios.post('/todos', todo).then(_ => {
commit('ADD_TODO', todo)
})
}
We are neither returning any promises from the action addTodo, nor waiting for them to resolve in our test code above. We just write an assertion getStore().its('state').should('deep.equal', <expected>)!
Will it fail if the server responds after a delay of a few seconds? How does the test know when the state has been updated? This code works because Cypress retries any assertion for N (default = 4) seconds, see the command timeouts doc. If the state object matches the expected value after 100ms - great, the test completes right after 100ms. If the server responds after 2 seconds, the assertion will keep silently failing for two seconds and then it will pass the test successfully. This intelligent retry mechanism is built-in and the reason why you almost never see wait(ms) commands in the Cypress tests.
When the store changes, the Vue component is updated automatically. Let us test it.
describe('Store actions', () => {
const getStore = () => cy.window().its('app.$store')
beforeEach(resetDatabase)
beforeEach(visit)
beforeEach(stubMathRandom)
it('changes the ui', () => {
getStore().then(store => {
store.dispatch('setNewTodo', 'a new todo')
store.dispatch('addTodo')
store.dispatch('clearNewTodo')
})
// assert UI
// - number of todos in the list
// - text of the first item in the list
getTodoItems().should('have.length', 1).first().contains('a new todo')
})
})
We should also look at the REST calls from the store to confirm the actions are synchronizing the data with the server, while we are at it.
describe('Store actions', () => {
const getStore = () => cy.window().its('app.$store')
beforeEach(resetDatabase)
beforeEach(visit)
beforeEach(stubMathRandom)
it('calls server', () => {
cy.server()
cy.route({
method: 'POST',
url: '/todos'
})
.as('postTodo')
getStore().then(store => {
store.dispatch('setNewTodo', 'a new todo')
store.dispatch('addTodo')
store.dispatch('clearNewTodo')
})
// assert server call
cy.wait('@postTodo').its('request.body').should('deep.equal', {
title: 'a new todo',
completed: false,
id: '1'
})
})
})
These three tests execute the same store commands, but assert their effects on the different components: the store itself, the DOM, and the call to the server. Of course, all our tests use the same logic to get the Vuex store reference and dispatch the same actions - the utility functions could be shared among all tests; make your code as DRY as you feel comfortable with.
Final thoughts
I have shown how to test a typical web application where the front end component is driving the central state store, which communicates with the server via HTTP calls. This application architecture is very common, and the Cypress Test Runner can exercise every part of the app quickly and accurately.
Do you need to write E2E tests for all three parts of your application? Probably not. Remember, tests add a certain drag to your development speed. At least at first, write just a few end-to-end tests that interact with the application through its user interface only. This allows you to figure out the desired UI, but still refactor and change the details hidden inside the implementation.
Later, when the user interface and interaction has solidified, write tests that check how the central store behaves during tests, or which HTTP requests it performs. Such tests will help you avoid introducing regressions when you start adding more features to your application. They also will help you understand how your application works in the future - because each test tells a story about the software components it exercises.
If you want to check out the complete application, see and run the tests yourself, visit cypress-io/cypress-example-recipes. I have used Cypress v1.1.0 to write the test code examples.