Sliding Down the Testing Pyramid

April 2, 2018

By Gleb Bahmutov

A good end-to-end test runner, like Cypress, can be very effective at performing integration tests and even at running unit tests. In this blog post I will show how to write view component tests for a Hyperapp application and how to unit test individual functions. The same approach can work for components written using other frameworks like Vue, React, Cycle, Svelte and others.

End-To-End tests

Recently I watched a screencast video recorded by Alex Barry where he learned Cypress by writing end-to-end tests for a small Todo application implemented using Hyperapp. You can find the full source code here, and a screenshot of the application’s end-to-end tests below.

The E2E tests interact with the page via Cypress’ API commands, here is a typical test:

// list_spec.js
beforeEach(() => {
  cy.visit('http://localhost:8080');
});

it('Can add an item via the button', () => {
  cy.get('input')
    .type('Get eggs');

  cy.get('button')
    .click();

  cy.get('li:first')
    .should('contain', 'Get eggs');
});

Cypress is framework-agnostic and can effectively test pretty much anything that runs in the browser. But, in this case, I wanted to get more bang for my “testing buck”. I wanted to exercise individual view components; for example how do they look given different parameters (maybe even set up some edge cases!) Ordinarily such tests would be called integration tests, a level down below the end-to-end tests in the testing pyramid.

In the above diagram I show my opinion that instead of climbing the pyramid (first writing unit tests, then using browser emulation to write integration tests), one can take an end-to-end test runner (like Cypress) and load a component under test. If we can load an HTML page with an entire application, bootstrapping a single piece of code should be simple! After we mount a single component, we still get the full browser experience and we can interact with the component through DOM events, storage, and network - 100% real world testing, 100% through Cypress’ API and its extensions.

Integration tests

Before we can test a single Hyperapp component we need to extract it into a separate file. Then we can load this component in Cypress and run tests. I moved a component that renders a single “todo item” into its own file, todo-item.js. Notice how much a Hyperapp “view” component looks like a React stateless functional component - it is a pure function that returns a virtual DOM node based on the input arguments.

// todo-item.js
import { h } from 'hyperapp';
const TodoItem = ({ id, value, done, toggle }) => (
  <li
    class={done && "done"}
    onclick={e =>
      toggle({
        value: done,
        id: id
      })
    }
  >
    {value}
  </li>
);

export default TodoItem

I then imported the TodoItem from the main application file:

// index.js
import { h, app } from 'hyperapp';
import { actions } from './actions';
import TodoItem from './todo-item';
import './index.scss';
// the rest of the code

Because of the imports and JSX I needed to setup bundling in order to generate code for the browser. Luckily, Alex Barry has already set up a webpack server to do this. Thus I could set up our test files to use the existing webpack.config.js file through Cypress’ preprocessors API.

// cypress/plugins/index.js
// reuse the application's webpack settings when bundling Cypress tests
const webpackPreprocessor = require('@cypress/webpack-preprocessor')
const webpackOptions = require('../../webpack.config.js')
module.exports = on => {
  on('file:preprocessor', webpackPreprocessor({ webpackOptions }))
}

Finally, in order to load just a single view component I needed some Hyperapp-specific scaffolding code. I have written such code in the cypress-hyperapp-unit-test package. It exports a single function to get the Hyperapp view function running inside the Cypress’ window frame.

import { mount } from 'cypress-hyperapp-unit-test'
// import or code state, action and view
beforeEach(() => {
  mount(state, actions, view)
})
// you get fresh mini-app running in each test

Let us write a couple of integration tests to confirm the “TodoItem” works as expected:

// cypress/integration/todo_item_spec.js
import { h } from 'hyperapp'
import { mount } from 'cypress-hyperapp-unit-test'
import TodoItem from '../../src/todo-item'

describe('todo item', () => {
  it('shows an item', () => {
    const view = (state, actions) => h(TodoItem, {
      id: '1',
      value: 'test item',
      done: false
    })
    mount(null, null, view)
    cy.contains('test item')
  })

  it('marks done items', () => {
    const view = (state, actions) => h(TodoItem, {
      id: '1',
      value: 'test item',
      done: true
    })
    mount(null, null, view)
    cy.contains('test item').should('have.class', 'done')
  })

  it('calls toggle on click', () => {
    const toggle = cy.spy().as('toggle')
    const view = (state, actions) => h(TodoItem, {
      id: '1',
      value: 'test item',
      done: false,
      toggle
    })
    mount(null, null, view)
    cy.contains('test item').click()
    cy.get('@toggle').should('be.calledOnce')
  })
})

In the above tests we work with the component as if it were a full blown application.

  • We can check how it renders into the DOM with cy.contains('test item')
  • We can look at the styling it declares with cy.contains('test item').should('have.class', 'done')
  • We can interact with the DOM with cy.contains('test item').click() and confirm that the component calls action functions with cy.get('@toggle').should('be.calledOnce')

It is not only the real browser that makes our integration tests so useful, but the Cypress GUI. For example, the last test confirms that when the user clicks on a TodoItem label, it executes the toggle function. Since we sent a spy to the view function, we can inspect the arguments passed by the “todo item” code back to the toggle callback!

In a sense, with this approach you are getting a Storybook.js-like experience while also testing your code.

Unit tests

Finally, Hyperapp is big on pure functions. For example, all actions triggered by the events are pure functions that get the state and parameters as arguments - there is absolutely no chance to see the this keyword used in a typical Hyperapp application. Here is the add action code for example.

// actions.js
export const actions = {
  add: () => state => {
    if (state.input === '') return;

    return {
      input: "",
      todos: state.todos.concat({
        done: false,
        value: state.input,
        id: state.todos.length + 1,
      }),
    };
  },
  // ...
  // toggle, input, filter action functions
};

Can Cypress move down one more level and test the individual functions that do not even output anything into the DOM?

Absolutely! Here is an example of a unit test that exercises the add function. The test calls the function with arguments and checks the result against an expected value. There is no need to load any additional utilities to scaffold the framework and mount a component - after all, this is just plain JavaScript.

// cypress/integration/actions_spec.js
import {actions} from '../../src/actions'

describe('actions', () => {
  context('add', () => {
    it('adds an item', () => {
      const state = {
        input: 'foo',
        todos: []
      }
      const result = actions.add()(state)
      expect(result).to.deep.equal({
        input: '',
        todos: [{
          done: false,
          id: 1,
          value: 'foo'
        }]
      })
    })
  })
})

Even in this test, the Cypress GUI comes in handy. For example we can inspect the values being compared.

Final thoughts

You can find the above integration and unit tests in this pull request, already merged into the main repo. Besides cypress-hyperapp-unit-test, I have written adaptors for other popular frameworks. Check out:

If anyone can help me finish scaffolding Angular components, I would be extremely grateful. The initial code is in cypress-angular-unit-test.

Finally, in February our team went down to Texas to speak at AssertJS. Brian Mann talked about Cypress best practices and I spoke about moving up and down the testing pyramid. You might find these videos and my slides to be a good complement to this blog post.