This post is the second of a series that explores how Component Testing can benefit different parts of your team or organization. If you’re wondering why you and your team should adopt Component Testing, this series is for you!
In our first post of this series, we discussed how Cypress Component Testing is
beneficial for developers. In this post, we'll delve into how Component Testing can
improve collaboration and alignment between development and QA
teams by allowing them to run end-to-end and component tests under the same
API, in the same user-friendly environment. This helps when "shifting testing left"
in the software development lifecycle, because QA professionals can more easily
get value from component tests written during development, and contribute to
those tests themselves if needed.
This post is broken up into sections discussing various aspects of how this
collaboration can be achieved. The sections are below, feel free to skip to the most
interesting parts for you:
QA Processes and Cypress—some background on why Cypress is used by both QA and Engineering teams already, and is a good place for them to collaborate.
What changed with Cypress Component Testing?—effects of component testing in a real browser that may be applicable in QA contexts.
We then expand on three ideas related to this change:
- Improved readability—how this kind of test is less cluttered and more approachable when working with test files themselves.
- Testing flexibility—how to combine component and end-to-end tests, so that each can focus on what it is best at.
- Component tests as reference—since the test code is the same as an end-to-end test, strategies for locating elements and testing the DOM can be lifted directly from component tests, extracted into helper functions, or just used as inspiration, depending on your needs.
Quality Assurance and Cypress
Every organization has its own ideas and processes around quality – how to measure it, how to manage it, and how to be confident that applications are working as expected. We depend on a range of tools and techniques for this. Cypress is often just one of many factors in an overall approach to quality assurance.
An overall approach to quality can include manual testing, exploratory testing, unit tests, API tests, component tests, end-to-end tests, and much more. The applications and systems tested can vary greatly, ranging from websites to mobile applications, embedded systems, and physical devices. This means there are all different kinds of automated testing, and tests are written by people at different moments in the development lifecycle, for different reasons, often on teams that aren’t directly talking to each other.
Though Cypress is capable of running many kinds of tests, it is primarily known as an end-to-end test runner for websites and web applications. As such, Cypress is a tool that is adopted by both application developers and quality engineers, and can be a great place for collaboration between those two worlds.
As part of the “shift left” movement in testing, developers have been encouraged to write and run end-to-end tests as they work, rather than simply throwing them “over the fence” to other teams. This improves feedback time and helps uncover defects before code is merged, preventing them from reaching customers.
Quality automation engineers have also sometimes become embedded in development teams, working to add testing earlier, discuss it as part of the development process, and perform early manual and exploratory testing of work in progress.
The foundation of this collaboration is usually the idea of testing a “build” of the application, either through the UI or with API tests. In that context, the individual components of a web application may still have tests, but those tests are essentially unit tests or integration tests of small parts of the system and are considered “developer-facing”. They typically run in a simulated, headless execution environment in the terminal, not a browser.
It might be useful for automation engineers to write low-level tests in some situations (I saw a wonderful talk on this by Annie Gardner at the Atlanta Test Automation Summit this year), but it’s not a common practice. Beyond organizational changes, it involves teams learning a different testing paradigm, and often more than one, to work in this new context.
What changed with Cypress Component Testing?
Cypress brings component tests into the browser, and into the Cypress testing environment that quality professionals are already familiar with. This simple change has some important effects:
- Testing in real, specific browsers instead of a node-based browser simulation removes a whole category of problems and edge cases that our tests previously had to work around. This allows component tests to focus more on user interactions. Tests interact with the component “through the front door” and can avoid triggering framework events and internal component methods except in unusual cases.
- The tests themselves become inherently more valuable as they use the same environment as our users. They can catch more real issues, and have fewer false positives.
- Testing in Cypress means that all the familiar Cypress commands for locating elements, interacting with them, and asserting the outcome are available. Even things like network stubbing with
cy.intercept()work the same way in both testing types.
- Because of this, if you can read and write a Cypress end-to-end test, you can read and write a Cypress component test.
- The Cypress testing UI with time-travel debugging features is available for component tests, capturing each step of the test so we can see exactly what happened and when.
- When appropriate, code can be directly shared between component and end-to-end tests. This is incredibly useful for complex components with multi-step test workflows and assertions that can be broken out into helper functions.
Taken together, these things mean that component testing becomes something that’s easy and natural for quality engineers to participate in. If the goal of “shifting left” is to get feedback earlier in the development lifecycle, component tests offer one of the fastest possible ways to deliver that feedback, since they run quickly and developers often have them open while working on a particular component.
In the past, component tests could be hard to read due to framework-specific workarounds, the natural differences between testing frameworks, and the common approach of unit-testing internal values and methods instead of testing their user-facing effects.
When we improve the readability of component testing and invite more participation, we have more options in deciding what to cover and where. Component tests are perfect for thoroughly checking every possible state of a component and ensuring it behaves correctly for users. It’s cheap to set up the state, and tests run more quickly, because we don’t have to run our entire application and create the exact conditions needed to set up a particular state. Often we can do everything with the component's own props, or by mocking a small number of dependencies.
Readability saves time and makes component testing more approachable, reducing resistance and pain points some developers feel towards writing tests at all.
End-to-end tests focus on complete user journeys that might involve multiple systems working together. These journeys take the user through various steps and pages of the application. It’s crucial to test that users can reach the correct end state and perform their goals. These tests can become cluttered, unfocused, and repetitive if they are also focused on fully checking the correctness of every part of every page. They also run more slowly and are typically run in CI, not when modifying a component, meaning it can be minutes or hours after making a change that developers push commits, trigger a CI run, and get test results.
In this way, we can describe end-to-end tests as “horizontal”. They move through many parts of the system and are concerned with successfully reaching key points along the way. Component tests, on the other hand, are more “vertical” in that they are concerned with the correctness of specific parts of the app.
This means we can choose the correct testing type based on our needs. If a behavior is owned by a certain component, and forms part of its “contract” with the rest of the system, that behavior should be specified in the component tests. This means there is a single source of truth about the component’s responsibilities, so developers working on the component get instant feedback if their code changes have broken the contract without having to search for the right tests.
When a defect reaches production, we can think about where to add a regression test for that defect. Certainly, an end-to-end test is useful and provides confidence. But we can sometimes test closer to the source of the problem. If the defect was fixed by a logic or HTML update to a component, that component’s own test should cover the behavior and should fail quickly if the defect reappears. If the defect isn’t traceable to a bug in a component, but to something else, like a value coming from the API being incorrect in a field, then an end-to-end test or API contract test makes the most sense.
We want to place the regression test as close as possible to the code that introduced the issue. We might also want to test at the end-to-end level if there are other risk factors for the same issue or if it forms part of a critical user flow.
Component tests as reference for other tests
Component tests can focus heavily on the specific elements rendered by a component. They are perfect for checking things like whether proper interactive buttons and links are used, and if they have the correct accessible labels and keyboard behavior. These are things that a front-end developer is concerned with when writing the component, and it makes sense for the component test to specify the accessible behavior as much as possible. This means that, in almost every case, an element we might want to use or assert about in an end-to-end test will already have a locator written in a component test. So when a locator is troublesome to find, we have a reference point with existing Cypress code we can directly copy to get a starting point for locating that element. This is especially useful when cleaning up element locators created by test recording tools that might prioritize uniqueness over stability.
An example of a locator generated by a test recorder could be:
cy.get('#headlessui-popover-button-4 > .flex > .font-medium')
While the component test might locate the element in a more stable way that reflects the qualities that matter most:
Difficulty finding locators for end-to-end tests might point to specific behavior missing from a component. If a clickable element that we need to reach for an end-to-end test is an SVG image, and we have to write something like
cy.get(‘#header > svg’).eq(2) in order to locate it from an end-to-end test, that represents a real accessibility problem in the application.
To be usable by people with certain disabilities, clickable SVG icons should be wrapped in something like a
link element and have clear label text for screen readers. These are things that are easy to test at the component level, and so even when a problem is found in end-to-end testing, it might point to an improvement that can be made in the component itself, and the corresponding test. This would in turn create a sensible locator for the same element in other contexts, and reduce the overall presence of delicate, unmaintainable, x-path style locators.
The addition of Cypress Component Testing to our setup allows us to “test what matters, where it matters”. We can consider component tests as an essential, top-level part of our approach to front-end testing. Component tests offer precision in their results combined with completeness documenting the responsibilities of a component in relation to the rest of the system.
Cypress component tests are a useful development tool, making it easier to develop components in isolation. But by moving the tests into a real browser, they also increase the overall confidence gained from running component tests against a codebase. They make the test code easier to follow, and less dependent on triggering framework-specific events and working around the limitations of simulated browser environments.
In the broader sense, Cypress Component Testing opens up the world of component testing to more people in the organization, who may come at quality from different perspectives, and allows them to work that much closer to the development lifecycle.
Stay tuned to our blog for our next post in this series about how managers can benefit from Component Testing. As always, if this feature is helpful or if you have other ideas or feedback for our team, let us know on Github.