Looking for Early Access? Sign up here.
Cypress broke new ground in testing by introducing an app that visually displays your tests running on your application in real-time. Since then, we’ve become one of the largest testing frameworks globally, enabling thousands of organizations to confidently deploy their websites with robust testing. However, over the years, we've identified another critical challenge: knowing what is and isn’t tested. Today, we’re excited to announce UI Coverage, a tool that empowers development teams and leaders to analyze and report on gaps in their existing Cypress tests.
UI Coverage introduces a visual component to the often obscure and complicated process of understanding test coverage. How do you ensure you've tested all critical pages and elements of your web application? How do you verify that all permutations of a specific view have been tested? As a leader, how can you quantify, measure, and report on progress? How can you detect when test coverage is reduced?
With UI Coverage in Cypress Cloud, developers write tests as they normally would - there is no extra setup required. Cypress automatically detects which pages and elements remain untested, providing a visual representation of uncovered areas. By shifting left and integrating UI Coverage early in the development process, everyone on the team can see precisely where test coverage is lacking before pushing changes to production. By highlighting these gaps, UI Coverage helps teams prioritize their efforts, ensuring comprehensive testing and greater confidence in the quality of their applications.
The Problem and Challenges in Achieving Comprehensive Test Coverage
Understanding where risk lies in your application is difficult. Risks include bugs, low-quality applications, lost revenue, and unhappy customers. Missed test coverage leads to these risks, and it’s often hard to know what is tested, where to add new tests, and where changes should be made. Additionally, managing a large number of tests and contributors complicates the process of understanding the overall test suite.
While the benefits of test coverage are undeniable, achieving comprehensive coverage comes with its own set of challenges:
Identifying Critical Paths: Determining which paths and functionalities are critical to your application can be a nuanced task. Test coverage requires a strategic approach to prioritize testing efforts on areas that are most impactful to the overall user experience.
Complex Setup and Configuration: Traditional code coverage tools often come with complex setup and configuration requirements, which can be a significant barrier to effective testing. These tools usually demand extensive configuration, specialized knowledge, and ongoing maintenance, which can divert valuable resources away from actual testing efforts.
Balancing Resources: Test coverage is not a one-size-fits-all metric. It requires a thoughtful balance between testing efforts and available resources. Teams must allocate resources efficiently to ensure comprehensive coverage without sacrificing development speed.
Adopting a Data-Driven Approach: Test coverage is most effective when it's driven by data. This requires teams to adopt a data-driven approach to testing, utilizing metrics and insights to continually optimize and enhance their testing strategies.
The Solution: UI Coverage
UI Coverage offers a seamless solution to these challenges by providing instant coverage reports for your Cypress tests without the need for additional instrumentation. Its visual, easy-to-understand reports highlight areas of your application with missed coverage and create a comprehensive mapping of your test suite to your application. This visual clarity helps all team members, regardless of their technical background, understand the extent of testing and ensures that no critical path is overlooked.
Key Benefits:
Understand Coverage: UI Coverage provides a dynamic visual map of your application's test coverage. This map highlights which interactive elements have been tested and which remain untested, giving you a clear and actionable overview of your testing progress. This visual clarity helps all team members, regardless of their technical background, understand the extent of testing and ensures that no critical path is overlooked.
Improve Testing: By transforming testing from an art into a science, UI Coverage leverages data-driven insights to enhance your testing process. It generates objective metrics that reliably track changes in coverage, focusing on critical paths and essential flows. These metrics form the foundation for setting goals, refining strategies, and ensuring that testing efforts are aligned with business objectives. This systematic approach ensures that your testing is both comprehensive and effective.
Monitor Changes: UI Coverage enables continuous monitoring of your test coverage over time. By tracking coverage scores, teams can promptly detect improvements or deficiencies. This ongoing assessment allows everyone in the software development lifecycle to make informed decisions based on real-time data, ensuring that your team’s testing strategy remains effective and adaptive to changing requirements. Monitoring changes helps you maintain a high standard of quality and quickly address any emerging issues.
Optimize Your Test Suite: UI Coverage helps you optimize your test suite by identifying both gaps and redundant tests. By removing or deduplicating redundant tests, you save on CI time and reduce technical debt. With comprehensive insights into the testing status of every interactive element, you can pinpoint areas that require more attention and those that have been over-tested. This optimization reduces unnecessary testing efforts and ensures that your resources are efficiently allocated. The inclusion of Test Replay within the same workflow also enhances debugging efficiency, further streamlining your testing process.
How UI Coverage Works
UI Coverage creates a dynamic visual map of test coverage across every page of your application. Each interactive element is identified and highlighted as either tested or untested, providing a clear and actionable overview of your test coverage. By capturing "snapshots" of every page visited by your Cypress tests (and every component mounted in Component Testing), UI Coverage represents all the unique states found across your test run. These states are analyzed to identify all the unique elements that make up your application.
As a result, a UI Coverage score is generated. This score quantifies coverage across your application by measuring the ratio of unique elements tested to the total unique elements found across your application. This mapping highlights coverage gaps and areas ripe for optimization, helping teams make informed decisions about where to focus their testing efforts.
UI Coverage breaks the traditional boundaries of a standard code coverage tool, providing an innovative approach to test coverage that transcends mere metrics and transforms the testing process into a measurable discipline. Key differentiations include:
Visual Clarity and Universal Usability: Unlike traditional code coverage reports that present raw data, such as lines of code or statements covered, UI Coverage introduces a dynamic visual overlay. This clear and intuitive representation of testing coverage is designed to be accessible to both technical and non-technical team members, promoting collaboration and ensuring a shared understanding of the extent of testing coverage.
Instant Adoption and No Performance Impact: Traditional code coverage tools often require extensive configuration and setup, which can be time-consuming and resource-intensive. UI Coverage leverages existing data from Cypress tests, eliminating the need for additional configuration. This automatic integration simplifies the testing process and reduces setup time.
Objective Metrics and Insights: While traditional code coverage reports may provide data, they often lack actionable insights that can guide testing efforts. UI Coverage offers a coverage score for your pages and components, providing a quantifiable metric that reveals what has been tested and what remains untested. These insights empower testing teams to make data-driven decisions and improve testing coverage.
Dead Simple Debugging: Traditional code coverage reports typically focus on static coverage metrics without real-time debugging capabilities. UI Coverage integrates Test Replay, allowing users to time travel through test runs in CI for real-time troubleshooting. This feature enhances the efficiency of issue resolution within the same workflow.
Embracing a Future of Application Quality
Comprehensive test coverage identifies gaps in your application where bugs and issues might be hiding. UI Coverage helps development teams proactively address these gaps before they reach end-users, reducing the risk of bugs, downtime, user frustration, and potential financial losses. This ensures your application is reliable and can handle real-world usage with confidence.
UI Coverage also promotes collaboration across the entire development team. When everyone—from developers to product managers, QA engineers, and engineering leaders—understands test coverage, it becomes easier to align on testing priorities, identify critical paths, and ensure overall application quality. Thoroughly tested applications provide a seamless, frustration-free user experience, and test coverage instills confidence in code deployments by reducing the likelihood of introducing regressions or unexpected behavior.
By recording your tests to Cypress Cloud, you can leverage UI Coverage to ensure your applications meet the highest standards of quality.
Why Join Early Access?
By joining the early access program for UI Coverage, you'll be among the first to experience a robust product that is already being adopted by leading organizations. This exclusive opportunity allows you to leverage advanced features and gain a competitive edge in ensuring your application’s quality. Enjoy the benefits of real-time insights, comprehensive test coverage, and streamlined testing processes that have already proven effective in real-world environments. Join a community of forward-thinking teams and start optimizing your testing strategy.
Ready to kickstart your journey with UI Coverage? Start a trial now!