We’ve received a ton of excitement (and a bunch of questions) about cy.prompt, so we wanted to take a moment to share some answers and give a quick update on where things stand.
If you haven’t already, check out Brian Mann’s full demo - it answers many of the early questions we’ve seen.
So, let’s dive in.
What's the difference between cy.prompt and an MCP server?
Cypress has always been opinionated and batteries-included. This approach extends to our AI features, such as cy.prompt, which is built to work right out of the box, offering a fast and robust testing experience.
While an MCP server exposes specific capabilities in a standardized fashion to LLMs (Large Language Models), the responsibility remains on the user to choose the right model, prompt appropriately, generate tests, maintain and improve them. With cy.prompt, we make some of these choices behind the scenes to keep you focused on the outcome - writing test prompts and running them reliably.
The result is a fast and intuitive experience integrated into your existing toolchain. Here is a video to prove it.
Do I need a Cypress Cloud account to use cy.prompt?
Yes, a Cypress Cloud account is necessary. cy.prompt leverages LLMs, and Cypress Cloud securely manages these requests, providing your organization visibility with clear, non-invasive usage details.
We want to emphasize that our models are NOT trained on customer data (including prompts). The models used are hosted securely on our managed Cloud infrastructure, and any information shared with the models is for the sole purpose of providing the feature. Model inputs and outputs are session-bound and not shared with any model provider. Further, our contractual agreements with sub-processors expressly prohibit the retention and use of customer data to train models.
Will cy.prompt make tests flaky or less reliable?
Cypress generates and caches code when a prompt runs for the first time, ensuring consistent behavior and good performance for subsequent test runs. If your application changes and the cached code becomes invalid, Cypress will automatically re-run the prompt to adapt the test.
The quality of the generated code is influenced by factors such as your app's support for stable interactions, its accessibility, and the clarity of your prompt. Regardless, the generated code is always visible, preventing a "black box" scenario.
Will this slow down my test suite?
LLM calls are optimized for efficiency, occurring only once per prompt. The generated code is then cached and reused in subsequent runs, ensuring a fast test suite and minimizing LLM usage. For complete independence from the LLM, you have the option to eject the code and commit it directly to your repository.
Why keep the prompt instead of just using static code?
Prompts enable tests to self-heal. Cypress automatically generates and consistently executes test code. If an application change causes a step to fail, Cypress retries the prompt and regenerates the code.
How will I know what the prompt did?
Every time a prompt is executed, the complete generated code is displayed. This code can be viewed in the browser during test execution, or it can be extracted and committed. Furthermore, prompts can be troubleshooted and debugged as a standard Cypress test, ensuring full transparency.
How much will it cost?
During the experimental release, access will be provided at no additional charge. While certain built-in limits on prompt calls will apply, these are not intended to block reasonable usage. Our objective is to collect feedback and utilize this information to guide future development of the feature.
Please note that pricing and usage limitations are subject to future adjustments, and we commit to keeping you informed of any such changes.
When can I use cy.prompt?
We're aiming for a release by the end of summer! We are currently in the final stages of development and are committed to ensuring it meets the high bar of stability, polish, and production readiness you've come to expect of Cypress.
If you want to get early updates or be among the first to try it, fill out our early access form. We’d love to hear what you think.