---
title: "How to Run Changed Specs First in a Pull Request"
url: "https://www.cypress.io/blog/how-to-run-changed-specs-first-in-a-pull-request"
description: "Imagine a large project with hundreds of Cypress spec files. If you work on just one feature and open a pull request with small code changes and corresponding…"
---

# How to Run Changed Specs First in a Pull Request

April 15, 2020

By Gleb Bahmutov

Imagine a large project with hundreds of Cypress spec files. If you work on just one feature and open a pull request with small code changes and corresponding updates to 1 Cypress spec file, it makes sense to try running _that changed_ spec file first. Chances are high that any new issues will be introduced in the changed code, so by running the changed files first, we will discover the problem more quickly.

After running Cypress with changed specs first, we still should run _all spec files._ The second test run ensures that any changes have not introduced bugs somewhere else. In this blog post I will show how to quickly set up such "double run" continuous integration workflow. I will use GitHub Actions, but the main shell code to determine and run the changed specs uses plain Git commands, and thus this blog post can be implemented on most CI vendors.

**Note:** you can find the finished project in repository [](https://github.com/bahmutov/changed-cy-tests?ref=cypress-io.ghost.io)[bahmutov/changed-cy-tests](https://github.com/bahmutov/changed-cy-tests?ref=cypress-io.ghost.io)

First, I will create a GitHub workflow to run on every pull request.

```yml
name: ci
on: [pull_request]
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout 🛎
        uses: actions/checkout@v2
```

The above `ci` workflow uses GitHub's own [](https://github.com/actions/checkout?ref=cypress-io.ghost.io)[actions/checkout](https://github.com/actions/checkout?ref=cypress-io.ghost.io) to bring the code from GitHub to the CI container. On every pull request, the checked out code is a merge request between the head (aka the current) and the base (aka the target) branches. For example, this [pull request #2](https://github.com/bahmutov/changed-cy-tests/pull/2?ref=cypress-io.ghost.io) has the head branch named [](https://github.com/bahmutov/changed-cy-tests/tree/update-test-b?ref=cypress-io.ghost.io)[update-test-b](https://github.com/bahmutov/changed-cy-tests/tree/update-test-b?ref=cypress-io.ghost.io) and the base branch `master`.

By default, only the merge commit is checked out - to save time. Thus, if we want to see what has changed between the `update-test-b` and `master` branches, we need to pull them from the remote `origin` to the local source.

**Tip:** you can see the remote branches by running `git ls-remote` command.

```text
$ git ls-remote
From git@github.com:bahmutov/changed-cy-tests.git
e5007aa819980ffd469dd1f893678970592be5d1	HEAD
acde0808e5c303136e95e6f6c039eb52d1c73d62	refs/heads/changed-tests
e5007aa819980ffd469dd1f893678970592be5d1	refs/heads/master
27ac3826934eb54cf4f119fb08afe64f5dd92011	refs/heads/update-test-b
acde0808e5c303136e95e6f6c039eb52d1c73d62	refs/pull/1/head
27ac3826934eb54cf4f119fb08afe64f5dd92011	refs/pull/2/head
503c73efc52f8c88db8316f174cb0a3afbd1bdf0	refs/pull/2/merge
```

Let's get the two branches from the remote origin (GitHub) into local CI container. We can hardcode the branch names or use [GitHub Action Expressions](https://help.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions?ref=cypress-io.ghost.io) to get the names from the current Pull Request event.

```shell
git fetch --no-tags --depth=1 origin ${{ github.base_ref }}
git fetch --no-tags --depth=1 origin ${{ github.head_ref }}
# get the actual source for two branches
git checkout origin/${{ github.base_ref }}
git checkout origin/${{ github.head_ref }}
# get back to the merge commit
git checkout ${{ github.sha }}
```

Now that the local Git has the information for both branches, let's find the changed files between two branches using [git diff](https://git-scm.com/docs/git-diff?ref=cypress-io.ghost.io) command.

```shell
git diff --name-only origin/${{ github.base_ref }} origin/${{ github.head_ref }}
```

We are only interested in the filenames, but we also want to limit ourselves to names of changed Cypress specs. In my example, all specs are in `cypress/integration` folder. Let's pass this folder name to `git diff` to limit the returned list to files in that subfolder.

```shell
git diff --name-only \
  origin/${{ github.base_ref }} origin/${{ github.head_ref }} \
  -- cypress/integration
```

Note the "--" separator between the Git branch labels from the folder name.

Pull request with a single changed spec file

Once we have found the changed Cypress specs, if this list is non-empty, we should run them first.

```shell
CHANGED_SPECS=$(git diff --name-only origin/${{ github.base_ref }} origin/${{ github.head_ref }} -- cypress/integration)
if [ -n "$CHANGED_SPECS" ]; then
  echo "Running the following changed specs"
  echo $CHANGED_SPECS
  npx cypress run --spec $CHANGED_SPECS
fi
```

We can see the quick first Cypress run limited to `spec-b.js`

Running just the single changed spec first

After running just the changed specs, we still have to run all specs to thoroughly test the entire system. In this case, we can use [Cypress GH Action](https://github.com/cypress-io/github-action?ref=cypress-io.ghost.io). This action allows us to split the install and test run steps, thus we don't have to worry about installing and caching Cypress in each run - it is done for us automatically.

```yml
- name: Install Cypress and dependencies 📦
  uses: cypress-io/github-action@v1
  with:
    runTests: false

# quickly run just the changed specs to fail fast
- name: Maybe run just the changed Cypress tests ⏱
  run: |
    CHANGED_SPECS=$(git diff --name-only origin/${{ github.base_ref }} origin/${{ github.head_ref }} -- cypress/integration)
    if [ -n "$CHANGED_SPECS" ]; then
      echo "Running the following changed specs"
      echo $CHANGED_SPECS
      npx cypress run --spec $CHANGED_SPECS
    fi

# run all tests to be sure
- name: Run all Cypress tests 🧪
  uses: cypress-io/github-action@v1
  with:
    # we have already installed all dependencies above
    install: false
```

Let's say while we are working on branch `update-test-b`, someone changes test file `spec-c.js` on `master`. Our check finds those two files and runs them both!

The pull request with two changed specs against master

Once we have the test jobs working, let's record results on the [Cypress Dashboard](https://on.cypress.io/dashboard-introduction?ref=cypress-io.ghost.io) - so we can see the passing or failing tests and quickly debug their failures. I have added `projectId` to `cypress.json` file, and set the `CYPRESS_RECORD_KEY` as a secret in GitHub repository settings. I will need to set the record key as an environment variable for two run steps that need it. Then we can use CLI arguments and GitHub Action parameters to turn on the recording mode. Let's give them group names and the same CI build id so both steps record into a single Dashboard "Run".

```yml
- name: Maybe run just the changed Cypress tests ⏱
  env:
    # pass the Dashboard record key as an environment variable
    CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
  run: |
    CHANGED_SPECS=$(git diff --name-only origin/${{ github.base_ref }} origin/${{ github.head_ref }} -- cypress/integration)
    if [ -n "$CHANGED_SPECS" ]; then
      echo "Running the following changed specs"
      echo $CHANGED_SPECS
      npx cypress run --record --group "Changed specs" --ci-build-id ${{ github.sha }} --spec $CHANGED_SPECS
    fi

# run all tests to be sure
- name: Run all Cypress tests 🧪
  uses: cypress-io/github-action@v1
  env:
    # pass the Dashboard record key as an environment variable
    CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
  with:
    # we have already installed all dependencies above
    install: false
    record: true
    group: All specs
    ci-build-id: ${{ github.sha }}
```

You can see the recorded runs at [](https://dashboard.cypress.io/projects/aobpjx/runs/?ref=cypress-io.ghost.io)[https://dashboard.cypress.io/projects/aobpjx/runs/](https://dashboard.cypress.io/projects/aobpjx/runs/?ref=cypress-io.ghost.io)

Single Pull Request test run with groups from both Cypress runs

**Question:** the above shell script looks a lot like Jest `--changedSince` flag. Why isn't this a CLI flag in the Cypress test runner?

**Answer:** Yes, this experiment and blog post were inspired by [Jest and the](https://dev.to/bnb/jest-and-the-changedsince-flag-in-github-actions-ci-468i?ref=cypress-io.ghost.io) `[--changedSince](https://dev.to/bnb/jest-and-the-changedsince-flag-in-github-actions-ci-468i?ref=cypress-io.ghost.io)` [flag in the GitHub Actions CI](https://dev.to/bnb/jest-and-the-changedsince-flag-in-github-actions-ci-468i?ref=cypress-io.ghost.io) blog post. But adding _any_ new feature to the Cypress test runner core is a decision we do not take lightly. Every flag adds new code, new tests, new documentation. Every feature requires maintenance and slows down further development. Thus if one can quickly implement the feature _outside_ the test runner, then it seems to be the appropriate solution.