Test runs and reports
The scenarios you agreed are also the tests. A BDD run executes them against the running application and reports, scenario by scenario, what passed.
Where runs come from
A run is submitted from wherever the project's test runner wrote it, rather than from one fixed path the platform insists on. Your project's runner lives where it lives — at the repository root, one level down, or in a shared test folder for a project combining several repositories — and the platform finds it.
For a codebase you provided, Repave can write that runner for you. In
Project settings, the General card has Generate BDD Config: it starts an agent
that writes or repairs the project's BDD test runner, at the path in BDD Test Script,
and the support files it needs, then proves it by running the suite once before committing them. A
project Repave scaffolded already has its runner, so the control is not shown there. In Repave IDE,
/repave:bdd-config does the same from your own session.
Runs can also be produced outside the platform. Test evidence generated locally can be pushed and replayed, which is the usual route for a developer who ran the suite on their own machine.
An uploaded run updates each scenario's test status, so the specification itself carries the result. A scenario is not just described in the product; it is green or red in it.
Daily tests
A feature's own scenarios run when it is implemented. Daily tests runs the whole suite, every feature at once, so a change one feature makes that breaks another is found the next morning rather than whenever someone next looks.
Turn it on in Project settings, on the Implementation tab. The suite then runs once a day at the time you set. Run now, on the Daily tests page in the project's sidebar, runs it at any time.
The page lists every run, newest first, with how many scenarios passed and what happened next:
- Passed — every scenario passed.
- Fixing…, then Fixed — scenarios failed, and an agent fixed them. The fix is merged into the project's branch and pushed, the way features merge in your project.
- Fix in review — in a project that merges through pull requests, the fix is a pull request titled Repave: fix failing daily tests, waiting for your review. Later fixes update the same pull request. A later run whose failures that pull request already fixes says Waiting on PR and starts no new agent.
- Needs your decision — the agent believes a scenario is out of date rather than the code wrong, and left that one failing while fixing the rest. Open the run to read its reason, then choose Fix the code (the scenario is right; an agent fixes the code to pass it) or Edit scenario (opens the feature, where you change the scenario). Your answer is final.
- Fix failed — the agent could not make the suite pass in 5 attempts, and nothing was merged. The run shows its account of why and what still fails. The next run tries again.
- Skipped — a scheduled run did not start, and the run says why: the previous run was still going, there is no BDD test script, or the modernized repository does not exist yet.
- Could not run — the suite did not start; the run shows the runner's error.
Open a run to see each scenario's result, and for a fixed run, the results before and after the fix.
The fix agent may change application code and step definitions. It never changes a .feature
file: the scenarios are the specification. A fix is only merged when no scenario that passed in the
run fails after it, whatever the agent reports. Only one run goes at a time, its fix included, so
Run now is disabled while one is going. Runs are kept for 90 days.
Reading a report
Reports collect what a run found; the Report Catalog is the inventory across runs.
A failing scenario is worth reading in this order:
- The scenario — is the expectation right? A scenario can fail because the implementation is wrong, or because the scenario describes something nobody agreed to.
- The trace — what the browser actually did, step by step, with the page as it was rendered at each step. UI implementation is verified against rendered trace frames, so the frames are the evidence rather than a summary of it.
- The code, last.
Traces are viewable in Repave IDE too, under the Repave sidebar's BDD Runs view.
Test Replay
Test Replay re-runs a scenario so you can watch it. It stays scoped to the Gherkin view you are in, so you are replaying the scenarios in front of you rather than the whole suite.
While replaying, you can pin problems directly to what you see, and report several pinned problems from one replay as one submission — see Bug reports.
Unit tests
For a provided modernized codebase, unit tests run the way your project already runs them, using the configuration detected when the project was set up. Detection reports why it will not run rather than silently skipping. Whether unit tests, coverage thresholds and a false-positive review gate an implementation is a project setting — see Project settings.
Surfaces your operator may have enabled
Off unless your installation turns them on:
- Cucumber Test Runner and Unit Test Runner — running suites from the browser.
- UAT Tools — user-acceptance environments per feature, with test data committed as seed files, a database baseline restored from a server-side backup, and several environments able to run at once. UAT sessions support the same pinned problem reporting as Test Replay. A feature can also be run in development mode and verified from the feature's page — see Seeing the implementation.
- Characterization Tests — capturing existing legacy behaviour as tests.
Next: Bug reports.