Test runs and reports

The scenarios you agreed are also the tests. A BDD run executes them against the running application and reports, scenario by scenario, what passed.

Where runs come from

A run is submitted from wherever the project's test runner wrote it, rather than from one fixed path the platform insists on. Your project's runner lives where it lives — at the repository root, one level down, or in a shared test folder for a project combining several repositories — and the platform finds it.

Runs can also be produced outside the platform. Test evidence generated locally can be pushed and replayed, which is the usual route for a developer who ran the suite on their own machine.

An uploaded run updates each scenario's test status, so the specification itself carries the result. A scenario is not just described in the product; it is green or red in it.

Reading a report

Reports collect what a run found; the Report Catalog is the inventory across runs.

A failing scenario is worth reading in this order:

1. The scenario — is the expectation right? A scenario can fail because the implementation is wrong, or because the scenario describes something nobody agreed to.
2. The trace — what the browser actually did, step by step, with the page as it was rendered at each step. UI implementation is verified against rendered trace frames, so the frames are the evidence rather than a summary of it.
3. The code, last.

Traces are viewable in the web IDE too, under the Repave sidebar's BDD Runs view.

Test Replay

Test Replay re-runs a scenario so you can watch it. It stays scoped to the Gherkin view you are in, so you are replaying the scenarios in front of you rather than the whole suite.

While replaying, you can pin problems directly to what you see, and report several pinned problems from one replay as one submission — see Bug reports.

Unit tests

For a provided modernized codebase, unit tests run the way your project already runs them, using the configuration detected when the project was set up. Detection reports why it will not run rather than silently skipping. Whether unit tests, coverage thresholds and a false-positive review gate an implementation is a project setting — see Project settings.

Surfaces your operator may have enabled

Off unless your installation turns them on:

• Cucumber Test Runner and Unit Test Runner — running suites from the browser.
• UAT Tools — user-acceptance environments per feature, with test data committed as seed files, a database baseline restored from a server-side backup, and several environments able to run at once. UAT sessions support the same pinned problem reporting as Test Replay.
• Characterization Tests — capturing existing legacy behaviour as tests.

Next: Bug reports.