How to analyze story results

After running a story or a test suite, you will see the results in the Runs tab. You can also turn on Slack or Discord to get notified in slack with test results.

Run results

A run can have one of three results

• PASS (green): the test has passed, your webapp works as expected
• CRASH (orange): heal cannot execute the test, either because of a bug or because the scenario is ambiguous. Try running interactively in the test studio to fix the test.
• FAIL (red): the test has failed, and heal has found a bug

Replay

You can access the replay by clicking on a story in the run tab. The replay shows you which steps were executed and their status, and a playable session replay of the test excution.

If the test failed, you will also get a root cause analysis (RCA) of what happened and what bug heal detected.

The flake button lets you give feedback on the run, either flagging that the agent has poorly executed the steps, or that the test result is unexpected (eg. a bug wasn't detected). This is used by the agent to re-run tests, and by the heal team to investigate issues.