Skip to main content

AI assertions (beta)

AI assertions are a powerful way to check complex conditionals about your page. Unlike deterministic assertions or expect statements from traditional testing frameworks, they allow for checking for high-level conditionals. You can treat the assertion agent as a junior QA engineer that just joined your team and has a only a superficial knowledge of your product.

Common use cases​

Checking for logic​

  • Check that the user is logged in
  • Check that a modal is open
  • Check that there are 3 search results
  • Check that the page is about software engineering

Checking for visual elements​

  • Check that there is a red title at the top of the page
  • Check that there is a navigation bar on the left
  • Check that there is a visible search bar

Prompting tips​

Test for general statements, not for exact text matches, because the agent might behave unpredictibly on exact text matches. Keep it general, and use deterministic assertions for testing the presence/absence of specific text.

✅ Check that there is a banner with a welcome message

❌ Check that there is a banner with the text 'welcome, my dear friend!'

Keep it general. No jargon or industry specific terms. The agent has a lot of general context, but cannot interpret very specific instructions.

✅ Use a javacript step or a deterministic assertion to implement your logic.

❌ Check that the ACH SEC code is within range.

Test for UX and logic, not for HTML The agent will visually check your page for a certain behaviour, not for specific HTML snippets.

✅ Check that there is a modal open

❌ Check that there is div > div.modal > p chunk

Unsupported use cases​

The following use cases are not recommended because they are either completely unsupported, or result in unstable behaviours.

  • ❌ Checking that a transient event did NOT happen. Eg. "check that no banner appeared"
  • ❌ Checking for events that last less than three seconds
  • ❌ Checking for complex logic. Eg. "Check that the VAT rate is correct".