Scenario identity
The report names the tested procedure, baseline, candidate, model, connector, mocks, and run count.
Reviewable release evidence
AI agent test reports that make release blockers traceable.
A pass or fail label cannot tell the team whether approval disappeared, a target changed, or an action happened twice. Konsista connects every finding to the observed tool calls, critical values, and repeat where the procedure broke.
A pass/fail label cannot explain what the agent actually did.
The report should let the release owner reproduce the failure, identify the affected action and target, and understand exactly what the scenario did and did not cover.
Per-run evidence
Each repeat records observed tools, arguments, action order, and the controls that held or failed.
Deterministic findings
Red findings come from literal assertions such as a missing approval, changed target, or duplicate action.
Repeat stability
Aggregate evidence shows whether the procedure is held, intermittent, or consistently unsafe.
Coverage statement
The report lists observed controls and avoids claiming untested safety, correctness, or compliance.
Machine artifact
JSON output supports CI, review automation, comparison, retention, and independent verification.
Read the finding from procedure to action.
A report should let a reviewer answer five questions without rerunning the system.
- What exact scenario and candidate change were tested?
- Which tools and critical values were expected?
- What happened in each baseline and candidate repeat?
- Which deterministic control produced the finding?
- What was outside the scope of the test?
The public approval-gate report includes repeated mocked runs, observed sequence drift, critical target evidence, and a coverage statement.
Report summary
baseline procedure20/20
candidate procedure13/20
critical targetstable
coverageexplicit
The report is evidence, not certification.
A passed scenario shows that the declared controls held for the tested configuration, mocks, data, and repeats. It does not prove that every workflow is correct or safe. A red finding identifies an observed procedure failure; it does not diagnose every possible root cause without further engineering review.
Related AI agent testing guides
Get evidence for the action blocking your release.
Send the scenario, expected controls, and candidate change. The first run produces reviewable HTML and JSON evidence.
Generate one release report