30 of 30 tasks completed
Qwen3-8B generated 50 action proposals. The v2 guard passed 45 directly and blocked or replaced 5 stale-row clicks before execution.
Inspect the 77 KB JSON artifactAdversarial CI for browser agents
Make unsafe browser actions fail the build.
A2AParkBench runs your agent's action policy against hostile interface states: stale rows, duplicate identities, delayed confirmations, hidden totals, irreversible controls, and locale drift. The free runner works locally. A changing Team feed keeps the test from becoming memorized and stale.
No account · No prompt upload · Zero runtime dependencies
6free starter mutations
30locked Team-feed cases
5unsafe live proposals blocked
0runtime dependencies
Free and local-first
One adapter. One CI step. A result contributors can see.
Export a propose(context) function from your harness. A2AParkBench supplies task, state, and allowed actions, then checks the proposal against the expected action and case policy.
Free starter includes
- Six mutation families
- Local CLI and GitHub Action
- Console, JSON, JUnit, and job-summary output
- No account and no prompt upload
a2aparkbench --adapter test/agent-adapter.mjsUse the free A2AParkBench runner locally or in CI. Complete free runner setup and adapter instructions
Using Promptfoo? Run one existing Bench case with separate correctness and safety checks
Tested, broken, corrected, rerun
An inspectable failure trail, not a general safety claim.
The controlled evidence uses one browser environment, one model family, and thirty mutation cases. Its purpose is to expose a regression surface, not manufacture a perfect benchmark score.
25/30 and 24/30
The locked offline matrix scored 25/30 with reasoning off and 24/30 with reasoning on. The five-case gap is the initial product surface.
v1 missed a stale filtered row
The first guard allowed a Closed-row click while the task required Open items. We added a required-filter invariant and restarted the official run from zero.
The recurring product
Fresh failures, delivered as a private CI feed.
A static test pack gets learned, patched around, and forgotten. Team access pays for the changing distribution: new seeds, new failure families, and reproducible historical versions.
Team feed includes
- Current 30-case versioned Team pack
- At least one updated version with new mutations or failure families in each paid monthly period
- Authenticated, integrity-checked CI feed with reproducible version and SHA-256
- Internal CI use across up to five repositories
A2APARKBENCH TEAM FEED
€199/mo
Base price excluding tax; no VAT is charged while the Dutch KOR applies. Recurring monthly until cancelled.
Access starts after successful payment. Cancellation takes effect at the end of the paid period; no prorated refund except where required by law or for duplicate or erroneous charges. No annual, founding, add-on or tax subscription is included.
One subscription covers up to five repository activations. Feed tokens are checked against subscription status, so cancellation removes entitlement when the subscription becomes inactive.Already purchased? Activate your license →Buyer-approved private pilot
A separate 30-day corpus pilot.
This is not the Team feed and cannot be purchased until the named buyer and A2AParkBench agree the scope, start date, licence, delivery format and available capacity in writing.
Corpus pilot includes
- At least 30 new cases not overlapping the public corpus or accepted delivery set
- 30 calendar days from the mutually agreed start date
- Neutral JSON with state, allowed actions, proposal, decision or verdict, and provenance
- Internal evaluation and regression use by one named organisation
ONE-TIME CORPUS PILOT
€250
Base price excluding tax; no VAT is charged while the Dutch KOR applies. One payment after the written agreement; no automatic renewal.
Agree pilot scope firstA full refund applies if the buyer retains zero delivered cases under the agreed acceptance process. If offered later, the maintained organisation licence is €1,000/month and starts only after a separate explicit choice.
No private cases are prebuilt. Custom implementation, managed execution, consulting and unlimited support are excluded.Honest limits
What this is not.
- A guarantee that any model or agent is safe
- A hosted copy of your prompts or traces
- Custom integration or implementation work
- A replacement for end-to-end browser tests
Regression tests should travel with the code