# a2a-proof

[![CI](https://github.com/aspix2k/a2a-proof/actions/workflows/ci.yml/badge.svg)](https://github.com/aspix2k/a2a-proof/actions/workflows/ci.yml)
[![Coverage](https://img.shields.io/badge/coverage-%E2%89%A599%25-brightgreen)](https://github.com/aspix2k/a2a-proof/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/aspix2k/a2a-proof)](https://github.com/aspix2k/a2a-proof/releases)
[![PyPI](https://img.shields.io/pypi/v/a2a-proof)](https://pypi.org/project/a2a-proof/)
[![Python](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)

Black-box contract tests for A2A agents.

`a2a-proof` discovers a deployed agent, sends real A2A requests, and checks its observable
behavior. It needs no access to the agent's source code, framework, prompts, or model provider.

The official [A2A TCK](https://github.com/a2aproject/a2a-tck) checks protocol conformance,
[A2A ITK](https://github.com/a2aproject/a2a-itk) checks interoperability between SDKs, and the
[A2A Inspector](https://github.com/a2aproject/a2a-inspector) supports interactive debugging.
`a2a-proof` adds repeatable contracts for the behavior of your deployed agent.

It targets A2A 1.0 over JSON-RPC, HTTP+JSON, and gRPC. The SDK compatibility layer also supports
AP2 v0.2.0 agents that expose A2A 0.3 JSON-RPC, including signed mandate-to-receipt payment flows.

## What you can prove

| Risk | Contract |
| --- | --- |
| A prompt, model, or backend change alters an answer | Text, structured data, JSON Schema, and file integrity assertions |
| An LLM succeeds only some of the time or gets slower | Repeated trials, statistically bounded pass rates, parallel runs, and p50/p95 latency |
| A long-running task breaks after acceptance | State trajectories, stream resumption, cancellation, persistence, and push delivery |
| Staging no longer behaves like production | Agent Card preflight and deployment diff |
| A contract change needs review without a live agent | Recorded cassettes replayed offline |
| Agent text leaks a secret or system prompt | Global invariants and bounded failure evidence |
| An orchestrator stops calling its specialist, or forwards credentials to it | Delegation contracts against a recording downstream agent |
| An agent produces an invalid payment proof | Signed AP2 mandate-chain and receipt verification |

## Quick start

With [uv](https://docs.astral.sh/uv/getting-started/installation/) installed, try a real loopback
A2A exchange without an API key or external service:

```console
uvx a2a-proof demo
uvx a2a-proof demo --fail
```

The second command deliberately shows a contract failure and exits `1`.

Point the runner at your agent:

```console
uvx a2a-proof init https://agent.example.com
```

`init` reads the Agent Card and creates `a2a-proof.yaml`. Replace its smoke scenario with behavior
your users depend on:

```yaml
version: 1

agent:
  url: https://agent.example.com

defaults:
  trials: 5
  pass_rate: 0.8

scenarios:
  - name: billing dispute routing
    message: A customer says their card was charged twice for order 4815.
    latency:
      p95_seconds: 15
    expect:
      state: completed
      data:
        - path: /queue
          equals: billing-disputes
        - path: /priority
          matches: "(?i)^high$"
```

```console
$ uvx a2a-proof run
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Result ┃ Scenario                ┃ Trials ┃ Time ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ PASS   │ billing dispute routing │    4/5 │ 3.2s │
└────────┴─────────────────────────┴────────┴──────┘

1 scenario passed in 3.2s
```

Run one scenario, save failure evidence, or compare the same contract against another deployment:

```console
uvx a2a-proof run --scenario "billing dispute routing"
uvx a2a-proof run --format junit --output a2a-proof.xml --evidence evidence
uvx a2a-proof diff --against https://candidate-agent.example.com
uvx a2a-proof record --output cassette.json && uvx a2a-proof run --replay cassette.json
```

## GitHub Actions

After checking out the repository, one step runs its default contract as a CI check:

```yaml
- uses: aspix2k/a2a-proof@v0.16.0
```

Set `config` only when the contract is not `a2a-proof.yaml`.

## Documentation

- [Writing contracts](docs/contracts.md)
- [Assertions](docs/assertions.md)
- [Task lifecycle](docs/lifecycle.md)
- [Push notifications](docs/push-notifications.md)
- [Delegation contracts](docs/delegation.md)
- [Recording and replay](docs/cassettes.md)
- [AP2 contracts](docs/ap2.md)
- [External agent showcases](docs/showcases.md)
- [Running in development and CI](docs/operations.md)
- [Configuration schema](schema/a2a-proof.schema.json)

The schema provides completion and inline validation in YAML-aware editors. `init` links it
automatically.

## Safety

Agent responses and file metadata are treated as untrusted input. Requests, response parts,
regular expressions, embedded schemas, evidence, and local file access are bounded. Redirects,
credential-bearing URLs, external schema references, and cross-origin interfaces are rejected by
default. Remote file URLs are never fetched or written to reports. Requests recorded by the
delegation agent stay in memory and are never written to reports or evidence.

See [SECURITY.md](SECURITY.md) for private vulnerability reports.

Contributions follow [CONTRIBUTING.md](CONTRIBUTING.md). Licensed under the [MIT License](LICENSE).
