# ABI Improvement Maturity Model

## Levels

### Level 1 — Manual Iteration (Baseline)
- Prompt tweaks based on intuition
- No structured failure tracking
- Manual HIL decisions

### Level 2 — Offline Eval Suite (Current: abi-evals)
- Task-specific regression testing
- Golden fixture baselines
- Promotion only if evals pass

### Level 3 — Observability + Failure Taxonomy (Target: this wave)
- Structured telemetry per run (abi-observability)
- Classified failure types (failure_taxonomy schema)
- Gate ledger with hash-chained receipts
- Calibration reports per run
- Governance regression blocking

### Level 4 — Evidence-Based Routing + Granular HIL (Next target)
- Calibration aggregation over time
- Routing suggestions from observed data (human-approved)
- HIL suggestions from observed data (human-approved)
- Model version canary before upgrades
- Failure-to-eval pipeline (automatic regression test generation)

### Level 5 — Bounded Self-Improvement (Future, requires strong telemetry)
- Automated candidate changes with gated promotion
- All changes versioned, replayable, auditable
- Never autonomous — always gated and reversible

## Current Assessment: Level 2, transitioning to Level 3

## Architectural Principles
- All improvement must be versioned
- All calibration must be reproducible
- Model upgrades are controlled events
- HIL is explicit and governed
- Prefer deterministic constraints over increased human review
- No opaque auto-adjustments without artifact trails
