The Checkpoint Scale scores a single decision point in an agentic system on five dimensions and returns a recommended handling: auto_execute, log_and_proceed, or hold_for_human. Its dimensions and anchors are published so that it can be applied, argued with, and cited. The weighting model is available to Checkpoint Copilot API users and research partners. Version 1.0, published 17 September 2026.
What it scores
The unit is one decision point, meaning one specific action an agent can take without a human, not a system, a workflow, or an industry. The same agent in the same workflow will produce very different scores across its own decision points, and that variance is the entire point of measuring it.
Reversibility
How hard the action is to undo once it has taken effect.
| Score | Anchor definition |
|---|---|
| 1 | Fully reversible by the agent itself. No trace, no external party affected. |
| 2 | Reversible by the team within hours. No one outside has acted on it. |
| 3 | Reversible with effort, and someone outside the team may already have seen or acted on it. |
| 4 | Reversible only through an exception process, escalation, or manual remediation with real cost. |
| 5 | Not reversible. Money moved, data deleted, message delivered, contract executed, physical action taken. |
Consequence magnitude
The size of the harm if the action is wrong and is not caught.
| Score | Anchor definition |
|---|---|
| 1 | Trivial. Noise in a log, seconds of wasted compute. |
| 2 | Minor. One user inconvenienced, recoverable in a single interaction. |
| 3 | Moderate. A customer relationship takes damage, or a bounded financial loss the team absorbs. |
| 4 | Severe. Material financial loss, a breached commitment, or damage visible outside the organisation. |
| 5 | Critical. Safety, legal liability, large financial loss, or reputational harm that outlives the incident. |
Confidence
The calibrated likelihood that this specific action is correct. Not the model's fluency, and not how certain the output sounds.
| Score | Anchor definition |
|---|---|
| 1 | Low. Ambiguous inputs, conflicting evidence, or the agent is extrapolating past its retrieved context. |
| 2 | Below average. Plausible, but resting on an inference the agent cannot check. |
| 3 | Moderate. Reasonable evidence, no direct confirmation. |
| 4 | High. Direct evidence in context supports the action, and this task shape has succeeded before. |
| 5 | Very high. Deterministic and verifiable before execution. |
Precedent
How well-trodden this decision type is in this system.
| Score | Anchor definition |
|---|---|
| 1 | Novel. First time this decision shape has appeared. No established handling. |
| 2 | Rare. A handful of occurrences, no settled norm. |
| 3 | Occasional. A pattern exists, but exceptions are common. |
| 4 | Routine. Handled regularly, with a documented expected outcome. |
| 5 | Fully standardised. High volume, stable, with a known correct answer and an existing audit trail. |
Regulatory exposure
Whether this specific action, if wrong, creates a disclosure obligation, a discriminatory-impact question, or a gap in the audit trail. Not whether the industry is regulated.
| Score | Anchor definition |
|---|---|
| 1 | None. No regulated artefact, no record-keeping obligation. |
| 2 | Incidental. Touches regulated data but creates no decision or record a regulator would examine. |
| 3 | Moderate. Produces a record that could be examined, with no direct effect on a person's rights or access. |
| 4 | High. Produces a documented reason, decision, or disclosure affecting a person's access, money, or standing. |
| 5 | Critical. Directly creates a legally operative decision, disclosure, or record in a supervised domain. |
Scoring
Each dimension contributes to a weighted risk score. Reversibility and consequence magnitude carry the most weight. Higher confidence and stronger precedent reduce risk. The score places a decision in one of three bands: auto_execute, log_and_proceed, or hold_for_human, and every result comes with a plain-language reason naming the dimensions that drove it.
The full weights and band thresholds are shared with Checkpoint Copilot API users and research partners under agreement. Contact hello@behaviorist.io.
Known properties and limits of v1.0
- The weights are theory-driven, not empirically calibrated. They were set a priori from the behavioural literature on automation bias, reversibility, and escalation cost. Calibrating them against real incident outcomes is an open question, not a solved one.
- A score is an ordinal position, not a probability of failure, and it should never be reported as one.
- No inter-rater reliability has been established yet. Two engineers scoring the same decision point may not agree, and how much they disagree is currently unmeasured. Establishing that is the next piece of work on this instrument.
- The five dimensions are treated as independent. They are probably not. Whether reversibility and consequence magnitude are empirically separable is an open question.
- This is a prioritisation instrument, not a safety certification and not a compliance control. A high score is a reason to put a named human on a decision. It is not evidence that a system is safe, and it does not discharge any obligation.
Using it
It can be applied by hand with the tables above to structure a review conversation. Checkpoint Copilot implements the full scoring model as an API at behaviorist.io, linked from the homepage.
Contribute to calibration
Two things would move this instrument from theory-weighted to evidence-weighted, and both need people outside Behaviorist. First, independent scorers: engineers willing to score the same set of decision points so inter-rater agreement can be measured and published. Second, outcome data: decision points scored before the fact, paired with what actually happened, which is what weight calibration requires. If you have either, the contact is hello@behaviorist.io.
Version history
v1.0, 17 September 2026. First public release. Five dimensions, theory-set weights, three-band recommendation.
How to cite
Behaviorist applies behavioural and decision science to human-agent interaction.
Explore Checkpoint Copilot on the homepage