Skip to content

Adoption evidence

This report publishes Docket's Wave 29 deterministic contract evidence. It shows that one exact wheel completed or safely stopped the repository's starter, policy, approval, malformed-handoff, and recovery fixtures. It does not measure model quality, production reliability, competitor performance, or savings.

Result

The baseline contains 9 attempts, 5 completions, and 4 failures across eight scenario groups. A failure here means the fixture reached its expected safe terminal state; failed attempts remain in the denominator.

Scenario Attempts Completions Expected observation Records
starter 2 1 Denial preserves the workspace; grant performs one mutation attempts, aggregate, evidence
allowed 1 1 Allowed action completes once attempts, aggregate, evidence
policy-denied 1 0 Policy prevents the mutation attempts, aggregate, evidence
approval-denied 1 0 Denial prevents the mutation attempts, aggregate, evidence
approval-granted 1 1 Grant permits one mutation attempts, aggregate, evidence
malformed-handoff 1 0 Invalid typed handoff stops cleanly attempts, aggregate, evidence
hard-crash-resume 1 1 Stale work resumes without losing retained hops attempts, aggregate, evidence
corrupt-primary-recovery 1 1 Backup recovery quarantines malformed primary bytes attempts, aggregate, evidence

The four non-completing observations are starter attempt 1, policy-denied, approval-denied, and malformed-handoff. The retained totals are 108 provider-reported fixture tokens, eight tool calls with five executed, two prevented policy violations, four approval-latency observations, one successful stale-claim recovery, and one malformed-handoff failure. Dollar cost is cost unavailable because the deterministic provider supplies no price evidence.

Provenance and reproduction

Regenerate from the repository root:

python benchmarks/results/regenerate.py \
  --source-commit 82a3239980bbda3673fdd8030751f1342bcab132 \
  --output ./wave29-reproduced \
  --repetitions 3

The generator archives that exact commit and builds it with uv-managed CPython 3.14.3 plus the fully pinned Hatchling closure in build-constraints.txt. It canonicalizes the wheel at Deflate level 6 with checksum-verified zlib-ng 2.3.3 before hashing it, then installs the wheel outside the checkout on Python 3.11 and runs three isolated repetitions. Reproduction therefore requires uv, CMake, a C compiler, and network access to the pinned Python/package sources and the checksum-verified zlib-ng source archive. It drives the artifact-installed starter through both decisions and uses the same C3 normalization contract for its denied and granted records. It also runs every adversarial/recovery case through the installed artifact. Provider credentials and port 8081 are not used.

Two regenerations have the same file set and canonical content after excluding only approval latency totals and wall-clock measurements. The manifest names those three timing fields and their bounded comparison tolerance. Published evidence omits temporary state, approval identities, prompts, tool arguments, provider payloads, and credentials.

Limits

These are deterministic fixtures against a local recording endpoint. They answer whether the documented governance and recovery contracts held for this artifact. They do not establish how a live model chooses tools, how often real workloads finish, what a provider bills, or how Docket compares with another orchestration system. Re-run the scenarios with your own deployment and workload before making an adoption decision.