Runtime role-coherence and mission-alignment measurement
Measurement work packages for autonomy programs — government programs, primes and integrators, and test & evaluation organizations
The problem
Autonomy programs must determine not only whether a multi-agent collective completed its task, but whether individual agents remained coherent with assigned roles and aligned to mission objectives while doing so, across long interaction horizons.
Mentim contributes the runtime measurement and behavioral-evidence layer for that problem. We measure observable agent behavior using accessible agentic-scaffolding telemetry — memory, tool-use constraints, stateful role assignment, and message and interaction records — to characterize role divergence, mission-alignment drift, and longitudinal coordination condition. No access to model weights or latent activations is required for the black-box measurement path.
Boundary of contribution
Mentim owns the role-drift-monitoring and mission-alignment-measurement components within a team’s control architecture. The program lead or controller teammate integrates those measurements with inference-time control and exposes the resulting interface and guarantees upward.
We solve a defined portion of the measurement problem. We do not displace your architecture.
Mentim brings an existing evidence base to the measurement side, rather than requiring a prime to develop and validate that component from scratch.
Current focus — DARPA DICE
Decentralized Artificial Intelligence through Controlled Emergence (DICE), BAA HR001126S0010 — Technical Area 1 / Technical Area 2 teaming. The BAA names, among Technical Area 2's deliverables, role-drift detectors and mission-alignment measurement frameworks — the measurement components of the TA2 control adaptor, and the components Mentim is built for. We are engaging with TA1/TA2 teams as a measurement work package subcontractor.
Representative work package
| Task | Deliverable |
|---|---|
| WP1 — Observation contract | Versioned telemetry / interface specification over the agreed telemetry surface |
| WP2 — Runtime measurement | Integrated monitor prototype — role-divergence and mission-alignment observables |
| WP3 — Longitudinal analysis | Versioned measurement outputs and analysis API: trajectory, persistence, recovery over long horizons |
| WP4 — Validation | Pre-registered matched-control studies of each program-specific observable: protocol, frozen evaluation set, full result report including nulls |
| WP5 — Evidence package | Inspectable run-evidence package — inputs, derivations, versions, replay materials |
| WP6 — Team integration | Versioned interface, controller-integration test results, interface-conformance report |
Evidence base — what exists today
| Artifact | Status |
|---|---|
| Mentim Ontology | Published — Zenodo, DOI 10.5281/zenodo.18707090. Five engineering primitives plus the composite Structural Accountability Index. |
| Unified Annotation Schema V1.0 | Published — Zenodo, DOI 10.5281/zenodo.18707593. Annotation grammar for behavioral evidence. |
| MP-045 | Available for prospective-teammate diligence · not yet publicly released. Matched-controls characterization study, six pressure mechanisms, 600-sample frozen dataset. Pilot-informed design with frozen comparator structure. Result: three mechanisms with sustained signal, two onset-only, one null — reported in full, including the null. |
| Replay convergence | Model-checked in TLA+/TLC. Bounded replay-convergence specification; verified property and model boundary documented. Production implementation and conformance not yet established. |
| Demonstration apparatus | Working prototype. Instrumented multi-agent simulations, prototype coordination-integrity scoring, live dashboard. |
Minimum integration assumptions
Exact required fields are architecture-dependent and settled in WP1. At minimum, the measurement layer needs:
- stable agent and run identifiers;
- declared agent roles and mission constraints;
- timestamped messages or interaction records;
- accessible scaffolding signals — memory, tool-use constraints, stateful role assignment — where the agent framework exposes them.