AI governance products increasingly claim runtime visibility. Buyers should separate four capabilities: observing activity, detecting a policy condition, deciding a response and enforcing that response in the system where the action occurs.
Four levels of runtime capability
- Telemetry: capture prompts, outputs, tool calls, identities and context.
- Detection: identify policy violations, anomalous behaviour or sensitive data.
- Decision: allow, deny, redact, constrain, require approval or escalate.
- Enforcement: make the chosen response effective at the model, gateway, identity, tool or workflow layer.
A supplier may be strong at one level and depend on integrations for another. That is not inherently a weakness, but the boundary must be explicit.
Ask the supplier to show the exact path from a risky agent tool call to a blocked or approved outcome, including latency, fallback behaviour and retained evidence.
Put controls where authority is exercised
Prompt filters cannot govern every consequence. If an agent can change a record, send a payment or deploy code, control may also be needed in identity, transaction, application or workflow systems. The right architecture often combines AI-specific context with established enforcement points.
Design for failure
Define what happens if the policy engine, model or integration is unavailable. High-impact actions may need fail-closed behaviour; lower-risk work may continue with logging and later review. Operators also need a clear path for false positives and emergency override.
Verify the outcome
Runtime governance is not complete when an alert is generated. Evidence should show the triggering context, policy version, decision, enforcement result and final state. That chain supports incident response, assurance and future tuning.