Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

ADR-0021: Discrete risk signals

Note

Architecture decision: accepted.

Context and problem statement

Some actions are riskier than others: a bulk deletion, reading a secret path, or running an opaque script. The engine must escalate these to pending for human review. But a numeric risk score is non-deterministic and can be gamed. The project needed a risk model that is deterministic and explainable.

Decision drivers

  • Risk assessment must be deterministic (see ADR-0009).
  • Operators must be able to understand why a signal fired.
  • Risk must never be used to allow an action that would otherwise be denied.

Considered options

  1. Discrete, deterministic risk signals (named properties) that escalate to pending.
  2. A numeric risk score where higher scores escalate.
  3. A machine-learning model that classifies risk.

Decision outcome

Chosen option: “Discrete, deterministic risk signals”, because a named property is reproducible and explainable.

  • Risk signals are discrete, deterministic properties of an action request, such as bulk_delete or secret_path.
  • A signal escalates the decision to pending unless a matching grant already covers it.
  • Signals never allow; they can only restrict or escalate.
  • There are no numeric risk scores.

Consequences

  • Good, because each signal is reproducible and auditable.
  • Good, because operators can see exactly which signal fired and why.
  • Bad, because discrete signals cannot capture subtle gradations of risk.

Confirmation

Pros and cons of the options

Discrete risk signals

  • Good, because they are deterministic and explainable.
  • Bad, because they cannot express gradations.

Numeric risk scoring

  • Good, because it can capture nuance.
  • Bad, because it conflicts with determinism and can be gamed.

Machine-learning risk model

  • Good, because it can detect subtle patterns.
  • Bad, because it is non-deterministic, unexplainable, and trainable by attackers.

More information

Each risk signal produces a reason code starting with pending. (for example, pending.opaque in How Shiin works). The full catalog is in the risk signals specification.