Understand risk and complexity

Read the reasons behind a grade and distinguish it from approval.

On this page

ArchDev helps you decide where to spend review time. A grade is a prompt to inspect the change and its evidence, not a guarantee that the code is correct or that tests and approvals can be skipped.

What the grade means

A risk assessment considers two questions:

  • Uncertainty: How much remains unproven about whether the change will work? Relevant tests and established patterns reduce uncertainty; missing integration coverage can increase it.
  • Consequence: If the change goes wrong, what can happen to users, data, access, or development? A change can be well tested and still have serious consequences if it fails.

Each question is assessed as low, medium, or high. When both have been assessed, ArchDev combines them with a fixed rule:

Uncertainty ↓ / Consequence → Low Medium High
Low Low Low Medium
Medium Low Medium High
High Low High Critical

For example, a well-tested change to a critical access path can still be medium because its consequence is high. Critical requires both high uncertainty and high consequence. The grade does not come from the number of files changed. Read the reasons, test boundaries, and limitations alongside it.

Risk assessment is not the same as a bug review. A suspected defect should be recorded as a separate concern, not silently folded into a grade. The components explain uncertainty and possible harm, not a verdict that the code is correct.

Assessment inputs

An assessment identifies its subject and version, such as a PR head or a changed range. Its evidence can include:

  • The objective, description, diff or proposed approach.
  • Relevant source definitions, callers and tests available to that assessment path.
  • Available check results and observed executions, distinguished from promised tests.
  • Related work and team lessons retrieved for the assessment.

A model or author judges uncertainty and consequence against written definitions and cites the evidence used. The combining rule is software-owned; the model does not freely choose the overall risk label. Missing inputs, limitations and coverage should remain visible. Receiving a full diff does not mean every range was assessed.

Assessments do not all have identical context. An author-written assessment and a hosted assessor can have different inputs. Check the producer and what was actually read; a different producer is not necessarily an independent, blind review.

Complexity is a separate signal

Complexity describes how involved the work is, not how harmful a failure would be. A difficult internal tool change can have low consequence; a one-line access check can have high consequence.

Where an agent's update carries a low, medium or high complexity hint, it is the originating agent's self-assessment. It is not the sealed uncertainty/consequence calculation, a test result or a delivery-time guarantee. Ask the agent which dependencies or unfamiliar work make it complex. A missing hint is not a low score.

Grades in a review

  • Low: Nothing in the assessed change calls for extra attention beyond normal review. Still read the diff and check the tests.
  • Medium: Look closely at the indicated files or hunks and the assumptions behind them.
  • High: Review the failure path and its safeguards carefully before proceeding.
  • Critical: Both important behavior and serious potential harm remain uncertain; resolve the gaps or make a deliberate decision with the right reviewers.

The review may also show risk labels on individual changed ranges and a short Start here list of the most important places to read. A range label points to a concrete concern in that part of the diff; it is not a separate assessment of the entire pull request. You can leave feedback on the relevant lines for your agent.

No risk means a reviewed change has no notable risk flagged in that view. Not assessed (or a missing grade) means there is not enough assessment to assign one. Not assessed is not low risk. Analysis can also be pending, skipped, or unavailable; review the change yourself rather than treating an absent badge as approval.

Grades and annotations refer to a particular change or pull-request head. After new commits, inspect the latest revision rather than relying on the previous grade.

Worked example: a tested access check

This is an invented example, not an accuracy benchmark. A change adds an ownership check before an account update. Integration tests exercise permitted and denied writes through the real handler.

  • Uncertainty: low. The check follows an established pattern and the tests exercise the relevant boundary.
  • Consequence: high. An incorrect check could let a person change someone else's account access.
  • Combined risk: medium. The matrix keeps the consequence visible even with low uncertainty.

Review the check's placement and the evidence for denied writes. If a later commit changes the write path, assess that new revision; the earlier grade does not certify it. This same example is shown in the review walkthrough.

Validation and disagreement

We evaluate assessments on frozen subjects against researched reference judgments. Reference work can inspect more code, history and later outcomes than a real-time assessor has. Reviews of those references help distinguish a missing input, a definition ambiguity and an unsupported judgment.

Those investigations are not evidence available to the earlier assessor, and agreement on a selected case set is not a universal accuracy rate. Read the reasons and limitations of the individual result rather than trusting a label alone.

If you disagree, identify the revision, component and specific claim, then point to the missing or contradictory evidence. If your app offers Is this grade right?, record your response there; otherwise contact the team. Feedback does not itself mean a saved grade changed or a model was retrained.

Check a change before you merge · Choose a workflow · Risk event webhooks