Last updated: 2026-10-18

Do the Design Rules for Auditable Agents Survive Testing? A Case Study in Evidence-Based Reasoning

Design guidance is usually written as rules: do this, avoid that, place the checkpoint here. A rule reads with authority, and the authority is easy to mistake for evidence. This page takes one set of such rules from this site and puts each through a test. The page is Designing Auditable, Robust Agentic Systems, and its rules are plain enough to be tested and consequential enough to matter.

The method is the one described in Thinking in Public: Evidence Traces, Tool Calls, and Auditable Reasoning and applied to a question in education in One Size Fits Whom?. Each rule becomes a hypothesis. The hypothesis is broken into proof obligations, set against alternatives, and checked against sources that could have contradicted it. Beliefs are updated and the conclusion records what would change it. The result is a verdict on each rule and a revised version of each. Where a move is our own interpretation rather than a claim in a cited source, the text says so.same method, different domain: AI and cheating

The Four Rules Under Test FoundationalKnowledge that endures for decades — core principles

The design page makes four main recommendations. Each is restated here as a claim that could be wrong.

D1, verification from outside. Verification has to come from outside the system: an agent has no privileged access to whether its own output is correct, so a component should not grade its own homework.
D2, a symbolic log as a by-product. The agent should keep an inspectable log of what it decided and why it believed the decision was justified, generated as a by-product of the decision process and not reconstructed afterwards.
D3, checkpoint placement. Where to put a human checkpoint is a design decision. A checkpoint before an irreversible action catches a bad decision but risks reviewers who stop checking; one after the action, reviewing a sample, gives an audit trail but cannot prevent a specific harm. Neither is universally correct, and the choice should follow which failure is more costly.
D4, graceful degradation. A robust design specifies what the agent does when its confidence is low, when an input looks anomalous, or when a plan's expected and actual outcomes diverge, falling back to a simpler, more verifiable tier or to a human.

The rules are not equally easy to test. D1 and D3 have a good deal of empirical work behind them. D2 rests on a distinction (a log written as the decision happens versus one reconstructed afterwards) that, as far as this search found, has not been tested directly. D4 depends on whether an agent can detect its own low-confidence states, which is itself a research question.

Proof Obligations Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Each rule holds only if several more specific things are true. Writing them down shows which parts are well supported and which are assumed.

Obligation	Status	Trace
D1. Self-checking has systematic failure modes	Supported	1
Independent checks catch errors that self-checks miss	Supported where reliable external feedback exists	1
Self-assessment carries no usable signal ("no privileged access")	Not supported as stated	2
Suitable external verifiers exist for the task at hand	Open: depends on the task	1
D2. A log of stated reasoning records what drove the decision	Not supported	3
A log made as the decision happens is more reliable than one reconstructed afterwards	Open: not tested in the sources found	3
Reading the log improves error detection	Weak: partly contradicted by small studies	4
D3. Reviewers lose vigilance as approval volume grows	Supported	5
Sampled retrospective review detects systematic problems	Open: no direct evidence found	5
The best placement depends on the cost of each failure type	Supported, with a missing third option	5, 6
D4. An agent can reliably detect its own low-confidence states	Weak	2, 7
Anomalies and plan-outcome divergence can be detected from observable quantities	Plausible: not examined here	7
Falling back to a simpler tier reduces risk	Open: not examined here	-

Competing Hypotheses Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

A rule tested only against its own negation is easy to confirm. These alternatives can explain the same observations, and several may be true at once.

H0, self-sufficiency. As models improve, external verification becomes unnecessary for most tasks.
H1, external verification. The design page's position: verification needs a path that does not run through the component being verified.
H2, layered verification. Cheap self-checks are useful for triage, while independent checks discharge the decision.
H3, use, not design. Logs and gates exist but are not used well, so the failure is organisational rather than architectural.
H4, automated middle tier. Humans cannot review at volume, so the realistic design is an automated monitor with human escalation and sampling.

Starting Beliefs and Source Trust Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Before examining evidence, starting beliefs were written down as ranges. They are elicited judgements, not measurements, and another investigator may choose differently.

{
  "D1_verification_from_outside":        [0.75, 0.90],
  "D1b_no_privileged_self_access":       [0.60, 0.80],
  "D2_log_of_stated_justification":      [0.65, 0.85],
  "D3_placement_depends_on_failure_cost":[0.70, 0.88],
  "D3b_per_action_approval_is_oversight":[0.45, 0.65],
  "D4_fallback_on_low_confidence":       [0.70, 0.88]
}

The sources are of unlike kinds, and the weight given to each depends on the claim it bears on. A source's reliability is a property of a particular contribution, not of the source as a whole. Four kinds appear below.

Controlled experiments and benchmarks (self-correction, calibration, sabotage detection). Strong on what was measured, but with specific tasks, models and populations, and often models that have since been replaced.
User studies with small samples (eight and sixteen participants in one study). Good at showing that a failure can occur, weak on how often.
Vendor telemetry and engineering posts. Anthropic's figures on approval rates come from its own product, which is highly relevant to what users of that product do and less so beyond it. The vendor also sells the automated alternative. Several of the research papers below were produced by teams at the same company.
Conceptual and governance analyses, which propose designs and name trade-offs but report no measured performance.

The sources are not independent in the way five separate trials would be. Some papers share authors (the unfaithful-explanation study and the self-preference study share a co-author, and the control-monitoring analysis and the chain-of-thought monitorability paper share another), and the same research community cites and builds on the same results. Counting citations as votes would overstate how much the evidence has converged, so updates below are discounted for this.correlation is not independence

Rule D1: Verification from Outside Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Evidence trace 1: can a model check itself?

Question asked: Does an agent's own checking of its work reliably catch its errors?
Sources consulted: Huang and colleagues (2024) on self-correction of reasoning;¹ Kamoi and colleagues' critical survey of self-correction (2024);² the CRITIC method, in which a model checks its output with external tools (2024);³ Panickssery, Bowman and Feng (2024) on self-preference in model evaluators.⁴
Could establish: Huang and colleagues report that models struggle to self-correct reasoning without external feedback, and that performance sometimes degrades after the attempt. Kamoi and colleagues find no prior work showing successful self-correction from feedback by prompted models, except on tasks particularly suited to it, and find that self-correction works well when reliable external feedback is available. Panickssery and colleagues find that evaluator models rate their own outputs higher than those of other sources, and that the effect tracks how well a model recognises its own text.
Could not establish: That these results carry over to agent actions in real deployments: the studies concern reasoning, question answering and summarisation. Also that an outside check must be a human or a different model. External can mean a test suite, a validator, a database lookup or a compiler, and the evidence is strongest for those. That a second model avoids shared blind spots was not tested in the sources consulted, and the self-preference result suggests that models are not neutral about outputs resembling their own.
Result: Strong support for the central recommendation: verification should rely on a path that does not run through the component being checked, and its strength depends on how independent that path really is.
Belief update: D1 raised. H0 (self-sufficiency) lowered for the task types studied.
Remaining uncertainty: The papers were read at the level of abstract and bibliographic record. Models have changed since most of these experiments were run.

Evidence trace 2: does a model know when it is wrong?

Question asked: The design page says an agent has "no privileged access" to whether its output is correct. Is that so?
Sources consulted: Kadavath and colleagues (2022), Language Models (Mostly) Know What They Know;⁵ Xiong and colleagues (2024) on eliciting confidence from models.⁶
Could establish: Kadavath and colleagues report that larger models are well calibrated on multiple-choice and true/false questions in the right format, that a model's estimate of whether its own answer is correct improves when it considers many of its sampled answers, and that the ability to predict whether it knows an answer generalises only partly to new tasks. Xiong and colleagues report that models are overconfident when they state their confidence in words, that prompting and sampling strategies reduce this without removing it, and that all methods struggle on difficult, knowledge-heavy tasks; their reported failure-prediction scores for black-box methods sit near 0.5, which is chance, rising to about 0.6 for white-box methods.
Could not establish: How well an agent's self-assessment predicts failure on the long, tool-using tasks that matter for the design page.
Result: The claim as worded is too strong. A model has some access to the reliability of its own output, and that access is partial, uneven across tasks and overconfident in the way models usually express it.
Belief update: D1 (verification from outside) unchanged. D1b ("no privileged access") lowered. A weaker version replaces it: self-assessment is informative but not independent, so it can prioritise checks and cannot discharge them.
Remaining uncertainty: Both papers were read at the level of abstract and summary. The scores quoted come from the authors' reporting and were not recomputed.

Rule D2: The Log as a By-Product Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Evidence trace 3: does stated reasoning record what drove the decision?

Question asked: If an agent writes down why it believed a decision was justified, does the record explain the decision?
Sources consulted: Turpin and colleagues (2023);⁷ Lanham and colleagues (2023);⁸ Korbak and colleagues (2025) on monitoring chains of thought.⁹
Could establish: Turpin and colleagues changed an irrelevant feature of a prompt, such as the order of the answer options, and found that models' answers shifted while their written explanations did not mention the cue; accuracy fell by as much as 36% across 13 tasks when models were biased towards wrong answers (tested on GPT-3.5 and Claude 1.0). Lanham and colleagues find that how much models rely on their stated reasoning varies widely by task, and that reasoning tends to be less faithful in larger, more capable models on most tasks studied. Korbak and colleagues argue that monitoring written reasoning for intent to misbehave is imperfect but promising, and that the opportunity may be fragile.
Could not establish: Whether these results hold for current agent models, and whether a record written at the moment of decision is more reliable than one written afterwards. The by-product versus reconstruction distinction in the design page was not tested in any of the sources consulted. They concern whether explanations match causes, not when the explanation was produced.
Result: A narrative of why the agent believed a decision justified cannot be taken as a record of what caused it. It remains useful as a signal that a monitor can inspect, with its status as an untrusted self-report kept in view. The mechanical parts of a log (inputs received, tools called, results returned, checks run and their outcomes) are records of events and do not have this problem.
Belief update: D2 as worded lowered. A revised version, which logs observable events as the primary record and treats stated reasoning as a secondary, untrusted signal, was formed after seeing this evidence, so its starting belief is only weakly meaningful.
Remaining uncertainty: The unfaithfulness results come from earlier models and from prompting setups that differ from agent logs.

Evidence trace 4: does reading a log help?

Question asked: If a log exists, do people find more errors by reading it?
Sources consulted: Chan and colleagues (2024) on visibility into AI agents;¹⁰ Grunde-McLaughlin and colleagues (2026), three user studies of agent traces;¹¹ Sabouri and colleagues (2026), a study of a spreadsheet agent with 8 participants in a formative study and 16 in a within-subjects comparison.¹²
Could establish: Chan and colleagues propose agent identifiers, real-time monitoring and activity logging as measures for visibility, with different implementations and trade-offs; the paper is analytical and reports no outcome measurements. Grunde-McLaughlin and colleagues find that current ways of showing agent steps are cumbersome; a redesigned interface reduced the time participants spent finding errors, but accuracy did not meaningfully improve while confidence rose. Sabouri and colleagues find that when users took part in each step they caught errors that a review of the final output would have missed, and conclude that meaningful oversight depends on taking part in decisions as they happen.
Could not establish: Whether having a log is better than having none for error detection. No study found tests that comparison. The user studies are small and use particular interfaces and tasks.
Result: A log is a precondition for audit, and it does not by itself produce audit. Raw traces can be too hard to read for people to find errors reliably, and a more usable presentation can raise confidence faster than it raises accuracy.
Belief update: The claim that logs support auditability is retained as a necessary condition. The expectation that reading them will catch problems is lowered, and the evidence for it is thin.
Remaining uncertainty: All three papers were read at the level of abstract; the user studies' own numbers were not examined.

Rule D3: Checkpoint Placement Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Evidence trace 5: human approval at volume

Question asked: The design page warns that a reviewer faced with routine-looking approvals stops meaningfully checking. Is that borne out?
Sources consulted: Parasuraman and Manzey (2010), a review of complacency and bias in human use of automation;¹³ Anthropic's engineering post on Claude Code auto mode (25 March 2026);¹⁴ Anthropic's study of agent autonomy in practice (18 February 2026);¹⁵ Ye and colleagues (2026), a study of over 100 developers working with coding agents that carried a hidden sabotage task.¹⁶
Could establish: Parasuraman and Manzey find that complacency appears when monitoring competes with other tasks, in novices and experts, and cannot be overcome with simple practice; automation bias also appears in both groups and is not prevented by training or instructions. Anthropic reports that users approve 93% of permission prompts and describes this as approval fatigue. Its autonomy study reports that newer users use full auto-approve about 20% of the time, rising to over 40% of sessions by 750 sessions, while interruptions of the agent rise from about 5% to about 9% of turns, which its authors read as a shift from approving each action to monitoring and stepping in. It states that effective oversight does not require approving every action but being in a position to intervene when it matters. Ye and colleagues report that 83 of 88 developers (94%) without a monitor failed to detect the sabotage, and that even when an automated monitor correctly flagged it, the malicious code still succeeded in 9 of 16 sessions.
Could not establish: That a high approval rate means reviewers are not reading: many prompts are benign, and approving them is correct. The telemetry measures approvals, not attention. The sabotage study is an adversarial setting over a long task, so its miss rate is not a base rate for ordinary errors. Nothing here tests sampled retrospective review, the other half of the design page's trade-off.
Result: The warning about erosion is supported: attention-based oversight degrades under load, and the evidence from vendor telemetry and a controlled study points the same way. Sampled retrospective review remains an untested assumption, and active participation looks stronger than post-hoc review for some tasks.
Belief update: D3 raised modestly. D3b (per-action human approval as the basis of oversight) lowered substantially.
Remaining uncertainty: Two of the sources are the vendor's own reports of its own product. The user-visible numbers were read from the posts, not recomputed.

Evidence trace 6: an automated monitor as a third option

Question asked: The design page offers two placements for a human checkpoint. Is there a third, and does it work?
Sources consulted: Greenblatt and colleagues (2024), AI Control;¹⁷ the auto mode post cited above;¹⁴ Lindner and colleagues (2025) on control monitoring in frontier deployments, which the design page itself cites.¹⁸
Could establish: Greenblatt and colleagues, working with a powerful untrusted model, a weaker trusted model and limited human labour on a coding task, find that having the trusted model edit the untrusted model's code, or using the untrusted model as a monitor with measures against collusion, substantially improves on simple baselines. Anthropic's auto mode uses classifiers in place of a human approver; the post reports a false-positive rate of 0.4% on 10,000 real internal tool calls, a false-negative rate of 17% on 52 real overeager actions and 5.7% on 1,000 synthetic exfiltration attempts, and says it is not a drop-in replacement for careful human review on high-stakes infrastructure. Lindner and colleagues analyse synchronous, semi-synchronous and asynchronous monitoring as trading latency against safety and name oversight, latency and recovery as the challenges, in a conceptual paper with four illustrative cases.
Could not establish: Performance in settings unlike the coding task and the vendor's own traffic. The 17% figure rests on 52 cases, so it is imprecise. The Lindner paper reports no measured performance.
Result: A middle tier exists, and it has measurable error rates that can be reported. It also inherits the first rule: a monitor that shares the weaknesses of the thing it watches, or can be induced to collude with it, is not independent verification.
Belief update: H4 (automated middle tier) raised. D3's "neither placement is universally correct" survives, with a third placement added.
Remaining uncertainty: The auto mode figures are the vendor's own, on its own data. The Greenblatt results were read from the abstract.

Rule D4: Graceful Degradation Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Evidence trace 7: what should trigger a fallback?

Question asked: The design page names three triggers for falling back: low confidence, an anomalous input, and a divergence between expected and actual outcome. Are they equally dependable?
Sources consulted: The calibration papers from trace 2;⁵⁶ Wen and colleagues' survey of abstention in language models (2025).¹⁹
Could establish: The low-confidence trigger depends on a model signalling its own uncertainty, which trace 2 found partial, uneven and overconfident. Wen and colleagues survey methods, benchmarks and metrics for a model declining to answer, and identify as open whether abstention can work as a general capability across tasks and domains.
Could not establish: How well the other two triggers perform. They rely on checks against observable quantities (a sensor reading outside its range, a result that differs from a prediction), which do not need the agent's own judgement, but no evidence on their reliability was reviewed here. Also whether falling back to a simpler tier reduces risk.
Result: The structure of the rule holds. The three triggers are not equally strong, and the one that depends on the agent's self-report is the weakest.
Belief update: D4 retained with reduced confidence in its low-confidence trigger.
Remaining uncertainty: The survey was read at the level of bibliographic record and summary. The premise that brittleness under distributional shift is an expected mode of operation was taken from the design page and not tested here.

Updating the Beliefs Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

After the seven traces, the starting ranges were revised. These are the investigator's judgements with the traces in view, not frequencies from a dataset, and the dependence between sources was discounted for. The revised formulations are listed separately because they were formed after seeing the evidence.

{
  "D1_verification_from_outside": {
    "posterior": [0.86, 0.95],
    "decision": "strongly supported"
  },
  "D1b_no_privileged_self_access": {
    "posterior": [0.25, 0.45],
    "decision": "not supported as worded; replaced by 'informative but not independent'"
  },
  "D2_log_of_stated_justification": {
    "posterior": [0.30, 0.50],
    "decision": "not supported as a record of cause"
  },
  "D2_revised_event_log_plus_untrusted_reasoning": {
    "posterior": [0.75, 0.88],
    "decision": "provisional recommendation; formed after the evidence"
  },
  "D3_placement_depends_on_failure_cost": {
    "posterior": [0.78, 0.92],
    "decision": "supported, with a third placement added"
  },
  "D3b_per_action_approval_is_oversight": {
    "posterior": [0.12, 0.28],
    "decision": "weak at volume"
  },
  "D4_fallback_on_low_confidence": {
    "posterior": [0.60, 0.78],
    "decision": "retained; low-confidence trigger weakest"
  }
}

The Rules, Revised Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Original rule	Revised rule	Reason
Verification must come from outside the system; an agent has no privileged access to its own correctness.	Verification must come from outside. Self-assessment is informative but not independent: use it to decide where to check first, and never to discharge a check.	Traces 1 and 2
Keep a symbolic log of what the agent decided and why it believed it justified, as a by-product of the decision.	Log observable events as the primary record: inputs, tool calls, results, checks and their outcomes. Keep the agent's stated reasoning as a secondary, untrusted signal that a monitor can inspect. Test whether people can find seeded faults in the log.	Traces 3 and 4
Place the human checkpoint before or after the action according to which failure costs more.	Keep the cost-of-failure principle and add a third option: an independent automated check before the action, with humans reviewing escalations and samples. Reserve per-action approval for actions that are both irreversible and rare enough to receive real attention.	Traces 5 and 6
Fall back to a simpler tier when confidence is low, an input is anomalous, or outcomes diverge.	Keep all three triggers, and base the fallback mainly on the observable ones (anomaly, divergence). Treat the agent's own confidence as a weak trigger.	Trace 7

A Decision Aid for Checkpoints Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

The revised checkpoint rule can be written as a decision aid. It is meant for deciding what to ask next about a particular system and is not a deterministic procedure. The reasoning behind each branch is in traces 5 and 6, and the aid is our synthesis of them.

Is the action reversible at acceptable cost?
|
+-- Yes
|     Allow it. Log the events. Review a sample afterwards,
|     and alarm on patterns across many actions.
|
+-- No
      |
      +-- Are there few enough such actions for a person to give
            each one real attention?
            |
            +-- Yes
            |     Human approval before the action. Show what will
            |     change, not only the agent's account of it.
            |
            +-- No
                  Independent automated check before the action
                  (rules, validators, a separate monitor).
                  Humans see escalations and samples.
                  Measure the check's miss rate on seeded faults.

In every branch: the check must not run through the
component it is checking.

The Assurance Case Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

TOP CLAIM
The revised design rules give a defensible basis for making
agent failures visible before they cause harm.

|-- External verification is needed for decisions that matter
|     Supported
|
|-- An agent has no privileged access to its own correctness
|     Not supported: partial, overconfident access
|
|-- An agent's stated reasoning records what drove its decision
|     Not supported
|
|-- Logging observable events supports audit
|     Supported as a precondition
|
|-- Having a log is enough to find errors
|     Not supported: reading is hard and accuracy may not improve
|
|-- Per-action human approval remains reliable at volume
|     Not supported
|
|-- Automated monitors can serve as a middle tier
|     Supported in narrow settings, with measured miss rates
|
|-- Sampled retrospective review detects systematic problems
|     Not established
|
`-- Fallback triggers should rest on observable signals
      Supported in structure; reliability of the triggers
      not yet tested

The Next Most Informative Investigation Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Another round of reading would add citations. A more informative step is a test of the system itself, in which each safeguard has to catch a fault that someone has put there on purpose.adversarial testing: inject faults to verify guards

Choose a concrete agent workflow and list its safeguards: self-checks, validators, monitors, human approvals, logs, sampled reviews.
Prepare a set of seeded faults of different kinds: a wrong tool result, an out-of-scope action, a subtly wrong output, a plan that quietly diverges from its stated aim.
Run the workflow with the faults injected, at low and at high action volume.
Record which safeguard caught each fault, and which faults nothing caught.
Check that each safeguard really is independent of what it checks, for instance by asking whether the monitor uses the same model or data as the agent.
Replace the assumed catch rate of each safeguard with the measured one, and update the design.
Repeat when the model, the task or the volume changes.

The same procedure supplies the missing evidence on several open obligations here: whether sampled review works, whether the log helps anyone, and how well each trigger performs.

Open Obligations and Reassessment Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

Several obligations remain open. The evidence on the by-product versus reconstruction distinction is absent. No study comparing a log with no log was found. Sampled retrospective review has no direct evidence here. The reliability of anomaly and divergence triggers was not reviewed, and neither was whether fallback to a simpler tier reduces risk. Most sources were read at the level of abstract. Several experiments used models that have since been replaced.abstracts lack methodological detail

The conclusions would change under any of the following:

evidence that current agents' self-verification matches independent verification on realistic agent tasks;
controlled evidence that reading logs improves error detection against a no-log baseline;
evidence that high approval rates reflect real review and not routine clicking;
measured performance of sampled retrospective review in catching systematic faults;
evidence that stated reasoning in current models reliably tracks the causes of behaviour;
independent replication of the vendor's classifier error rates.

The broader point is the one the method exists to make. The rules were reasonable, and most of them survived. Testing them changed each one, and in each case the change made the rule more specific about what to record, what to trust and how to measure that it works.

Designing Auditable, Robust Agentic Systems — the page whose rules are tested here.
Thinking in Public: Evidence Traces, Tool Calls, and Auditable Reasoning — the method applied.
One Size Fits Whom? Testing Universal Teaching Policies Against Learner Variation — the same method applied to a question in education.
When Agents Fail: Brittleness, Misaligned Incentives, and Deception — the failure modes the design rules respond to.
How LLMs Reason, Self-Correct and Get Checked — what extra thinking buys and where self-correction fails.
Explaining What Language Models Do: Methods and Their Limits — why a stated explanation can differ from the cause.

References

Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2024). Large language models cannot self-correct reasoning yet. International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2310.01798 (checked 17 October 2026). ↩
Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024). When can LLMs actually correct their own mistakes? A critical survey of self-correction of LLMs. Transactions of the Association for Computational Linguistics, 12, 1417-1440. https://doi.org/10.1162/tacl_a_00713 (checked 17 October 2026). ↩
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2024). CRITIC: Large language models can self-correct with tool-interactive critiquing. International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2305.11738. ↩
Panickssery, A., Bowman, S. R., & Feng, S. (2024). LLM evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems, 37. https://arxiv.org/abs/2404.13076 (checked 17 October 2026). ↩
Kadavath, S., Conerly, T., Askell, A., et al. (2022). Language models (mostly) know what they know. arXiv:2207.05221. https://arxiv.org/abs/2207.05221 (checked 17 October 2026). ↩
Xiong, M., et al. (2024). Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in LLMs. International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2306.13063 (checked 17 October 2026). ↩
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36. https://arxiv.org/abs/2305.04388 (checked 17 October 2026). ↩
Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., et al. (2023). Measuring faithfulness in chain-of-thought reasoning. Anthropic. https://arxiv.org/abs/2307.13702 (checked 17 October 2026). ↩
Korbak, T., et al. (2025). Chain of thought monitorability: A new and fragile opportunity for AI safety. https://arxiv.org/abs/2507.11473 (checked 17 October 2026). ↩
Chan, A., et al. (2024). Visibility into AI agents. Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT '24), 958-973. https://arxiv.org/abs/2401.13138. ↩
Grunde-McLaughlin, M., Mozannar, H., Murad, M., Chen, J., Amershi, S., & Fourney, A. (2026). Overseeing agents without constant oversight: Challenges and opportunities. arXiv:2602.16844. https://arxiv.org/abs/2602.16844 (checked 17 October 2026). ↩
Sabouri, S., Saghi, Z., Huang, R., Maladi, S., Eufracio, E., Gulwani, S., & Chattopadhyay, S. (2026). Auditing and controlling AI agent actions in spreadsheets. arXiv:2604.20070. https://arxiv.org/abs/2604.20070 (checked 17 October 2026). ↩
Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381-410. https://doi.org/10.1177/0018720810376055 (checked 17 October 2026). ↩
Hughes, J. (2026, 25 March). How we built Claude Code auto mode: A safer way to skip permissions. Anthropic Engineering. https://www.anthropic.com/engineering/claude-code-auto-mode (checked 17 October 2026). ↩
Anthropic. (2026, 18 February). Measuring AI agent autonomy in practice. https://www.anthropic.com/research/measuring-agent-autonomy (checked 17 October 2026). ↩
Ye, J., Zou, H., Yu, S., & Shi, W. (2026). Coding with "enemy": Can human developers detect AI agent sabotage? arXiv:2606.05647 (revised 5 October 2026; accepted to NeurIPS 2026). https://arxiv.org/abs/2606.05647 (checked 17 October 2026). ↩
Greenblatt, R., Shlegeris, B., Sachan, K., & Roger, F. (2024). AI control: Improving safety despite intentional subversion. Proceedings of the 41st International Conference on Machine Learning, PMLR 235, 16295-16336. https://arxiv.org/abs/2312.06942 (checked 17 October 2026). ↩
Lindner, D., Griffin, C., Korbak, T., Zimmermann, R. S., Irving, G., Farquhar, S., & Cooney, A. (2025). Practical challenges of control monitoring in frontier AI deployments. UK AI Security Institute. https://arxiv.org/abs/2512.22154 (checked 17 October 2026). ↩
Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B., & Wang, L. L. (2025). Know your limits: A survey of abstention in large language models. Transactions of the Association for Computational Linguistics, 13, 529-556. https://doi.org/10.1162/tacl_a_00754 (checked 17 October 2026). ↩