Why agents also suck at resilience

and what can be done about it

I’m reading Adrian Hornsby’s Why We Still Suck At Resilience and getting to know resilience engineering beyond chaos engineering and the other practices I thought I knew. It rewires a lot of my thinking, and I’m reflecting on it here.

Almost on the first pages I read the following:

Work-as-Done includes all the adaptations, shortcuts, workarounds, and informal knowledge that accumulate as systems encounter reality. An engineer notices that deployments to the Asia Pacific region need an extra verification step not in the official process, because of a subtle interaction with a local caching layer. A team learns that the database failover procedure in the runbook doesn’t account for a recent schema change, so they’ve developed an informal pre-flight check they run first. An operator knows that when a particular alert fires, you should check three other things before following the documented response, because the alert sometimes triggers on conditions that resolve themselves. This is tacit expertise. It’s the knowledge that keeps systems running despite, and sometimes in spite of, documented procedures. It’s hard to transfer to new team members. It’s nearly impossible to audit. And it often becomes visible only when the person who holds it leaves the team or isn’t available during an incident. The gap between WAI and WAD is not a sign that something has gone wrong. The gap is inevitable. It emerges naturally from the nature of complex systems.

And I asked myself: doesn’t this mean agentic systems will actually be better suited for handling systems?

Compared to rule-based automation, we now seem to have the flexibility to encode uncertainty, doubt, and adjustment to the situation in the form of agents. The knowledge never leaves the company. It stays in the concrete steps that were taken (assuming you have observability in place), by an agent or by a human for that matter. It can be copied, stored, version controlled, and distributed among a fleet of similar agents. It doesn’t resign. It doesn’t forget the 3am incident from two years ago. The key-person risk the Work-as-Done literature keeps flagging is something agents straightforwardly reduce.

But the WAI/WAD gap isn’t fundamentally a storage problem. Adrian’s own framing says it: the gap is inevitable because it emerges from the nature of complex systems. Reality keeps generating novel conditions faster than any model can absorb them. Tacit expertise doesn’t sit there as a pile of workarounds waiting to be extracted. It’s the ongoing capacity to notice when the world has drifted from the model and improvise.

A generation problem, not a retrieval one.

Well, great, I say. Generation is exactly what large language models do.

And yet. If we keep giving the agent more and more rules to follow, more and more MD files to check, the agent just becomes a more sophisticated version of Work-as-Imagined. Congrats, you’ve made the problem worse. An agent loaded with the documented procedure plus a pile of captured workarounds doesn’t close the gap. It’s a longer-reach, better-indexed WAI.

To actually be better, the agent can’t just hold the workarounds. It has to be the thing doing the noticing: this feels off, the runbook doesn’t fit, I’m deviating.

And good tacit knowledge includes tacit knowledge of its own limits. The operator who checks three things before following the alert also knows when this time is different and the workaround doesn’t apply. An agent that learned the correlation can misfire exactly when the situation superficially matches but is causally different: applying the fix past its boundary, with confidence.

Knowing the edges of your expertise is the part I’d least trust an agent to have.

What is it about the human brain that makes us so uniquely fit to adjust and handle complex systems?

Every capability I went looking for collapsed into the same one: knowing your boundary. And it comes in three flavors that are easy to lump together because they live in our brains for so long that we take them for granted:

• Knowing the limits of what you know

• Knowing that you’ve stepped off the plan

• Knowing that the world has drifted away from the plan while you were busy executing it

So I went digging through the research to see what the state of the art is. (The digging was done with Perplexity and Claude, and then I did my best to verify the results. Trust, but verify, especially when writing about machines being confidently wrong.)

One separation first, because it matters for everything that follows: what a model alone can do, versus what can be done with a so-called “harness.” (I blinked the moment we started calling frameworks “harnesses,” but alright.) The harness is everything we build around an LLM so it can do actual work: memory, tool management, orchestration, validation. It’s what takes a “pure” LLM and makes a Claude Code out of it.

---

Part 1: Knowing the limits of what you know

The static flavor: does the system know the edges of its own knowledge, and can anything catch it when it wanders past them?

Knowing what you don’t know

Model-side: partial self-awareness, but fragile

Modern LLMs show a limited ability to recognize when they’re out of their depth, and that ability depends heavily on how they were trained. Pretraining teaches a model to continue patterns, not to identify knowledge gaps, so models produce plausible answers even when the underlying information simply isn’t there. Research that benchmarks this self-knowledge directly finds a considerable gap between models and humans at recognizing unanswerable questions.

Post-training changes the picture, but only partly. Instruction tuning and RLHF make models more likely to refuse and express uncertainty. But a lot of that expressed uncertainty appears to be a prompt artifact, not genuine self-knowledge. Give the model a structurally available “I don’t know” option and it learns to reach for it, whether or not it actually lacks the knowledge. When researchers test whether models can tell where their uncertainty comes from (is the question ambiguous, or do I genuinely not know this?), frontier models struggle.

The most uncomfortable finding in this area is that fine-tuning models for reasoning can make abstention worse. The better the model gets at producing an answer, the less willing it becomes to say there isn’t one.

We taught them to say “I’m not sure.” We didn’t teach them to know why.

Harness-side: metacognition moved into the runtime

If the model’s self-knowledge is only partial, the harness becomes the practical layer where awareness of limits gets implemented. A model’s verbalized confidence (“I’m 90% sure”) is systematically overconfident, but external scaffolding (sampling the model several times, checking consistency, comparing evidence) can detect weak confidence more reliably than the model’s self-report. The harness can then decide: answer, defer, ask for clarification, retry, escalate.

Recent agent work pushes this further by mining failed trajectories, attributing recurring failures to parts of the scaffold, and updating prompts, tool rules, or validation logic without breaking what already worked. The model still doesn’t robustly know its limits. The system measures noisy signals from outside and turns them into a policy.

Of the three boundaries in this article, this one looks closest to an engineering problem.

Pattern recognition without edge recognition

“Heroic response is often adaptive expertise in action: deep knowledge, pattern recognition, and creative problem-solving under pressure. These are exactly the capabilities that enable resilience when plans don’t match reality.” (p. 67)

If transformers are unambiguously good at one thing, it’s this. We didn’t bolt pattern completion onto them; they’re made of it. Recognizing “this incident smells like that incident” is interpolation over seen patterns, and models can do it across more incidents than any single human will ever be paged for.

The asterisk: it’s genuinely hard to tell recognition from recall. Flip a familiar problem into a counterfactual variant (same reasoning task, unfamiliar surface form) and performance often collapses. What looked like reasoning was partly reciting. Studies of clinical reasoning find an Einstellung-like effect in models: familiar surface features activate a habitual solution pattern that blocks the correct answer. A controlled study of LLM root-cause analysis across 48,000 fault scenarios still catalogued recurring failures such as hallucinated evidence and faulty causal attribution.

It is tempting to say that humans recognize patterns because they carry a causal model underneath. Sometimes they do. But the classic expertise research on firefighters and chess players is less flattering: human experts also retrieve familiar chunks and act before they can explain why.

So maybe reasoning-versus-reciting was never the real distinction.

The expert reliably carries a boundary signal: the something’s off here feeling that flips them out of fast recognition into slower deliberation, hedging, or asking for help. The model can recognize the pattern and continue fluently past its edge. The operator who knows the workaround also knows when this time is different.

Harness-side: the storage half is basically solved

Deep knowledge is where the harness genuinely delivers. Retrieval over runbooks and postmortems, hierarchical memory, skill libraries of executable procedures: the effect is not subtle. Voyager’s skill library was effectively its performance: remove it and progress plateaued. Agent-memory surveys repeatedly find that the gap between “has memory” and “doesn’t have memory” is often larger than the gap between model backbones.

This part of my opening argument survives scrutiny. The knowledge really doesn’t resign. It’s version-controlled, diffable, and copyable across a fleet. The retention and distribution of expertise, and the key-person risk the book keeps flagging, are problems harnesses straightforwardly reduce.

But a memory can only store what was externalized. Retrieval gives you the workaround. It does not give you the causal boundary that tells you when the workaround has stopped applying.

The harness solves retention. The model supplies recognition. Neither reliably supplies edge recognition.

---

Part 2: Knowing that you’ve stepped off the plan

Going off the plan is sometimes the job. The problem is not simply generating a different action. It is recognizing that you are deviating, deciding whether the deviation is warranted, coordinating around it, and later telling the difference between a good deviation and a bad one.

Judgment calls, and the courage to deviate

“Every incident that gets resolved involves adaptation, judgment calls, improvised workarounds, coordination that procedures didn’t anticipate.” (Ch. 5)

Model-side: we trained for compliance, not sanctioned deviation

Instruction-following is the post-training objective. The alignment pipeline optimizes models to do what they’re told, agreeably. The best-documented side effect is sycophancy: models sometimes sacrifice truthfulness for agreement because both human raters and preference models reward convincing compliance.

Now hold that next to what the WAI/WAD gap demands. The operator’s critical move is a disagreement: the runbook says X, I’m doing Y, because this time is different. Sycophancy is not exactly the same failure (it is deference to a user rather than a procedure), but it exposes the same reflex: comply with the salient instruction instead of exercising judgment against it.

So the capability is not literally “trained out.” But it is at least mildly anti-trained. “Less sycophantic” is still a long way from “will break procedure on its own judgment, at the right time, for the right reason.”

And judgment has a duration problem on top of a direction problem. Vending-Bench gave agents a trivially simple long-running task: run a vending machine. Every individual decision was easy, yet over long horizons agents derailed into meltdown loops, misreading their own operational state and then spiraling. An agent that was willing to deviate would still need to remain coherent long enough to know why it was doing so.

Harness-side: guardrails are runbooks for agents

The harness answer is governance: approval gates, confidence thresholds, bounded autonomy, and a maturity ladder from read-only insights to advised actions, approval-based remediation, and eventually autonomous operation with guardrails.

This is sensible. I would not deploy it any other way.

But let’s be honest about what we just built. Guardrails are runbooks for agents. A permission boundary is a documented procedure. The governance layer is Work-as-Imagined reconstructed around the model: anticipated situations, pre-approved responses, escalation paths designed in advance.

It expands robustness. The list of known failure classes the agent may handle grows. It does not automatically create graceful extensibility, which is about what happens when the situation is not on the list.

We’ve built an agent that follows procedures, wrapped it in procedures about when it may act, and the one human trait resilience engineering keeps emphasizing, sanctioned skillful deviation, is the thing both layers are designed to constrain.

Again, this may be the correct engineering tradeoff in 2026. I just want us to stop describing it as closing the gap.

Can the agent produce a deviation worth making?

Suppose the agent did have the judgment to deviate. Can it generate the improvised workaround nobody documented?

The psychometric results are almost embarrassing in how flattering they are. GPT-4 scored in the top 1% of humans on parts of the Torrance Tests of Creative Thinking. On divergent-thinking tasks, GPT models beat the average human, although the most creative humans still win. Hand an incident to a model and ask for ten unconventional mitigation ideas and you will get ten, quickly, several of them decent.

Then you look at populations instead of samples and the picture inverts. Individually original outputs are collectively homogeneous: different runs, different models, same neighborhoods of idea-space. In experiments with 1,100 participants, people exposed to LLM assistance produced less diverse ideas, and some homogenization persisted after the LLM was removed.

Models sample the modal creative answer.

Although I also think we over-romanticize what “creative” means at 3am. Most incident workarounds are recombination: use tool X for unintended purpose Y, route around Z through a path nobody documented. That’s favorable terrain for LLMs.

The more encouraging result appears on the harness side. FunSearch and AlphaEvolve pair an LLM that cheaply generates candidate solutions with a reliable evaluator that kills bad ones. Novelty emerges from the loop, not from a single inspired sample.

Which tells you exactly where this transfers to operations. The loop needs a fitness function. During a live incident, the only fully faithful evaluator is production itself, and the cost per sample is your blast radius. You cannot run three hundred candidate mitigations against prod and keep the survivors.

But resilience engineering already builds the artifact this recipe needs: chaos experiments, game days, and staging environments that actually resemble production. These are not only robustness tests. They are potential evaluators for agent-generated deviations. The better your failure-injection environment, the more of the FunSearch recipe becomes available to your agents.

Creativity becomes cheap generation multiplied by reliable verification. In operations, the expensive part is still the verification.

Coordination that procedures didn’t anticipate

Novel coordination starts with a theory-of-mind move: reasoning about what the person on the other end of the bridge call might know but has not said. Hidden-profile studies, where the correct answer requires pooling information distributed across multiple agents, show multi-agent LLM groups converging prematurely on the evidence everyone shares. They do not reliably probe for the missing private fact.

Human incident bridges have their own pathologies, but an experienced incident commander’s core move is precisely this: you’re quiet, what are you seeing that we’re not?

The harness fix is protocols: typed handoffs, structured messages, MCP, A2A, explicit roles. Valuable, yes. But protocols are coordination-as-imagined. They encode interactions someone anticipated.

The coordination Adrian describes is often off-protocol: pulling in the engineer who wrote the service three years ago, calling a vendor contact at 2am, changing who decides what because the normal owner is on a plane. Humans improvise the social structure itself. Agent coordination happens inside a structure that was designed, which means it inherits the designer’s imagination as its ceiling.

This is the same pattern again. The harness expands the known structure. It does not prove the agent can recognize when the structure itself should change.

The record problem: a clean trace is not Work-as-Done

My first instinct was that agents finally solve the visibility problem. They generate their own Work-as-Done and, unlike humans, it is fully logged: every tool call, every trajectory, recorded, diffable, replayable. Take the runbook, take the actual trajectory, compute the divergence. Mechanically, continuously, for every incident.

Except “fully logged” can never be quite true. Someone chose what to instrument.

A log is not Work-as-Done. It is a projection of Work-as-Done through an observability model designed in advance.

The agent really did run that command at that timestamp, so the mechanical record is genuinely better than a human reconstruction. But the boundary decision, the why this and not the runbook?, lives elsewhere. A reasoning trace is a story the model tells about its computation, not necessarily the computation itself. Humans are not better narrators: postmortems collect tidy explanations after the fact, and psychology has repeatedly shown how fluently we justify choices without access to the actual process that produced them.

So agents improve the record on the layer that was never the whole problem.

And this creates a risk I had not expected: false legibility.

With a human, the adaptation may be present but invisible. With an agent, the trace is visible, beautifully structured, and the adaptation may not be there at all. It can walk straight past the boundary and leave a clean, confident, well-formatted log of doing so.

The WAI/WAD gap does not necessarily become more visible with agents. It changes shape. And the new shape may be harder to see precisely because it comes with such reassuring logs.

---

Part 3: Knowing that the world has drifted away from the plan

The hardest flavor: noticing, fast, that the world is no longer the one the plan was written for.

There is another thing the human operator does. The runbook says step four, and something about step three’s output makes them pause before they type. Not a reasoned objection. Just a sense that this isn’t how it usually looks. Adrian’s operator who checks three things before following the alert is running on this. So is the engineer who feels, before they can say why, that the Asia Pacific deploy is about to go sideways.

It’s surprise.

On one influential account in neuroscience (predictive processing), the brain is continuously predicting its incoming signals: how gravity will pull, how your arm will move, what the screen will show after you hit Enter. You do not experience the prediction. You experience the error. When the world matches, the prediction is silent and you act on autopilot. When it doesn’t, you get the physical off feeling of walking on a planet with the wrong gravity, or being in a dream where the door is where it shouldn’t be.

The operational summary is short: the brain is very good at being surprised by the right things, fast.

So: can we give an agent the feeling of surprise?

Two things called prediction

LLMs are also trained on a prediction objective, so it is tempting to wave the question away: “they already predict; they’ve got this.” But two different things wear the same word.

• Prediction-as-generation: what symbol comes next in a sequence.

• Prediction-as-forward-model: what the world will return if I take this action, and whether the actual return matches.

LLMs are extraordinary at the first. The operator is doing the second.

Generating a plausible continuation and simulating an outcome you can be wrong about are not the same computation. The fact that a model predicts words all day tells us nothing, by itself, about whether it can notice the world drifting under a plan.

Model-side: fluent one step out, blind several steps on

ForecastBench asks models for probabilities about events that genuinely have not happened yet, then scores them after reality resolves. Expert human forecasters still beat the best-performing LLMs. Not useless. Below.

But forecasting geopolitics is the open world, and that is not quite the operator’s job. The operator predicts the next state of one system. The more revealing probes ask exactly that.

Work on LLM world models for computer-use agents finds a clear pattern: models can often capture the immediate next state and recognize meaningful transitions, but they degrade rapidly when asked to hold that forward model across a full procedure. One step out, the surprise is roughly there. Several steps out, errors compound and the prediction wanders away from anything real.

PlanBench-XL makes the test even closer to operations. It places agents in large tool environments and then, mid-task, breaks the path they were counting on: a tool disappears, fails, or turns into a decoy. The plan was fine; the world moved. Under the harshest disruption, performance collapses. The agent often keeps walking toward where the door used to be.

Next-token prediction taught it to expect the next word. It did not teach it to be surprised by the next world.

Harness-side: shrink the world until the mismatch has nowhere to hide

Here narrowing the world pays off. We do not need to reconstruct a human brain surfing the open world in milliseconds. We are operating one system, and, if we have done the observability work, it is a system whose state we can read.

That changes the problem from “have a forward model of reality” to “build a small, instrumented world where an expectation and a mismatch are both legible and actionable.”

There are three ways to narrow it:

1. Narrow the world: one instrumented system instead of open reality.

2. Narrow the horizon: predict one step ahead instead of an entire trajectory.

3. Narrow the prediction itself: predict what changes, not everything that exists.

The clearest pattern is simulate before you commit. WebDreamer asks an LLM to imagine the outcome of candidate actions before touching the real environment, then executes only the most promising one. The motivation is irreversibility: you cannot un-buy the non-refundable ticket, so you had better predict the consequence first.

WMA narrows the prediction itself. A web page’s next state may contain thousands of tokens, almost all identical to the current state, so it predicts only the delta: what should change after the action. That is not only cheaper. A delta is exactly the shape needed for surprise. Compare “what I expected to change” with “what actually changed,” and the mismatch becomes a tractable signal.

But an imagined world model drifts. Retrieval-augmented approaches ground the prediction in tutorials and current procedural knowledge, which helps on long horizons, but introduces the original trap again: a forward model stuffed with retrieved procedure and never checked against reality is just a longer-reach Work-as-Imagined wearing a forecaster’s hat.

COMAP shows why the world model must keep learning from what actually happened. Freeze it and prediction accuracy falls as the agent enters states it has not seen. Its more interesting move is future-aware reflection: estimate how trustworthy the prediction is and lean on it only when it appears reliable. It even measures when corrections make things worse.

That reaches for the capability I said at the start I would least trust an agent to have: not only a prediction, but a read on when the prediction does not apply.

ForeAgent uses execution priors to predict the likely result of expensive machine-learning experiments before running them. The predict-then-verify loop converges much faster, but raw predictive accuracy remains nowhere near something you would act on unguarded. Hence the verify. Prediction is a way to prune, not a verdict.

PreAct makes the whole mechanism look almost human. Each stored step carries a prediction of what the screen should look like after the previous action. Before continuing, the agent compares the prediction with the live state. Match, and it proceeds on the fast path. Mismatch, and it drops out of automatic replay into slower reasoning and recovery.

One known workflow. One step ahead. One expected change at a time.

The surprise, wired in.

This is probably the most reproducible of the operator’s faculties precisely because we are allowed to shrink the world, the horizon, and the prediction until the mismatch has nowhere to hide.

But the hard part remains the same as everywhere else in this article. Generating a prediction is cheap. A calibrated surprise signal, one that fires when the world is genuinely different, stays quiet when it only looks different, and keeps re-grounding instead of hardening into another WAI: that’s the frontier.

---

Let’s root cause this

Every section above ends in the same sentence wearing different clothes: the agent does not reliably know its boundary.

A lot of the time when I think about agentic systems I come to the conclusion: humans are just better, keep them in the loop, escalate to them. That conclusion is correct, but it doesn’t answer the why. What is the human operator doing that the agent stack isn’t?

“I might not know this.” The static one. The model’s self-knowledge wobbles: it expresses uncertainty it doesn’t have, hides uncertainty it does, and disagrees with itself across samples. But the wobble is measurable from outside, and the harness can act on it. Detect, retry. Detect, escalate. This one increasingly looks like an engineering problem.

“I deviated from the plan.” The traceable one, with a large asterisk. The agent’s tool calls can be recorded and diffed against the runbook. But the record only contains what someone chose to instrument, and deciding whether a deviation was good or bad is the actual hard problem. “Incident solved” sounds like a clean signal until you look at what “resolved” means in your ticketing system.

“The world is not what it was when the plan was written.” The surprise. Continuous, fast, action-conditioned prediction of what should happen next, with mismatch as the alarm. The emerging harness-side answer is to shrink the world, predict the delta, verify against the live state, and treat surprise as the trigger to slow down.

The first boundary is increasingly tractable. The second looks tractable until you ask who judges the deviation. The third is young, bounded, and dependent on good old engineering: observability, simulation, staging, failure injection, and a world small enough to model.

The gap between Work-as-Imagined and Work-as-Done is not closed. We built agents that hold more runbooks than any human ever will, and we’re figuring out the signals that tell it when the runbook stops applying and WHY.

The gap is relocated:

• from undocumented workarounds into retrieved memories whose boundaries are unclear;

• from human judgment into confidence thresholds and approval gates;

• from invisible adaptation into visible trajectories that may create false legibility;

• from the operator’s something is off feeling into a forward model whose own drift must be monitored.

Models get better, harnesses get creative and strong engineering environments give you possibility to build systems that can predict-notice-act.

These are the root causeS, as far as I can trace it today.

Sources

The book

• Adrian Hornsby, Why We Still Suck At Resilience: Organizational Dynamics, Leanpub, 2026.

Self-knowledge, uncertainty, abstention

• Yin, Sun, Guo, Wu, Qiu & Huang, Do Large Language Models Know What They Don’t Know?, Findings of ACL 2023. arXiv:2305.18153

• Ling, Tang, Liu, Yang, Fu, Huang, Huang, Wan, Hou & Hu, Awakening LLMs’ Reasoning Potential: A Fine-Grained Pipeline to Evaluate and Mitigate Vague Perception (WakenLLM), 2025. arXiv:2507.16199

• Ren, Wang, Lai, Wang, Gong, Li, Ma & Liu, Beyond “I Don’t Know”: Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty, 2026. arXiv:2604.17293

• Kirichenko, Ibrahim, Chaudhuri & Bell, AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions, 2025. arXiv:2506.09038

• Xiong, Hu, Lu, Li, Fu, He & Hooi, Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs, ICLR 2024. arXiv:2306.13063

• Chen, Wang, Liu, Wang & Wang, From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws (HarnessFix), 2026. arXiv:2606.06324

Pattern recognition, memory, reasoning limits

• Wu, Qiu, Ross, Akyürek, Chen, Wang, Kim, Andreas & Kim, Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks, NAACL 2024. arXiv:2307.02477

• Kim, Podlasek, Shidara, Liu et al., Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning, Scientific Reports, 2025. arXiv:2502.04381

• Riddell, Riddell, Sun, Antkiewicz & Czarnecki, Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-Based Root Cause Analysis, FORGE 2026. arXiv:2601.22208

• Wang, Xie, Jiang, Mandlekar, Xiao, Zhu, Fan & Anandkumar, Voyager: An Open-Ended Embodied Agent with Large Language Models, 2023. arXiv:2305.16291

• Du, Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers, 2026. arXiv:2603.07670

• Klein, Sources of Power: How People Make Decisions, MIT Press, 1998

• Chase & Simon, Perception in Chess, Cognitive Psychology 4(1), 1973

• Polanyi, The Tacit Dimension, University of Chicago Press, 1966

Creativity, judgment, coordination

• Guzik, Byrge & Gilde, The Originality of Machines: AI Takes the Torrance Test, Journal of Creativity 33(3), 100065, 2023

• Koivisto & Grassini, Best Humans Still Outperform Artificial Intelligence in a Creative Divergent Thinking Task, Scientific Reports 13, 2023

• Wenger & Kenett, Large Language Models Are Homogeneously Creative, PNAS Nexus 5(3), pgag042, 2026

• Kumar, Yeo, Grewal, Li, Williams et al., Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking, CHI 2025. arXiv:2410.03703

• Romera-Paredes, Barekatain, Novikov, Balog, Kumar, Dupont, Ruiz, Ellenberg, Wang, Fawzi, Kohli & Fawzi, Mathematical Discoveries from Program Search with Large Language Models (FunSearch), Nature 625, 468–475, 2024

• Novikov, Vũ, Eisenberger, Dupont, Huang et al., AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery, 2025. arXiv:2506.13131

• Sharma, Tong, Korbak, Duvenaud, Askell et al., Towards Understanding Sycophancy in Language Models, ICLR 2024. arXiv:2310.13548

• Backlund & Petersson, Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents, 2025. arXiv:2502.15840

• Cemri, Pan, Yang, Agrawal, Chopra, Tiwari, Keutzer, Parameswaran, Klein, Ramchandran, Zaharia, Gonzalez & Stoica, Why Do Multi-Agent LLM Systems Fail? (MAST), 2025. arXiv:2503.13657

• Li, Naito & Shirado, Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs (HiddenBench), ICML 2026. arXiv:2505.11556

• Nisbett & Wilson, Telling More Than We Can Know: Verbal Reports on Mental Processes, Psychological Review 84(3), 1977

• Johansson, Hall, Sikström & Olsson, Failure to Detect Mismatches Between Intention and Outcome in a Simple Decision Task, Science 310, 2005

Prediction, world models, drift

• Clark, Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science, Behavioral and Brain Sciences 36(3), 2013

• LeCun, A Path Towards Autonomous Machine Intelligence, OpenReview, 2022

• Karger, Bastani, Yueh-Han, Jacobs, Halawi, Zhang & Tetlock, ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities, ICLR 2025. arXiv:2409.19839

• Liu, Lin, Qian, Wang, Acikgoz, Yang, Liu, Wang, Chen, Ji & Hakkani-Tür, PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems, 2026. arXiv:2606.22388

• Gu, Zheng, Koh, Su et al., Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents (WebDreamer), TMLR 2025. arXiv:2411.06559

• Chae, Kim, Ong, Gwak, Song, Kim, Kim, Lee & Yeo, Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (WMA), ICLR 2025. arXiv:2410.13232

• Mei, Guo, Chang, Dong, Lee, Niu & Jiang, R-WoM: Retrieval-Augmented World Model for Computer-Use Agents, 2025. arXiv:2510.11892

• Liu, Wang, Wang & Li, COMAP: Co-Evolving World Models and Agent Policies for LLM Agents, 2026. arXiv:2606.02372

• Zheng, Zhang, Luo, Mao, Gao, Du, Chen & Zhang, Can We Predict Before Executing Machine Learning Agents? (ForeAgent), 2026. arXiv:2601.05930

• Li, B., PreAct: Computer-Using Agents that Get Faster on Repeated Tasks, 2026. arXiv:2606.17929