Not when it produces a long transcript. A long-horizon agent must preserve goals and state, coordinate dependent actions, detect drift, recover from failure, and keep its evidence usable after the original context window is gone.
A task is long-horizon when success depends on a sequence of interdependent decisions whose relevant state, cost, risk, or duration exceeds what the agent can safely solve as one local inference.
The site distinguishes mechanisms, reported evidence, evaluation instruments, and research bets.
Eight interacting systems problems. Papers can belong to several.
Build a vocabulary → 02 · SynthesisDirect results are separated from cross-paper inference and hypotheses.
Inspect the claims → 03 · MeasurementWhat each benchmark observes—and what it leaves invisible.
Choose an eval → 04 · ProgramsFalsifiable directions grounded in limitations, failures, and missing controls.
Find a project →Click a mechanism to filter the source library. The boundaries are analytic, not architectural: a real system crosses all eight.
Long-horizon competence emerges from the loop and its state architecture, not from any one prompt.
The same model can behave very differently under another interface, context policy, verifier, action budget, or sandbox.
observed capability = f(model, harness, tools, environment, budget, evaluator)
Clean test: freeze the harness, tools, budget, and environment; swap only the model.
Clean test: freeze the model and task set; ablate one harness component at a time.
Agentic RL increasingly trains a model inside its deploy-time harness. A harness supplies the state and actions the model can learn; the trained model changes which scaffold components remain useful. Report both main effects and interactions with a crossed model × harness evaluation.
Open each card for the source-level evidence, then inspect the synthesis, unresolved gap, and falsifiable hypotheses.
A benchmark is an instrument. Its task unit, verifier, horizon, and hidden confounders determine what its score can mean.
Loading…
These are framed as measurable programs, not generic calls for “better planning” or “more memory.”
Estimate survival curves over consequential, dependent decisions; separate recoverable, irreversible, and silent failures.
Minimal test Hold model, harness, and domain fixed; procedurally vary dependency depth and checkpoint density.Represent assumptions and learn when new evidence invalidates only a subplan versus the global strategy.
Minimal test Inject observable, silent, and misleading blockers; measure repair cost and unnecessary plan churn.Attach source, time, confidence, dependency, and expiry to memories; test update, contradiction, and selective forgetting.
Minimal test Use the same retrieved facts but vary whether the environment changed after storage.Measure correlated blind spots and allocate checks where expected risk reduction is highest.
Minimal test Cross generator, verifier, evidence access, and checkpoint placement under a fixed verification budget.Optimize summaries for future choices, not reconstruction or semantic similarity to the discarded transcript.
Minimal test Branch future tasks after compaction so relevance cannot be inferred from one known continuation.Study when high-level tools reduce horizon and when they hide state, weaken diagnosis, or enlarge blast radius.
Minimal test Expose equivalent tasks through atomic, compositional, and macro tools with matched information.Assign value to model calls, tool choices, memory writes, recoveries, and orchestration branches without shortcut rewards.
Minimal test Replay from stored checkpoints with counterfactual actions and compare credit estimators.Measure main effects, interactions, transfer, complexity, and how quickly a scaffold's assumptions go stale.
Minimal test Evaluate several models under several frozen harnesses with identical tools, resources, and task seeds.Test agents that must stop safely, resume after external change, or transfer work to a fresh agent or human.
Minimal test Interrupt tasks at adversarial points; score resumption cost, state fidelity, and unsafe duplicate actions.tau-bench → METR Time Horizons → HORIZON → How Fast Do Agents Rot? → OSWorld 2.0
Lens: Does risk track duration, steps, dependencies, or irreversible state changes?Generative Agents → MemGPT → LongMemEval → MemoryAgentBench → Effective context engineering
Lens: What is authoritative state, what is experience, and what can be forgotten?SWE-agent → Agentless → AI Agents That Matter → Agent Lightning v1.0 → Infrastructure noise
Lens: Model, interface, orchestration, resources, evaluator—or their interaction?Paper, project site, or official engineering report; secondary summaries do not anchor claims.
Reported results stay attached to their model, harness, task, budget, and evaluator.
Direct findings, official observations, synthesis, and hypotheses carry different labels.
Agent scores are treated as system measurements unless an experiment isolates the model.
Cutoff note. The library includes first-party sources available by September 24, 2026. Very recent preprints are marked adjacent or caveated when independent replication is limited.