LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
Abstract
Large language model (LLM) agents increasingly undertake long-horizon tasks that require sustained reasoning, tool use, and revision across many interdependent steps. However, existing agent harnesses maintain task execution, task state, and completion assessment within a growing context, making the state difficult to track and allowing incorrect self-assessments to propagate into later decisions. We reformulate long-horizon execution as a task-state management problem and propose LongHorizon-Harness, which maintains the task state explicitly outside execution and updates it only with facts independently verified from the environment. Its Manage-Execute-Audit(MEA) loop uses a manager to maintain the task state and determine the next subtask, a fresh-context executor to perform it, and a read-only auditor to verify the resulting environment state before the next round. A lightweight AgentAdapter supports interchangeable model and harness backends without modifying their native agent loops. LongHorizon-Harness improves Qwen~3.7-Plus from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench~2.1, and from 2.8% to 8.3% on OSWorld~2.0. It also raises Claude Opus~4.7 from 20.0% to 34.3% on an OSWorld2.0 subset, demonstrating consistent gains across models, harnesses, and interaction domains.
Community
LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
Github: https://github.com/AMAP-ML/LongHorizon-Harness
Website: https://lh-harness.pages.dev
Interesting approach! Separating task state from execution and verifying it independently seems like a practical way to improve reliability for long-horizon AI agents. The benchmark improvements are impressive. MyMilestoneCard login
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure (2026)
- WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces (2026)
- SEAGym: An Evaluation Environment for Self-Evolving LLM Agents (2026)
- DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations (2026)
- STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios (2026)
- AgentS4D: Benchmarking Runtime Risks across the Execution Lifecycle of LLM-Based Workspace Agents (2026)
- Beyond Global Replanning: Hierarchical Recovery for Cross-Device Agent Systems (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2608.01964 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper