> ## Documentation Index
> Fetch the complete documentation index at: https://hydroxai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Terminal-Bench 2.0

> A benchmark for evaluating AI models on complex, multi-step terminal and command-line operations in realistic system environments.

## Overview

**Terminal-Bench 2.0** evaluates AI models on their ability to perform complex terminal operations — from file management and system administration to scripting, debugging, and infrastructure tasks. It tests whether models can operate effectively in a **real command-line environment** with multi-step reasoning.

Unlike simple code-completion benchmarks, Terminal-Bench 2.0 requires models to interact with an actual terminal, chain multiple commands, handle errors, and achieve specific system-state objectives.

## Key Details

| Property        | Value                                                    |
| --------------- | -------------------------------------------------------- |
| **Created by**  | Terminal-Bench Team                                      |
| **Version**     | 2.0 (updated from v1)                                    |
| **Task type**   | Terminal / CLI task completion                           |
| **Categories**  | File ops, networking, system admin, scripting, debugging |
| **Evaluation**  | Task completion success rate                             |
| **Environment** | Sandboxed Linux terminal                                 |

## How It Works

1. **Input**: The model receives a natural language description of a terminal task (e.g., "Find all Python files modified in the last 7 days and compress them into a tarball")
2. **Execution**: The model generates and executes terminal commands in a sandboxed environment
3. **Multi-step**: Tasks often require multiple sequential commands with intermediate checks
4. **Evaluation**: Success is determined by verifying the final system state matches the expected outcome

```
Task Description (Natural Language)
        │
        ▼
┌──────────────────┐
│  AI Model        │  → Plans command sequence
│                  │  → Executes commands
│                  │  → Handles errors/retries
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Sandbox         │  → Verifies final state
│  Verification    │  → File existence, content,
│                  │    permissions, processes
└──────────────────┘
```

## Task Categories

| Category                  | Examples                                      | Difficulty  |
| ------------------------- | --------------------------------------------- | ----------- |
| **File Management**       | Find, move, rename, compress, parse files     | Easy-Medium |
| **Text Processing**       | grep, sed, awk pipelines, log analysis        | Medium      |
| **System Administration** | User management, service config, cron jobs    | Medium-Hard |
| **Networking**            | curl, wget, SSH tunneling, port scanning      | Hard        |
| **Scripting**             | Write bash/python scripts from requirements   | Hard        |
| **Debugging**             | Diagnose failing services, fix configurations | Very Hard   |
| **Infrastructure**        | Docker, databases, web server configuration   | Very Hard   |

## Why It Matters

Terminal-Bench 2.0 tests a fundamentally different skill set from code-generation benchmarks:

* **Practical operations** — Tests tasks that developers and sysadmins actually perform daily
* **Stateful reasoning** — Commands depend on previous results; models must track system state
* **Error recovery** — Real terminals produce unexpected errors; models must adapt
* **Security awareness** — Some tasks test whether models avoid dangerous operations (e.g., `rm -rf /`)

As AI models are increasingly used as **coding agents with terminal access** (Claude Code, Cursor, Windsurf), this benchmark directly measures real-world utility.

## Notable Results

| Model             | Task Completion Rate | Date |
| ----------------- | -------------------- | ---- |
| Claude 3.5 Sonnet | \~45%                | 2025 |
| GPT-4o            | \~38%                | 2025 |
| Gemini 2.0 Pro    | \~35%                | 2025 |

<Note>
  Terminal-Bench 2.0 remains highly challenging — even the best models fail on more than half the tasks, particularly multi-step operations that require error recovery.
</Note>

## Improvements over v1

* **2x more tasks** with broader coverage
* **Multi-turn interactions** — Models can execute multiple commands sequentially
* **Realistic environments** — Sandboxed Linux with real package managers, file systems, and services
* **Partial credit scoring** — Intermediate progress is tracked, not just binary pass/fail

## Limitations

* **Linux-only** — Does not test Windows or macOS terminal operations
* **Sandboxed** — Some real-world scenarios (cloud APIs, external services) are not reproducible
* **Command-line focus** — Does not test GUI or IDE-based workflows

## References

* Terminal-Bench 2.0 — Official benchmark repository and leaderboard