> ## Documentation Index
> Fetch the complete documentation index at: https://hydroxai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# ARC-AGI-2

> The premier benchmark for measuring genuine abstract reasoning and fluid intelligence in AI systems — designed to resist memorization and reward true generalization.

## Overview

**ARC-AGI-2** (Abstraction and Reasoning Corpus for Artificial General Intelligence, version 2) is the successor to François Chollet's original ARC benchmark, widely considered the most important test of **genuine reasoning ability** in AI. Each task presents a small set of input-output grid examples, and the model must infer the underlying transformation rule and apply it to a new input.

ARC-AGI-2 is specifically designed to be **unsolvable through memorization or scale alone** — it requires the kind of fluid intelligence and abstraction that humans use effortlessly but machines find extraordinarily difficult.

## Key Details

| Property         | Value                                            |
| ---------------- | ------------------------------------------------ |
| **Created by**   | François Chollet / ARC Prize Foundation          |
| **Version**      | 2.0 (evolved from ARC-AGI-1)                     |
| **Task type**    | Visual abstract reasoning (grid transformations) |
| **Dataset size** | 1,000+ tasks                                     |
| **Format**       | Input-output grid pairs → predict new output     |
| **Evaluation**   | Accuracy (exact grid match)                      |
| **Prize**        | ARC Prize — \$1M+ for significant progress       |
| **Leaderboard**  | [arcprize.org](https://arcprize.org)             |

## How It Works

Each ARC task consists of:

1. **Demonstration pairs**: 2-5 input grids and their corresponding output grids
2. **Test input**: A new grid that follows the same transformation rule
3. **Challenge**: The model must produce the exact correct output grid

```
Demo 1:  Input  → Output
Demo 2:  Input  → Output
Demo 3:  Input  → Output
         ─────────────
Test:    Input  → ???    (Model must predict this)
```

### Task Example (Conceptual)

Imagine grids where colored squares form patterns:

* Demo 1: A red L-shape → A red L-shape rotated 90°
* Demo 2: A blue T-shape → A blue T-shape rotated 90°
* Test: A green Z-shape → ??? (Must produce the Z-shape rotated 90°)

The model must infer the rule ("rotate the shape 90° clockwise") and apply it perfectly to the test input. Real ARC tasks involve much more complex and varied transformations.

## What Makes ARC-AGI-2 Unique

### Novelty by design

Every task uses a **unique transformation** not repeated elsewhere in the dataset. There is no pattern to memorize across tasks — each one is a brand-new puzzle.

### No training distribution

ARC tasks are deliberately outside any training distribution. They test **on-the-fly learning** from just a few examples, not recall of similar problems.

### Human baseline is high

Average humans solve \~85% of ARC tasks. The benchmark was calibrated to be easy for humans but hard for machines — the opposite of benchmarks like MATH or MMLU.

### Exact-match evaluation

Outputs must be **pixel-perfect**. There is no partial credit — either the model produces exactly the right grid or it fails.

## ARC-AGI-2 vs. ARC-AGI-1

| Feature             | ARC-AGI-1                  | ARC-AGI-2                       |
| ------------------- | -------------------------- | ------------------------------- |
| **Difficulty**      | Hard                       | Significantly harder            |
| **Task complexity** | Moderate transformations   | Multi-step, compositional rules |
| **Best AI score**   | \~55% (with heavy compute) | \~40%                           |
| **Human score**     | \~85%                      | \~80%                           |
| **Focus**           | Basic abstraction          | Compositional generalization    |

## Why It Matters

ARC-AGI-2 is the benchmark most closely associated with the question: **"Are we making progress toward AGI?"**

* **Tests intelligence, not knowledge** — No amount of training data helps; the model must reason in real-time
* **Immune to scale** — Bigger models don't automatically perform better
* **Measures generalization** — The core capability that separates intelligence from memorization
* **ARC Prize** — François Chollet's \$1M+ prize has made this a flagship challenge in the AI community

## Notable Results

| Model / System                            | Accuracy      | Date |
| ----------------------------------------- | ------------- | ---- |
| ARC Prize 2024 Winner (program synthesis) | \~55% (ARC-1) | 2024 |
| OpenAI o3 (high compute)                  | \~40% (ARC-2) | 2025 |
| Claude 3.5 Sonnet                         | \~25% (ARC-2) | 2025 |
| GPT-4o                                    | \~20% (ARC-2) | 2025 |

<Note>
  The gap between human performance (\~80%) and the best AI systems (\~40%) on ARC-AGI-2 remains one of the largest among major benchmarks, making it a key indicator for progress toward genuine reasoning.
</Note>

## Limitations

* **Grid-based only** — Tests visual/spatial reasoning but not linguistic, social, or physical reasoning
* **Binary scoring** — No partial credit for "almost correct" solutions
* **Compute-sensitive** — Some approaches brute-force solutions with massive compute, gaming the benchmark
* **Narrow reasoning type** — Does not capture all forms of intelligence

## References

* [ARC Prize](https://arcprize.org) — Official prize and leaderboard
* [On the Measure of Intelligence](https://arxiv.org/abs/1911.01547) — François Chollet's original paper defining the ARC framework
* [ARC-AGI-2 Announcement](https://arcprize.org/blog/arc-agi-2) — Technical details on the updated benchmark