Humanity’s Last Exam (HLE) is a crowdsourced benchmark of approximately 3,000 questions spanning over 100 academic disciplines, created by domain experts from around the world. It was designed with a provocative mission: to be the last exam humanity would need to create — one so difficult that only a truly general intelligence could pass it.Created by the Center for AI Safety (CAIS) and Scale AI, Humanity’s Last Exam represents the most ambitious attempt to test the frontier of AI knowledge and reasoning across the full breadth of human expertise.
Expert contribution: Thousands of researchers and domain experts submitted their hardest questions — problems that would challenge even PhD-level specialists
Curation: Questions were filtered for quality, uniqueness, and difficulty
Format: Mix of multiple-choice and short-answer questions
Evaluation: Models are tested zero-shot with standard prompting
Humanity’s Last Exam addresses a critical problem in AI evaluation: benchmark saturation. As models quickly approach ceiling performance on existing benchmarks (MMLU, GPQA, etc.), the field needs harder tests that can differentiate frontier capabilities.
Expert-caliber difficulty — Questions are designed to be challenging even for specialists in each field
Breadth over depth — Tests the generality of knowledge, not just narrow expertise
Contamination-resistant — Novel questions not found in training data
AGI milestone — Performance on HLE is widely tracked as a proxy for progress toward general intelligence
Humanity’s Last Exam remains far from solved. The best models get approximately 1 in 4 questions correct, highlighting the vast gap between current AI and comprehensive human-level expertise.
