benchmarks

Type: Concept

Standardized tests for model performance; criticized as flawed, gameable, and weakly correlated with real-world capability (e.g., models winning IMO but failing at PDFs).

Field: AI evaluation

Mentioned in 1 podcast episode

Who brought it up

Related Entities