human evaluation
Type: Concept
Expert annotators deeply assess model outputs (fact-checking, code correctness, physics equations) as a preferred measure of progress over benchmarks/leaderboards.
Field: AI evaluation
Mentioned in 1 podcast episode
Who brought it up
- The $1B Al company training ChatGPT, Claude & Gemini on the path to responsible AGI | Edwin Chen on Lenny's Podcast — Lenny Rachitsky