Lessons from talks worth rewatching.
Visual lessons made from talks, videos and articles, with hand-drawn figures you can step through.
- Lesson 02
Grade what the agent did, not what it said
Notes on Anthropic’s guide to agent evals: tasks, trials and graders, why pass@k and pass^k tell opposite stories, and nine steps from zero to evals you can trust.
Anthropic · article
- #evals
- #agents
- #article-notes
- Lesson 01
Evals are fuzzy unit tests, and the judge can lie
Notes on Tejas Kumar’s IBM talk: what an eval is, why keyword checks and LLM judges fail, and how to calibrate a judge against human verdicts.
Tejas Kumar · IBM · 60 min talk
- #evals
- #llm-judge
- #talk-notes
hardeep.cv
© 2026 Hardeep Singh