Jury and evaluation
Calibrate scores before a jury round
Updated 30 Sep · 2 min · Program teamUpdated · 2 min read · For Program team
Two jurors can read the same application and score it a 3 and a 9. Usually the rubric is the problem, not the jurors. Calibration is a short exercise that surfaces those differences before they reach your shortlist.
Steps
- Pick three applications from the pool: one clearly strong, one clearly weak and one in between.
- Ask every juror to score all three against the rubric, independently, before you meet.
- Put the scores side by side. For any criterion where scores differ by more than three points, ask jurors to explain their reasoning.
- Rewrite the descriptor for that criterion until the jurors agree on what a low, middle and high score means. Then start the round.
Why it works
Jurors bring their own reference points. An investor's "strong traction" and a professor's are different numbers. Calibration replaces those private scales with a shared one, which matters most when you are comparing applications scored by different people.
Was this article helpful?