Calibrate your confidence
Answer trivia, rate your certainty, and compare your confidence with your accuracy.
Ten questions. How sure are you?
- Choose an answer without looking it up.
- Pick your confidence: 50%, 75% or 100%.
- Check your curve, then try a fresh round.
Each round mixes ten questions across five topics.
About 3 minutes 5 fresh rounds
What does 75% sure mean?
Across four answers you rate 75%, you should expect about three to be correct.
After each round, your calibration curve compares how sure you felt with how often you were right. Three confidence choices collect more answers at each point. Look across several rounds to see the pattern more clearly.
How calibration and scoring work
Calibration: among answers you rate 75%, about 75% should be correct over many attempts. Accuracy alone measures how much trivia you know. Calibration measures whether your certainty fits that knowledge.
Brier score: for each answer, we square the difference between your confidence (as a decimal) and the outcome (1 for correct, 0 for incorrect), then average. A 75% answer scores 0.0625 if correct and 0.5625 if wrong. Scores range from 0 to 1; a 50% guess always scores 0.25. This measures overall probability accuracy, not calibration alone.
Keep perspective: confidence groups can contain only one or two answers in a round. A small gap between averages is not proof of good calibration. Use the curve and sample sizes across rounds. This trivia exercise does not establish how well you forecast real-world events.
Practice and feedback: Tetlock and colleagues’ work on forecasting tournaments provides broader context for practicing explicit probabilities and reviewing outcomes.
Inspired by Julia Galef’s The Scout Mindset. All 50 questions are original to Cognitive Bias Lab.
Progress saved in this browser. No account needed.