Common questions
What is a calibration test?
A calibration test measures whether your confidence matches reality. You rate how likely each statement is to be true, and the test checks whether the things you were, say, 80% sure about turned out true about 80% of the time. Good calibration means your confidence is trustworthy.
What is a good Brier score on this test?
The Brier score is the average squared gap between your probabilities and the truth, where lower is better and 0.25 is what pure 50/50 guessing gives. On this test, below about 0.18 is well-calibrated and below 0.12 is sharp.
What does it mean to be overconfident?
Overconfidence means stating probabilities that are too extreme, being 90% sure when 70% is the honest number. It's the most common forecasting bias, and this test flags it when your high-confidence answers turn out wrong more often than your confidence implied.
How can I get better calibrated?
Practice assigning probabilities and check them against outcomes, notice where you're systematically over- or under-confident, and correct it. Tracking real predictions over time in a prediction journal is the most effective way to improve.