The Brier score measures the accuracy of probability forecasts.
For a binary Yes-or-No event, subtract the outcome from your forecast probability, square the difference, and average the errors. Lower is better.
A 70% forecast scores 0.09 when the event happens and 0.49 when it does not. The calculator below handles one forecast or a full set.
The arithmetic is simple. Interpretation is not.
A raw score does not tell you whether a forecaster is skilled, whether a market is accurate, or whether a trade was profitable. You need a fair benchmark, the same questions, and forecasts recorded at the same time.
Method note: This guide uses the common binary form of the Brier score, scaled from 0 to 1. Yes is recorded as 1 and No as 0. The original multi-category form uses a 0-to-2 scale.
What Is a Brier Score?
A Brier score is the mean squared error of probability forecasts.
The statistician Glenn W. Brier introduced the measure in a 1950 paper on weather forecast verification. It is now used in forecasting, machine learning, risk models, and prediction markets.
The score answers one question:
How close were your stated probabilities to the outcomes that occurred?
It rewards high probabilities on events that happen and low probabilities on events that do not. It also makes confident mistakes expensive.
Suppose an event resolves No:
| Forecast | Brier error |
|---|---|
| 55% Yes | 0.3025 |
| 70% Yes | 0.4900 |
| 90% Yes | 0.8100 |
All three forecasts missed. But the 90% forecast receives the largest error because it expressed the most confidence.
The Brier score is a strictly proper scoring rule. Over time, the best way to minimize your expected score is to report the probability you actually believe. Acting more certain than you are does not help. Gneiting and Raftery give the formal treatment in their paper on proper scoring rules.
Brier Score Formula
For binary forecasts:
Brier score = (1 / N) × Σ(pᵢ − oᵢ)²
| Symbol | Meaning |
|---|---|
N | Number of forecasts |
pᵢ | Forecast probability for Yes, written from 0 to 1 |
oᵢ | Resolved outcome: 1 for Yes, 0 for No |
Σ | Add the squared errors |
For one forecast, use:
(probability − outcome)²
Example: 70% and Yes
Write 70% as 0.70. A Yes outcome is 1.
(0.70 − 1)² = 0.09
Example: 70% and No
A No outcome is 0.
(0.70 − 0)² = 0.49
The second error is larger because the forecast put 70% on an outcome that did not occur.
Brier Score Calculator
The example starts with four forecasts:
| Probability of Yes | Outcome | Squared error |
|---|---|---|
| 70% | Yes | 0.09 |
| 40% | No | 0.16 |
| 80% | Yes | 0.04 |
| 30% | No | 0.09 |
Their total error is 0.38. Divide by four and the average Brier score is 0.095.
Edit the values, add rows, or remove them. The calculation runs in your browser. Pariflow does not save the forecasts.
Interactive calculator
Calculate a binary Brier score
Enter each probability before the event resolves. Then select the actual outcome.
Positive means lower squared error than predicting 50% on these same rows. It does not measure profit or statistical significance.
The benchmark in the calculator is a constant 50% forecast. It scores 0.25 on every binary event. That makes it easy to understand, but not automatically appropriate for every dataset.
What Is a Good Brier Score?
There is no universal cutoff.
For the binary form used here:
| Score | Meaning |
|---|---|
0 | Perfect: every forecast matched the outcome exactly |
0.25 | The score from predicting 50% every time |
1 | Maximum error: complete confidence in the wrong outcome |
The common mistake is treating 0.25 as a general line between good and bad.
Imagine a dataset where only 10% of events resolve Yes. A forecaster who predicts 10% every time has an expected score of:
(0.10 × 0.90²) + (0.90 × 0.10²) = 0.09
That looks much better than 0.25. Yet the forecast adds no information beyond the base rate.
Rare-event datasets often produce lower raw scores than balanced, difficult datasets. This is not just a theoretical concern. A U.S. Weather Bureau analysis of precipitation forecasts found that raw Brier scores changed with the observed frequency of precipitation. The report recommended comparing forecasts with an appropriate reference rather than comparing raw scores across different climates.
Use a Brier score to compare:
- two forecasters on the same questions
- your forecast with the market price at the same timestamp
- a model with a base-rate forecast
- two periods that followed the same inclusion rules
If the questions, timing, or scoring convention changes, the comparison weakens.
Brier Skill Score
The Brier Skill Score compares a result with a reference forecast:
Brier Skill Score = 1 − (your Brier score / reference Brier score)
- above
0: better than the reference - exactly
0: equal to the reference - below
0: worse than the reference - exactly
1: perfect
For the calculator example:
1 − (0.095 / 0.25) = 0.62
The result is 0.62, or 62% lower squared error than the 50% reference.
It does not mean 62% accuracy. And a weak reference can flatter an ordinary forecast. Use a benchmark someone could have known before the outcome.
Brier Score vs Calibration
Brier score and calibration are related, but they are not the same.
A forecaster is calibrated when events assigned 70% happen about 70% of the time. You need many forecasts to test that pattern.
Now consider someone who predicts 50% on every question in a balanced dataset. Half of the events happen and half do not. The forecaster can be perfectly calibrated while providing no useful separation between likely and unlikely events.
The classic Brier decomposition includes:
- reliability: whether stated probabilities match observed frequencies
- resolution: whether forecasts separate events into meaningfully different groups
- uncertainty: how variable the outcomes are
Allan Murphy set out this decomposition in 1973.
You do not need the full decomposition for a basic forecast journal. But it explains why one average cannot describe every part of forecast quality.
Using Brier Scores in Prediction Markets
A binary prediction market price is often read as an implied probability. A Yes contract trading at $0.62 suggests a 62% market probability.
That interpretation is useful, not exact. Spread, fees, liquidity, risk preferences, and market structure can separate a tradable price from a clean consensus belief.
Wolfers and Zitzewitz examined these conditions in Interpreting Prediction Market Prices as Probabilities. Their conclusion was measured: market prices usually provide useful estimates of average beliefs, but those estimates can be biased.
Compare the Same Moment
Suppose the market price is 0.62, your forecast is 0.70, and the event resolves Yes:
- market error:
(0.62 − 1)² = 0.1444 - your error:
(0.70 − 1)² = 0.09
Your forecast did better on this event.
If the event resolves No:
- market error:
(0.62 − 0)² = 0.3844 - your error:
(0.70 − 0)² = 0.49
The market did better.
One result proves little. A calibrated 70% forecast should still fail about three times out of ten.
For a fair comparison:
- Record your forecast and the market price at the same timestamp.
- Use one price rule, such as midpoint or best executable price.
- Keep the bid, ask, and spread for thin markets.
- Define which questions enter the sample before they resolve.
- Score every eligible question, not only memorable calls.
The last traded price may be stale. The midpoint may not be executable. Pick a measure, document it, and keep using it.
Brier Score Is Not Trading Profit
A lower Brier score does not guarantee a profitable strategy.
Profit also depends on:
- the price paid
- fees and spread
- slippage
- position size
- the ability to exit
- settlement rules
You can estimate probabilities well and still overpay. You can also make money over a short period with weak forecasts and aggressive risk.
Use the Brier score to evaluate probability judgment. Use the Expected Value Calculator to evaluate a price. Use the Kelly Position Size Calculator to control exposure.
Those are separate questions.
Brier Score vs Log Loss
Both are proper scoring rules. They treat extreme mistakes differently.
| Measure | Main behavior |
|---|---|
| Brier score | Squares the probability error and stays bounded |
| Log loss | Applies a much larger penalty to confident forecasts that fail |
The Brier score is easier to explain and audit. Log loss is useful when extreme confidence should carry a severe penalty.
Neither measure is always better. Choose before reviewing the results. Do not switch because another metric makes the record look stronger.
Why Some Brier Scores Use a 0-to-2 Scale
The common binary formula in this guide ranges from 0 to 1.
The original multi-category formula sums the squared error across every possible class. That form ranges from 0 to 2.
This difference can make two correct results appear to disagree by a factor of two. Current scikit-learn documentation makes the convention explicit: binary scores are usually scaled to 0–1, while the original multi-class form keeps the 0–2 range.
Before comparing published scores, check:
- Is the task binary or multi-class?
- Was the result divided by two?
- Which outcome was treated as positive?
- Were the same observations included?
If those details are missing, the comparison may not be valid.
Common Mistakes
- Calling 0.25 “random guessing.” It is the result of a constant 50% forecast, not a universal model of random behavior.
- Comparing different question sets. Base rates and difficulty change the raw score.
- Picking a favorable timestamp. A forecast made one hour before resolution had more information than one made six months earlier.
- Keeping only memorable calls. Dropping boring or embarrassing forecasts creates selection bias.
- Treating a small sample as proof. Ten questions may reveal obvious overconfidence. They rarely establish a durable edge.
- Confusing accuracy with value. A sound probability can still be a bad trade at the available price.
For each prediction, keep the question, resolution rule, probability, timestamp, market price, bid and ask, evidence, and final outcome. The trade journal template generator provides a starting structure.
Frequently Asked Questions
Is a Lower Brier Score Better?
Yes. In the common binary form, 0 is perfect and 1 is the largest possible error. Compare scores only when the questions and scoring convention are compatible.
Is 0.25 a Good Brier Score?
It is the score from predicting 50% on every binary event. Whether it is a useful benchmark depends on the event base rate and question difficulty.
Does a Better Brier Score Mean Higher Profit?
No. Profit depends on entry price, fees, slippage, position size, and settlement as well as forecast quality.
Sources and Editorial Method
This guide uses the binary 0–1 convention throughout. The worked examples are covered by automated calculation tests, including the 0.095 average and 0.62 skill score shown in the calculator. Definitions and interpretation were checked against primary or official sources:
- Glenn W. Brier, “Verification of Forecasts Expressed in Terms of Probability”, Monthly Weather Review, 1950.
- Allan H. Murphy, “A New Vector Partition of the Probability Score”, Journal of Applied Meteorology, 1973.
- Tilmann Gneiting and Adrian E. Raftery, “Strictly Proper Scoring Rules, Prediction, and Estimation”, Journal of the American Statistical Association, 2007.
- Justin Wolfers and Eric Zitzewitz, “Interpreting Prediction Market Prices as Probabilities”, NBER Working Paper 12200, 2006.
- Scikit-learn Brier score loss documentation, including binary and multi-class scaling.
- Woodrow W. Dickey, “Verification of Operational Probability of Precipitation Forecasts, April 1966–March 1967”, U.S. Weather Bureau Technical Memorandum WR-25, 1967.
This article is educational. It does not provide financial, betting, or investment advice.
Nikolay Golovin
Co-Founder & CEO
Nikolay is the Co-Founder of Pariflow with expertise in data science and machine learning. He studies how prediction markets turn uncertain events into measurable probabilities.