Calibrating story points means periodically checking that your team's "3", "5" and "8" still mean what they meant a few months ago, and correcting course if they don't. You do it with a small catalog of reference stories, a blind re-estimation session once a quarter, and two or three simple accuracy measures that tell you whether point sizes still separate small work from big work.
Without calibration, a team's baseline drifts. Six months ago "add form validation" was a 5. Today the same work gets a 3. The velocity chart shows a nice upward trend, nothing more is being shipped, and every forecast built on that chart is now off.
What estimation drift is (and isn't)
Estimation drift is a gradual, unnoticed shift in what a point means. The same amount of work gets a different number depending on when the team estimates it.
Not every change is drift. The test is whether the team can explain it:
| Change | Real improvement or drift? | How you can tell |
|---|---|---|
| "We built a shared auth library, so OAuth work is genuinely smaller now" | Real improvement | Specific, nameable cause; cycle time for that work actually dropped |
| "We know this codebase much better than we did" | Real improvement (partly) | Shows up as shorter cycle times, not just lower numbers |
| "It just feels like a 3 now" | Drift | No cause anyone can name |
| "We've been hitting 35 points lately, so this can't be a 5" | Drift | Circular reasoning: velocity is setting the estimate |
| "Tech debt makes everything harder" (estimates going up) | Real change in difficulty | Cycle times rose too |
| "Nobody remembers why this was an 8" | Drift | Lost reference point |
Good drift is conscious and explainable. Bad drift is invisible.
Why baselines drift
- No reference stories. Without concrete examples of each size, the team anchors on whatever it estimated last week, and the baseline moves a little every sprint.
- People join and leave. New members bring the scale from their old team. Three new hires in six months can shift the collective scale without anyone deciding to.
- Pressure on velocity. When management celebrates rising velocity, estimates tend to inflate (or a team sandbagging against pressure deflates commitment). Nobody says "call it a 5 instead of a 3"; the bias is quieter than that.
- Rushed refinement. Discussion gets shorter, people stop comparing to examples, and the scale goes loose.
Warning signs between calibrations
- Velocity has risen steadily for several sprints but the team doesn't feel like it's shipping more.
- New members consistently vote higher (or lower) than the rest of the team.
- The share of stories at each size has changed even though the work hasn't. Say 40% of stories used to be 3s and now 55% are.
- Planning poker reaches instant agreement on nearly everything. Nobody is checking estimates against anything.
- Your 3-point and 8-point stories take about the same time to finish (more on measuring this below).
Step 1: Build a reference-story catalog
A reference catalog is a short list of real, completed stories the team agrees are good examples of each size. It's the anchor that keeps the scale stable. Keep it on a wiki page or at the top of your refinement doc, and use it in every estimation session.
Template (copy and adapt)
| Points | Story (ticket link) | What it involved | Why this size | Cycle time | Added on | Still representative? |
|---|---|---|---|---|---|---|
| 1 | PAY-412: Change copy on payment error | Text change, one test update | No logic, no unknowns | 0.5 days | 2026-04-02 | Yes |
| 2 | PAY-388: Add field to CSV export | One query change, tests | Known code, one small schema touch | 1 day | 2026-04-02 | Yes |
| 3 | AUTH-201: Email validation on signup | Front-end + API validation, tests | Two layers, well understood | 2 days | 2026-03-10 | Yes |
| 5 | AUTH-233: Password reset with email token | New endpoint, token expiry, email template | Several parts, one external service | 4 days | 2026-03-10 | Yes |
| 8 | AUTH-260: Google OAuth login | Third-party integration, account linking edge cases | External dependency, real unknowns | 7 days | 2026-02-18 | Review: built shared OAuth lib since |
| 13 | RPT-90: Monthly usage report | New data pipeline, UI, permission checks | Multiple unknowns; would split today | 12 days | 2026-01-20 | Retire, too big to reference |
Guidelines for picking references:
- One or two per size to start; build up to 3 per size over time. More than that and nobody reads it.
- Typical work, not outliers. Pick stories that went roughly as expected, not ones where everything went wrong.
- Clear "why this size." The rationale column is what makes the catalog useful. A ticket link alone doesn't help someone new.
- Keep it current. The "Added on" and "Still representative?" columns make stale entries obvious.
- Separate domains if needed. A team that does both front-end and infrastructure work may want a short list for each, while still tracking one velocity.
How to use it in planning poker
When votes split between 5 and 8, the facilitator asks: "Is this bigger or smaller than AUTH-233, the password reset?" Comparing against a concrete story settles most debates faster than arguing about hours. Walk every new team member through the catalog in their first week; 30 minutes saves months of their votes pulling against the team's scale.
Step 2: The quarterly calibration session
Once a quarter, test whether the scale has moved. This fits in 60 minutes and can replace part of a refinement session.
Agenda
| Time | Activity | Details |
|---|---|---|
| 0–5 min | Set up | Explain the goal: "We're checking our scale, not grading anyone." |
| 5–30 min | Blind re-estimate | Present 5–8 completed stories from about six months ago, spanning 2 to 13 points. Hide the original estimates. Estimate each with normal planning poker. |
| 30–40 min | Compare | Reveal originals next to today's votes in a table. |
| 40–55 min | Diagnose | For each story that moved, ask "Why would we size this differently today?" Sort answers into real change vs drift (use the table above). |
| 55–60 min | Decide and record | Update the catalog, write down the decision, set one action. |
Picking stories: choose ones the current team can still understand, that were completed without major scope change, and that resemble current work. Remove the Story Points field from what you show, or copy just the description and acceptance criteria into a doc. If people see the original number, anchoring bias (the effect Kahneman and Tversky described) does the rest and you'll find no drift.
Hidden votes matter here even more than usual. In Alignlee's free planning poker, votes stay hidden until reveal, so no one is nudged by a senior engineer's number.
Worked example
| Story | Original (Feb) | Re-estimate (Aug) | Change |
|---|---|---|---|
| Email validation on signup | 3 | 2 | -1 |
| Password reset flow | 5 | 3 | -2 |
| Google OAuth login | 8 | 5 | -3 |
| CSV export field | 2 | 2 | 0 |
| Notification preferences page | 5 | 3 | -2 |
| Usage report | 13 | 8 | -5 |
Five of six stories came in lower, so the pattern is consistent deflation. The diagnosis discussion found two different causes:
- OAuth: the team built a shared OAuth library in April. That's real; OAuth-type work is smaller now. Action: replace the OAuth reference with a newer 8-point story.
- Password reset and notification preferences: nobody could name a reason beyond "it feels smaller." That's drift. Action: keep 5 as the reference value for that kind of work, re-read the rationale column aloud, and refer back to it in the next few refinements.
Write the decision down: "August calibration: OAuth reference retired (shared library). Re-committed to existing 3/5 references. Next calibration November." When stakeholders ask why velocity changed, this note is your answer.
If results are mixed (some stories up, some down, no pattern), there's probably no systematic drift. Note it and move on. If you can't tell, repeat with different stories in 4 to 6 weeks.
Cross-team calibration
Organizations with several teams can run the same exercise across teams: everyone estimates the same 5 stories, then compares. The goal is not to force identical numbers but to understand differences. "Team B votes lower on anything touching payments because they built that service" is a useful finding. See standardizing estimation across enterprise teams for when to push further.
Step 3: Measure estimation accuracy (simply)
Story points are not supposed to predict hours, so "accuracy" here means: does the scale separate small work from large work, and does the team finish what it plans? Three metrics cover it, and all of them come from data you already have.
1. Say/do ratio
Say/do = points completed / points committed, per sprint.
| Sprint | Committed | Completed | Say/do |
|---|---|---|---|
| 14 | 34 | 26 | 76% |
| 15 | 32 | 27 | 84% |
| 16 | 30 | 29 | 97% |
| 17 | 31 | 30 | 97% |
This team overcommitted in sprints 14 and 15, then planned closer to their average velocity. Ratios that stay below about 80% mean planning is optimistic. Ratios at 100% every sprint can mean the team is sandbagging. Either way, look at the trend, not one sprint. (The velocity guide shows how to set the commitment from history.)
2. Estimate-vs-actual spread by point size
This is the most useful calibration check and the one most teams skip. Take the last quarter's completed stories, group them by points, and look at cycle time (days from In Progress to Done) for each group.
| Points | Stories | Median cycle time | Range (fastest–slowest) |
|---|---|---|---|
| 1 | 6 | 0.5 days | 0.5–2 |
| 2 | 9 | 1.5 days | 1–3 |
| 3 | 14 | 2.5 days | 1–6 |
| 5 | 11 | 3 days | 2–9 |
| 8 | 5 | 6 days | 3–14 |
| 13 | 2 | 12 days | 9–15 |
How to read it:
- Medians should rise with point size. They do here, which means the scale works overall.
- 3s and 5s barely differ (median 2.5 vs 3 days, heavily overlapping ranges). The team isn't really distinguishing them. Put a clear 3 and a clear 5 into the reference catalog and compare against both in the next few sessions.
- 8s have a very wide range (3 to 14 days). That's uncertainty, not size. Stories at 8 and above should be split or preceded by a spike. See estimating large epics.
- Track it quarterly. If the median for 5-point stories drops from 3 days to 2 with no process change, your 5s have gotten smaller. That's drift showing up in the data before anyone feels it.
Cycle time is a sanity check on the scale, not a conversion rate. Don't turn "a 5 takes 3 days" into a rule; that turns points back into hours (see story points vs hours).
3. Outliers and re-estimation rate
Count stories where reality was two or more sizes away from the estimate (estimated 3, behaved like an 8 or more). A few per quarter is normal. If they cluster ("anything touching the legacy billing module"), add a reference story for that area or require a spike first. Also count stories that had to be re-pointed or split mid-sprint, and separate scope added by the product owner from complexity discovered during the work. The first is a refinement and product issue; the second is estimation learning.
Running the review
Fifteen minutes in the last retro of each month is enough:
- Show the numbers (5 min): say/do for the month, the spread table, outliers.
- Find one pattern (5 min): "What stands out?"
- Pick one action (5 min): for example, "Add a reference 3 for API-only work" or "Any story touching billing gets a spike first."
What not to do with accuracy metrics
- Don't rank individuals by how "accurate" their votes were. People will copy the senior engineer's vote or avoid the hard stories.
- Don't use the metrics in performance reviews. Estimation is a team activity.
- Don't chase zero variance. If every estimate is right, you're spending too long estimating or only taking safe work.
- Don't compare teams' numbers. Different baselines, different work.
Capturing uncertainty at estimation time
Some inaccuracy is uncertainty you could have seen coming. A quick way to surface it: after the vote, ask how confident each person is. If several people say "low," the story probably needs a spike or more refinement before it goes into a sprint. Alignlee builds this in: each vote has a 0–100% confidence slider, and the result is weighted so low-confidence votes count less (the confidence voting guide explains how to use it). Logging confidence next to the final estimate also gives you a column to compare against the outliers in your quarterly review.
A one-page calibration checklist
- Reference catalog exists with at least one story per size, each with a "why this size" note
- Catalog reviewed in the last 6 months; stale entries retired
- New team members walked through the catalog in week one
- Facilitator compares disputed stories to a reference during planning poker
- Quarterly blind re-estimation of 5–8 older stories, original estimates hidden
- Decisions logged: what changed, real improvement or drift, what you did
- Say/do ratio tracked every sprint
- Cycle-time spread by point size reviewed quarterly
- Outliers and mid-sprint re-pointing reviewed monthly, scope change separated from discovery
FAQ
What is estimation drift in agile?
Estimation drift is a slow, unnoticed change in what a story point means to a team, so the same amount of work gets a different number than it would have months earlier. It usually shows up as velocity rising without any real increase in delivery. A quarterly blind re-estimation of old stories is the simplest way to detect it.
How do you calibrate story points?
Keep a catalog of real reference stories for each point size, compare new stories against them during estimation, and once a quarter re-estimate 5 to 8 older stories without showing the original numbers. Where estimates have moved, ask whether the team can name a real reason. If it can, update the references; if it can't, re-commit to the existing baseline.
What is a story point baseline?
A story point baseline is the shared understanding of what each point value means, anchored by specific completed stories. A common starting point is a small, well-understood story that everyone agrees is a 1 or a 2, with every other size defined relative to it. Write the baseline down with examples, or it will drift.
How do you measure estimation accuracy with story points?
Track the say/do ratio (completed divided by committed points per sprint) and compare cycle times across point sizes. If medians rise with point size and ranges don't overlap too much, the scale is working. Heavy overlap between two sizes, or a very wide range for large stories, tells you where to recalibrate or split.
How often should a team recalibrate story points?
Quarterly works for most teams, plus an extra check after significant team changes such as several people joining or leaving. Review the reference catalog itself at least every six months and retire stories that no longer reflect how the team works.