Engineering Management

Metrics That Matter vs. Vanity Metrics: Why Velocity Is the Wrong KPI for Agile Teams

8 min read
Table of Content
Share this

Join our newsletter

The best collaborative work insights.

Newsletter
⚡ Quick Answer

Why is velocity the wrong metric for agile teams?

Velocity measures how much a team ships, not whether that work created value or whether the team burned out to hit the number. It's a vanity metric; easy to inflate, easy to game, and disconnected from outcomes. Team health, psychological safety, and cycle-time stability predict long-term performance far better.

  • Velocity is a planning input for one team's own sprint forecast, not a KPI for comparing teams or tracking performance.
  • If a metric can go up while the team gets worse, it's a vanity metric, and velocity fails that test easily.
  • Track team health alongside velocity: cycle-time stability, psychological safety, escaped defects, and retro action-item follow-through.

Every quarter, some engineering leader pulls up a velocity chart in a planning review and asks why the line went down. Nobody in the room ever asks why it went up. That's the tell. A metric everyone scrutinizes on the way down and never questions on the way up isn't measuring performance. It's only measuring compliance with a number. Velocity is agile's most popular vanity metric, and most teams are optimizing for it without ever deciding to.

Why Velocity Became Agile's Default Success Metric

Velocity was never designed to be a performance indicator. It's a planning input: a rough average of story points a team completes per sprint. It is meant to help that same team forecast how much work it can realistically commit to next time. Atlassian's own documentation on the velocity report is explicit about velocity being a capacity-planning tool, scoped to one team, never intended to compare teams or track improvement over time.

Somewhere between the whiteboard and the dashboard, that scope got lost. Velocity is the easiest agile number to pull out of Jira, so it became the default answer to "how's the team doing?" A number built to help a team plan its own sprint turned into a scoreboard leadership reads from the outside. Once a metric becomes a scoreboard, teams start playing to it instead of using it.

The Problem With Measuring Speed Instead of Outcomes

Once a team knows its velocity is being watched, the incentives quietly flip. Estimates inflate. Stories get split into smaller pieces that look like more throughput. Technical debt gets deferred because paying it down doesn't move the number this sprint. None of that shows up as a red flag. It shows up as a healthy, rising velocity chart, which is exactly what makes it dangerous.

Jellyfish's analysis of vanity metrics in engineering makes the same point from the data side: metrics that are easy to game get gamed, and velocity is one of the easiest. A team can hit an all-time-high velocity while shipping lower-quality work, burning out its strongest engineers, and quietly building a backlog of debt that shows up as a crisis two quarters later. The chart looks great right up until it doesn't.

This is the core problem with output metrics: they measure what's easy to count, not what's true. Speed is easy to count. Whether the team can sustain that speed, whether the work solved the customer's problem, or whether people are burning out to hit the number, none of that shows up in a velocity report.

What "Vanity Metric" Actually Means for Agile Teams

A vanity metric is a number that looks good on a dashboard but doesn't tell you whether your team, product, or business is improving. In agile teams, velocity (story points completed per sprint) is the most common vanity metric: it can rise while quality drops, scope gets padded, or burnout builds, because velocity only tracks output, not outcomes or team health.

The test for whether a metric is a vanity metric or a real one is simple: can the number go up while the team gets worse? With velocity, the answer is yes, easily. With something like cycle-time stability, or a team's own reported sense of psychological safety, the answer is no. Those numbers only move in a good direction when something real changes underneath them.

The Metrics That Actually Predict Team Health

If velocity measures how much a team ships, team health measures whether the team can keep shipping, sustainably, without burning out or eroding trust. That distinction is exactly what Jira-native team health checks with TeamPulse are built to surface, which means no switching to a separate tool, with morale, workload, and psychological safety tracked right next to the sprint data leadership already reviews, so a dip shows up weeks before it becomes a resignation letter or a missed release.

A few metrics worth tracking alongside velocity (or in place of it):

  • Cycle-time stability. Not how fast work moves, but how consistent that speed is sprint over sprint. Wild swings are a better early-warning signal than a slowing average.
  • Psychological safety and morale trend. Whether the team feels safe raising a blocked task or a bad idea before it becomes a bigger problem.
  • Escaped defect rate. How much of what ships comes back as a bug, the honest counterweight to any throughput number.
  • Retro action-item follow-through. Whether the things a team says it will fix, get fixed, or get raised again next retro. Teams that turn retro feedback into tracked Jira action items can measure this sprint over sprint instead of guessing.

None of these are harder to track than velocity once they're built into your existing Jira workflow. They're less visible by default, because nobody put them on the standard dashboard.

TeamPulse tracks those health signals next to your sprint data, and Agile Retrospectives for Jira closes the loop by turning what the team raises in each retro into Jira issues someone owns.

Try TeamPulse Free on Jira → Install TeamPulse

Pair It With Agile RetrosTry Agile Retros Free on Jira

How to Introduce Team Health Metrics Without Losing Leadership Buy-In

The fastest way to lose this argument with leadership is to propose dropping velocity outright. Don't. Velocity still has a narrow, legitimate job: helping one team forecast its own sprint capacity. And taking it away entirely just leaves a gap someone else will fill with a worse metric. The move isn't replacement, it's addition: keep velocity as a planning input, and put team health metrics next to it as the number leadership actually reviews in a retro or a skip-level.

A workable rollout looks like this: start tracking one or two team health signals for a single sprint before you present anything, so you walk into the conversation with real data instead of a theory. Frame the pitch around risk, not ideology: "this catches burnout three sprints before it costs us a resignation" lands better with an engineering leader than "velocity is a vanity metric."

Then build on rituals the team already trusts. If you run async standups, pair it with async standups via StandBot: the same cadence that surfaces blockers day-to-day is a natural place to fold in a lightweight health check. And the retrospective is where that signal turns into action. Open each retro with the latest health trend (a fresh retrospective template helps if the format has gone stale), then log what the team commits to fix as tracked Jira issues. Standups, health checks, and retros all read the same underlying signal: is this team actually okay, not just on schedule.

See How TeamPulse Tracks Team Health → Explore TeamPulse and the DevEx suite

Run Async Standups in Slack → Install StandBot Free on Jira

Velocity Was Never the Point. Team Health Is.

Velocity was built to help one team plan one sprint. It was never built to answer the question leadership actually cares about: is this team going to keep performing six months from now? And every time it gets used that way, it quietly rewards the wrong behavior: padded estimates, deferred debt, burnout dressed up as productivity.

The teams that get this right don't throw velocity out. They stop treating it as the scoreboard and use it as a planning input instead. Then they put team health and retro outcomes in front of leadership, because those are the numbers that predict whether the good sprint repeats. That's a smaller shift than it sounds like, and it's the one your next retro is worth spending on.


Related reading:

Luis Ortiz

Luis Ortiz

Luis is the Co-Founder and leads Growth at Catapult Labs. He writes about practical Agile workflows across Jira, Confluence, and Slack, helping teams run better stand-ups, retrospectives, and improve productivity without meeting overload.