DORA Metrics: What They Measure and How to Track Them
The DORA measures describe the speed and stability of software delivery. They are useful when they are defined carefully and harmful when they become individual targets.
DORA, the DevOps Research and Assessment programme, has studied software delivery performance for many years. It identified a small set of measures that together describe how quickly a team delivers change and how stable that delivery is. The set has been refined over time, so check the current definitions published by the programme before you adopt them. The four classic measures are below.
The four measures
- Deployment frequency: how often you deploy to production. It measures throughput.
- Lead time for changes: the time from a change being committed to it running in production. It measures how long work waits in the delivery path.
- Change failure rate: the share of deployments that cause a failure needing remediation, such as a rollback, a hotfix or an incident. It measures stability.
- Time to restore service: how long it takes to recover from a failure in production. It measures resilience.
Why they come as a set
Speed and stability are often assumed to trade against each other. Taken together, the measures show whether you are improving one at the expense of the other. A team that deploys more often but fails more often has not improved. Never report one in isolation.
Define each carefully
The measures are easy to quote and hard to define. Decide what counts as a deployment: every environment, or production only? Decide which failures count towards the change failure rate and how a failure is linked to the change that caused it. Decide when the lead time clock starts and stops. Write the definitions down and keep them stable, or trends will reflect the definitions rather than the delivery.
Where the data comes from
- Deployment frequency and lead time come from your pipeline and version control.
- Change failure rate needs a link between a failure and the release that caused it. A release record that lists scope and deployments, and incidents that note what changed, supply it.
- Time to restore comes from incident records, using a consistent definition of when the failure began and when service was restored. Atlas keeps mitigated and resolved apart, so you can use the time customer impact ended.
Using Atlas records
Release Management in Atlas records releases, their status, rollbacks and the deployments to each environment with status and times. Atlas does not run your pipeline, so send deployment events from the pipeline through the API to keep the record complete. The API lists and reads releases and their deployments, so you can compute your own measures from the data. Incident Management records the instants needed for time to acknowledge, mitigate and resolve.
Traps
- Turning the measures into targets for individuals or teams in competition. People will game them.
- Comparing teams with different products and risks as though the numbers were comparable.
- Counting deployments that are not meaningful, such as configuration toggles, to raise frequency.
- Ignoring the distribution. Look at the slowest changes and the longest outages, not only the average.