The CTO's Guide to Engineering Metrics That Matter
The desire to measure engineering productivity is natural. When engineering represents one of the largest budget items in a technology-driven organisation, leadership rightly wants visibility into how that investment translates to business outcomes. The problem is that most organisations measure the wrong things, create perverse incentives, and ultimately degrade the performance they are trying to improve.
After years of working with engineering organisations ranging from fifty-person startups to multi-thousand-person enterprises, I have found that the difference between effective and destructive measurement comes down to understanding what questions you are actually trying to answer, and having the discipline to resist metrics that feel satisfying but drive counterproductive behaviour.
The DORA Framework: A Solid Foundation
The most rigorously validated framework for software delivery performance comes from the DORA (DevOps Research and Assessment) team, whose multi-year research programme has established statistical links between specific engineering practices, delivery metrics, and organisational outcomes.
The four key metrics from the DORA research are:
Deployment Frequency: How often does your organisation deploy code to production? This measures the batch size of changes and the organisation’s ability to deliver value incrementally. Elite performers deploy on demand, multiple times per day. Low performers deploy between once per month and once every six months.
Lead Time for Changes: How long does it take from code commit to code running in production? This captures the efficiency of your entire delivery pipeline, including code review, testing, approval processes, and deployment automation. Elite performers measure this in under one hour. Low performers measure it in months.
Change Failure Rate: What percentage of deployments cause a failure in production that requires remediation? This measures the quality of the delivery process. Importantly, the DORA research shows that speed and stability are not trade-offs — elite performers achieve both high velocity and low failure rates.
Time to Restore Service: When a failure occurs, how quickly can the organisation restore service? This measures the resilience of both the technical systems and the organisational processes. Elite performers restore service in under one hour.
These metrics work because they measure outcomes rather than activities. They do not prescribe specific tools or practices; they measure the results that good practices should produce. A team could achieve elite performance through any number of technical and organisational approaches.
For CTOs implementing DORA metrics, the critical success factor is measurement automation. These metrics should be derived automatically from your deployment pipeline, incident management systems, and version control platforms. Manual data collection introduces bias, creates overhead, and quickly becomes stale.
The Danger of Individual Productivity Metrics
Where organisations consistently go wrong is attempting to measure individual developer productivity through activity metrics like lines of code, number of commits, story points completed, or pull requests merged.
These metrics are problematic for several reasons:
Goodhart’s Law: When a measure becomes a target, it ceases to be a good measure. Developers who are measured on lines of code will write verbose code. Developers measured on pull requests will submit smaller, more fragmented changes. The metric improves while the underlying quality degrades.
Value vs. Volume: The most impactful engineering work is often the least visible in activity metrics. An architect who spends a week designing a system that prevents six months of technical debt generates enormous value but produces few lines of code. A developer who refactors a critical module, reducing its complexity by forty percent, shows negative lines of code while dramatically improving maintainability.
Collaborative Work: Modern software development is inherently collaborative. Code reviews, mentoring, architecture discussions, and cross-team coordination are essential to team performance but invisible to individual activity metrics. Measuring individual output incentivises solo work at the expense of team effectiveness.
Context Variation: Different types of work have fundamentally different productivity profiles. Building a new feature, debugging a production incident, migrating a legacy system, and writing infrastructure automation require different skills, produce different artifacts, and operate on different timescales. No single metric can meaningfully compare productivity across these activities.
Instead of individual metrics, I recommend focusing on team-level measures of flow, quality, and outcomes. The team is the fundamental unit of software delivery, and optimising team performance produces better results than optimising individual utilisation.
Building a Metrics Culture Without Creating Dysfunction
The implementation of engineering metrics is as important as the selection of what to measure. I have seen organisations with excellent metric choices fail because of how they introduced and used those metrics.
Start with diagnostic intent: Metrics should be tools for understanding, not weapons for accountability. When a team’s deployment frequency drops, the right response is curiosity, not criticism. What changed? What obstacles are slowing them down? What support do they need? If engineers perceive metrics as management surveillance, they will game the system or disengage entirely.
Maintain context: Numbers without context are dangerous. A team with a high change failure rate might be working on a legacy system with inadequate test coverage, or they might be deploying to an environment with flaky infrastructure. The metric identifies where to investigate; it does not diagnose the root cause.
Limit the number of metrics: Cognitive load applies to leadership as much as it applies to engineering. An organisation tracking fifty metrics is effectively tracking none. I recommend starting with the four DORA metrics supplemented by two or three business-specific measures, such as time to onboard a new customer or mean time to resolve customer-reported defects.
Invest in dashboards, not reports: Static monthly reports create a false sense of control while providing stale information. Real-time dashboards that teams can access directly create shared awareness and enable rapid response. Tools like Grafana, Datadog, and specialised platforms like LinearB and Sleuth are making it increasingly practical to maintain live engineering intelligence dashboards.
Review trends, not snapshots: A single data point means nothing. A trend over weeks or months tells a story. The most valuable conversations happen when a metric changes direction — when deployment frequency starts declining, or when lead time starts increasing. These inflection points often signal organisational or technical changes that warrant attention.
Connecting Engineering Metrics to Business Outcomes
The ultimate purpose of engineering metrics is to connect technology investment to business value. This requires bridging two worlds that often speak different languages.
Product delivery metrics translate well to business conversations. If the organisation can deploy new features three times faster than last year, that directly impacts time to market and competitive advantage. If the change failure rate has halved, that translates to improved customer experience and reduced support costs.
The bridge between engineering and business metrics typically requires a middle layer of product metrics: feature adoption rates, customer satisfaction scores, and revenue impact of specific releases. Without this connection, engineering metrics remain an internal concern that struggles to justify continued investment.
For CTOs reporting to boards and executive teams, I recommend presenting engineering metrics as leading indicators of business performance. DORA metrics predict organisational performance — the original research demonstrated correlations between delivery performance and profitability, market share, and employee satisfaction. This framing helps non-technical leadership understand why engineering process improvements matter.
The organisations that get this right treat engineering metrics not as a performance management tool, but as a strategic intelligence system. They use data to identify constraints, allocate resources, and make informed decisions about where to invest in improvement. The goal is not to measure engineers — it is to understand the engineering system well enough to continuously improve it.
This distinction — between measuring people and understanding systems — is the single most important principle for CTOs building an engineering metrics programme. Get it right, and metrics become a powerful tool for organisational improvement. Get it wrong, and they become a source of dysfunction that drives away your best people.