Data Analytics starts from a fairly simple idea that is also fundamental to any organization. To improve something, we first need to be able to measure it.
Measurement allows us to understand what is happening, compare periods, detect changes, identify problems, and evaluate whether a decision produced the expected result. That is why indicators play such a central role in data analysis. They turn aspects of a business into variables we can track, compare, and interpret.
Within that system, KPIs play an especially important role. A KPI, or Key Performance Indicator, seeks to represent a relevant aspect of performance through a specific measure. It may be the sales conversion rate, customer response time, retention, margin, defect rate, or any other variable linked to an important organizational objective.
And, of course, it makes sense to want to improve them.
If a company takes too long to respond to its customers, reducing that time can represent a real improvement. If a factory has a high rate of defective products, lowering it is a reasonable goal. If a business converts only a small share of opportunities into sales, working on its conversion rate can also create value.
Measuring, setting goals, and seeking better results are a natural part of data-driven management.
The interesting point appears when a metric stops serving only as a way to observe reality and also begins to shape the behavior of the people involved. Under certain conditions, the effort to improve an indicator can cause that number to evolve in a way that differs from what it was originally intended to represent.
- We can respond faster without solving problems more effectively.
- We can increase sales at the expense of margin.
- We can close more tickets without actually improving the service.
The KPI improves, while the reality we wanted to improve may advance far less, remain unchanged, or even deteriorate.
This phenomenon lies at the heart of one of the most important ideas for understanding how metrics and incentives work.
Goodhart’s Law.
What Goodhart’s Law Says
Charles Goodhart, a British economist, formulated an observation in 1975 in the context of monetary policy. His original argument was more technical than the phrase commonly used today to summarize his idea, but the central principle is very clear. A statistical relationship that is useful for observing a system can change when we begin to use it deliberately to control that system.
Over time, that idea became popular through a simple formulation
“When a measure becomes a target, it ceases to be a good measure.”
The phrase is not a direct quotation from Goodhart, but rather a later reformulation of his argument, and it expresses the problem very well.
Let us return to the response time of a customer support team. If, after analyzing the data, we discover that customers who receive faster responses tend to be more satisfied, response time can be considered a useful indicator. There is a reasonable relationship between the metric we observe and the experience we want to improve.
Now imagine that the company sets a target of answering every inquiry within four hours and that part of the team’s performance evaluation depends on meeting that target.
The situation changes because the indicator begins to influence behavior. The team may genuinely improve its processes, organize shifts more effectively, and resolve customer inquiries faster. In that case, the KPI improves and the service improves as well. But another strategy may also emerge, such as sending a quick, generic response simply to meet the time target and resolving the actual problem several hours later.
The system will still record an excellent improvement in first-response time. However, that improvement no longer necessarily represents better customer service.
This is the central point of Goodhart’s Law. The problem does not lie in measuring or in setting goals. It appears when the effort to improve a metric finds ways to raise the indicator without producing an equivalent improvement in what we originally wanted to achieve.
One useful way to think about this is to understand a KPI as a proxy. It is an observable variable that we use to represent something more complex. Response time can represent one aspect of service quality, but quality also depends on whether the problem was resolved, the accuracy of the response, and the customer’s overall experience. Conversion rate can represent part of commercial performance, although margin, retention, and the value of acquired customers also matter.
As long as the relationship between the KPI and the real objective remains intact, the metric works well. The difficulty appears when the system learns to improve the proxy through a path different from the one we originally expected.
A Well-Known Real-World Case, Wells Fargo
A particularly clear example occurred at Wells Fargo, one of the largest banks in the United States.
For years, the bank promoted a cross-selling strategy, meaning the sale of multiple financial products to the same customer. The business logic was reasonable. A customer using a checking account, a credit card, a savings account, and other products could be interpreted as having a deeper relationship with the bank.
The number of products sold per customer therefore became an important commercial metric, and the organization established highly demanding sales targets.
That is where the distortion began.
According to the U.S. Department of Justice, pressure to meet those targets led thousands of employees to open millions of accounts and provide products without customer authorization or under false pretenses between 2002 and 2016. The practices identified included opening accounts, issuing credit cards, and activating other services that customers had not requested.
The indicator could show more products sold per customer and, from the perspective of a dashboard, appear to signal stronger commercial performance. But the original business objective was to build deeper and more valuable relationships with customers. In some cases, the exact opposite occurred.
The metric continued to rise, but it had stopped reliably representing what the company was actually trying to achieve.
In 2020, Wells Fargo agreed to pay $3 billion to resolve criminal and civil investigations related to these practices. The U.S. Department of Justice itself stated that excessive sales targets and management pressure contributed directly to employee behavior.
The case is extreme, but it is very useful for understanding Goodhart’s Law because it shows something that can also occur in much more ordinary situations. A metric may have been perfectly reasonable at the beginning. The problem appears when it becomes so important that people start optimizing the number rather than the outcome that number was meant to represent.
That is why, whenever we work with a KPI, there is one question worth keeping in mind
If this indicator improves, what evidence do we have that what truly matters is improving as well?