[Reading time: 52 seconds]
Yesterday I wrote about Goodhart’s law: all metrics will be gamed.
And I told you about ways to manage this on a human level. But that only gets you so far.
But here’s another, even neater trick: never use a metric in isolation!
Always find pairs, or even entire networks, of metrics that balance each other.
Maybe you care about deployment frequency. But naively, deploying half-baked unfinished stuff would do wonders for your deployment frequency metric. It would just mean that your product becomes unstable, angering users and obviously serving no one.
So how about you also measure deployment failure rate?
That way, if someone (knowingly or not) over-optimises for deployment frequency, you’d be alerted to it by a rise in deployment failure rate.
This is what I call defensible metrics: ones that are guarded like chess pieces, offering insight and information while giving you a fighting chance to keep the system as a whole healthy.