Scott Langille

Goodhart's Law vs. Product Management

Goodhart's Law states: "When a measure becomes a target, it ceases to be a good measure." (Or, more formally: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.")

Because people "game" the metric, it's no longer a good metric. Unless, as George Hotz contends, the metric is "an evaluation function beyond the complexity that humans can model."

This seems to be in contrast with most of the outcome-oriented "North Star metric"-driven product management theory out there. So how can both ideas co-exist?


Interesting article: Measuring Goodhart's Law by OpenAI