Eliezer Yudkowsky on the planning fallacy

The planning fallacy is that people think they can plan, ha ha.

A clue to the underlying problem with the planning algorithm was uncovered by Newby Clark et al., who found that:

– Asking subjects for their predictions based on realistic “best guess” scenarios; and
– Asking subjects for their hoped-for “best case” scenarios…

… produced indistinguishable results.

When people are asked for a “realistic” scenario, they envision everything going exactly as planned, with no unexpected delays or unforeseen catastrophes – the same vision as their “best case.”

Reality, it turns out, usually delivers results somewhat worse than the “worst case.”

Unlike most cognitive biases, we know a good debiasing heuristic for the planning fallacy… Just use an “outside view” instead of an “inside view.”

People tend to generate their predictions by thinking about the particular, unique features of the task at hand, and constructing a scenario for how they intend to complete the task – which is just what we usually think of as planning… You have to plan out where, when, how; figure out how much time and how much resource is required; visualize the steps from the beginning to successful conclusion. All of this is the “inside view” and doesn’t take into account unexpected delays and unforeseen catastrophes…. Asking people to visualize the “worst case” still isn’t enough to counteract their optimism – they don’t visualize enough Murphyness.

The outside view is when you deliberately avoid thinking about the special, unique features of this project, and just ask how long it took to finish broadly similar projects in the past. This is counter intuitive, since the inside view has so much detail – there’s a temptation to think that a carefully tailored prediction, taking into account all available data, will give better results.

But experiment has shown that more detailed subjects’ visualization, the more optimistic (and less accurate) they become.

Eliezer YudkowskyMap and Territory

I'd love to hear your thoughts and recommended resources...