Dan Milstein from Hut 8 Labs has written a lengthy post about why software developers often struggle to estimate the time required to implement their projects
. Drawing on lessons from a book called Thinking Fast and Slow
by Dan Kahneman, he explains how overconfidence frequently leads to underestimations of a project's complexity. Unfortunately, the nature of overconfidence makes it tough to compensate. Quoting:
"Specifically, in many, many situations, the following three things hold true: 1- 'Expert' predictions about some future event are so completely unreliable as to be basically meaningless 2- Nonetheless, the experts in question are extremely confident about the accuracy of their predictions. 3- And, best of all: absolutely nothing seems to be able to diminish the confidence that experts feel. The last one is truly remarkable: even if experts try to honestly face evidence of their own past failures, even if they deeply understand this flaw in human cognition they will still feel a deep sense of confidence in the accuracy of their predictions. As Kahneman explains it, after telling an amazing story about his own failing on this front: 'The confidence you will experience in your future judgments will not be diminished by what you just read, even if you believe every word.'"