IT Project Success: Getting Better, but Big is still Bad

The biennial “Chaos Report” on IT project success from The Standish Group reports that the success/failure ratio flipped between 1994 and 2006. In 1994 the ratio for “flat failures” vs. “complete successes” was a depressing 31% vs. 16%; in 2006 it was a more encouraging 19% vs. 35%. (The work is reported in CIO; the Standish Group web site is remarkably sullen, and doesn’t seem to have any press releases, let alone publicly available recent data.)

On page 2 of the CIO story, the Standish CEO says: “Seventy-three percent of projects with labor cost of less than $750,000 succeed. . . . But only 3 percent of projects a with labor cost of over $10 million succeed. I would venture to say the 3 percent that succeed succeeded because they overestimated their budget, not because they were managed properly.” A $750,000 project is pretty tiny: six developers for six months, at $250,000/developer/year fully loaded. Even a $10 million project is 20 developers for two years.

This result matches received wisdom that large projects are more likely to fail, which I attribute at least in part to the cognitive challenge of wrapping one’s head around large problems.

What should one do about it? It implies that smaller projects are the only way to go – but what if one has ambitious goals? If it’s true that one can construct complex solutions out of many small, simple parts, everything’s fine. But I’m deeply suspicious of the “divide and conquer” or “linearization” assumption. There are many important problems that just can’t be broken up, from inverting a matrix to simulating non-linear systems.

This may be a cultural reality check: many ambitious goals may simply not be achievable. Humility may be the best way to ensure success. I doubt politicians and business executives want to hear this. Trying to fly too high brought Icarus down – exactly as his engineer-father Daedalus had warned.

And things may not get better: as technology progresses, the complexities of our systems will grow, and linear solutions become even less useful. As the interconnectedness and intangibility of society grows, we may have to become more humble, not more bold, because that will be the only way to get stuff done. It’s counter-intuitive that as technology progresses we need to become less, not more, ambitious, but that may be the way things work with the new intangibles.

NOTES

My thanks to Henry Yuen for referring this story.

I have some reservations about the Standish data. It’s proprietary, and there are academics who’ve questioned it for years. CIO provides some background on Chaos Report and its methods in an interview with the CEO; it also summarizes questions about their method. One has to wonder how the sample population has changed over the years. If the number of small projects in the sample has grown over time, then success reported above would increase simply because smaller projects fail less often, not because project management performance has improved.