CHAOS Chronicles, focusing on failures and possible improvements in IT projects
1 July 2016onderzoeken
Jim JOHNSON, Chairman and founder of the Standish Group International Inc.,
Boston, Massachusetts, USA
Prof. dr. ing Hans MULDER, University of Antwerp Management School,
The Standish Group started in 1985 in the business of IT market forecasts and predictions using Artificial Intelligence and cased-based reasoning technology. In 1994, we turned to predicting project outcomes, improving software development, and building a world-class database. Standish’s cumulative research encompasses 22 years of data on why projects succeed or fail, representing more than 50,000 active completed IT projects. In this paper we clarify how we got here, where we are, and how academia next to practitioners can be part of the next stage of the CHAOS journey. The vehicle that drives our journey is the CHAOS University System.
Keywords: Information Technology projects; research of success, failure, and value factors; education and research.
INTRODUCTION OF THE JOURNEY
8:31 a.m., Nov. 26, 1968. I drove my 1967 Volkswagen Bug into the upper Parker Street parking lot of the old mill building. It was a crisp fall day. The dried leaves crunched under my feet as I walked from my car into Digital Equipment Corp.’s then one and only executive office and manufacturing facility. “You know, Jim, we only have today to get the project done and turn it over to the owner tomorrow for his final acceptance,” said Al. Thus began my journey and fascination with software projects . Over the next 18 years my life would be around building, and sponsoring, software projects to help grow the IT industry. These projects included the job bank for the unemployed, trading systems for both traders and stock exchanges, bank ATM and retail payments systems, manufacturing and ERP, airplane and airport monitoring systems, and many other types of applications.
START YOUR ENGINES
In 1985, The Standish Group started with the simple premise that if you wanted to understand the marketplace you would have to ask a lot of people a lot of questions up and down the spectrum. We needed to do in-depth research to really understand the marketplace and the potential directions. From then until about the year 2000, our major business was predicting future trends, and we had about an 80% success rate. However, in 1994, we started the transition from a market research firm to a project advisor organization. In 1994, we did our normal three-stage research, which consisted of interviews, focus groups, and surveys to gain a basic understanding of the issues around project success. This basic research created the original 1994 CHAOS Report .
At the time, we expected this report would be a one-time event and we would move on to continue to look at emerging technologies. That would not be the case, as people wanted more; and the more they wanted to understand, the more we wanted to understand. The more curious they were, the more curious we were to find the answers for them and us. During the next ten years we created many different events and instruments to understand more about the issues and try to come up with solutions. We would write many research papers and even books on the subject of project success and failure. We wanted to dive deep into the abyss to find that Holy Grail that would make software projects successful. By 2004, we realized there is no Holy Grail; there are no right answers. There are only more questions with some very clear directional signposts.
THE STANDISH DATABASE
So, what is new and what seems to remain the same in IT projects over the last 20 or 30 years? It has been 22 years of CHAOS. During this time we have examined 110,000 projects, countless workshops, and numerous benchmarks. It has taken this long to create “the winning hand” for project success.
There are five things you need to do to create a winning hand. First, a project needs to be small. This means six team members (maximum) with a time box of six months or less. Second, the process must agile, such as the Scrum methodology. Third, the agile team must be highly skilled in both the agile process and the technology. Fourth, the product owner or sponsor must be highly skilled. And fifth or last, the organization must be highly skilled at emotional maturity .
FINDINGS FROM THE CHAOS DATABASE
If you do these five things and do them well you have an 81% chance that a project will come in on time and on budget, with satisfied customers. You have only a 1% chance the project will fail and only an 18% chance it will be challenged in some way or other. More importantly, the project will have a 64% chance of returning very high to high value and only a 15% chance of returning no to low value. If you do not do these five things well, however, the chances of a failed, challenged, or low-value result increase. On the other hand, everything else you do (outside of these five things) is most likely a waste of time and money or has very low to negative impact.
 B. Joy and K. Kennedy, Information Technology Research: Investing in Our Future, tech. report, President’s Information Technology Advisory Committee, Feb. 1999.
 J.B.F. Mulder and J. Johnson, The Next Step of IT Project Research in Practice: The CHAOS University System, The 10th International Multi-Conference on Society, Cybernetics and Informatics: IMSCI 2016, July 5 - 8, 2016 – Orlando, Florida, USA.