I think we have an issue in the field of project management. Defining project success and failure only in terms of on-time and on-budget performance is simplistic and wrong. Software projects are about solving problems for customers.
I have seen countless resumes from aspiring project managers and performance reviews that claim proficiency in project management measured by these two factors only. While they are important components of project management, being on time and on budget does not mean that the end result met the customer’s needs.
On the flip side, being over budget or behind schedule does not mean ipso facto that the project was necessarily a failure. In the real world of software development, the initial requirements that drove the schedule and budget will change. The business environment is always in flux and the problems the customer wants solved will be shaped by changes in their operating environment. Let me say this one more time: change is a constant and must be accounted for in any project.
That shift in priorities and constraints is the key reason that project managers must be in constant contact with their customer and stakeholders.
The need to include quality and user satisfaction as part the duties of a project manager is not a new idea. The DeLone and McLean Information System success model from many years ago offers additional factors that should be considered in evaluating the success of a software project. Here are their suggestions summarized in Defining and Measuring Project Success by Danie van der Westhuizen:
• System Quality: measure of the information processing system itself
• Information Quality: measure of information system output
• Information Use: measure of recipient consumption of the output of an information system
• User Satisfaction: measure of recipient response to the use of the output of an information system
• Individual Impact: measure of the effect of information on the recipient
• Organizational Impact: measure of the effect of information on organizational performance
I think their last criteria—organizational impact—is really important to track as a measure of success or failure. If a piece of software or even a new process is timely and cost efficient but is never used by the customer, to me that is a project failure. On the other hand, if a project evolves to meet customer’s changing requirements, is used by the customer, and improves the customer’s bottom line—that project was successful even if it took more time to complete and required modifications to cost and scope.
Another interesting perspective on expanding the definition of project success and failure was presented by Scott Ambler writing for Dr. Dobb’s Journal. He reports on a 2007research project that surveyed 105 professionals including project managers, IT managers, and business stakeholders about how they comparatively ranked three common project success criteria:
|
% who agree with this statement |
Project managers |
IT managers |
Business stakeholders |
|
Shipping when the system is ready is more important than shipping on schedule |
50.3% |
63.1% |
73.3% |
|
Providing the best ROI is more important than delivering under budget |
62.4% |
76.9% |
86.7% |
|
Delivering high quality is more important than delivering on time and on budget |
78.2% |
87.7% |
86.7% |
Clearly a justifiable conclusion from this small survey is that customers are more interested in the utility of the resulting software product than they are in forcing compliance with schedule and cost parameters set at the beginning of the project. It is up to the project manager to work closely with the business stakeholders throughout the development cycle to clearly explain the reasons and benefits of modifications in schedule and cost, so that everyone ends up convinced in the delivered project’s success.
One last thought or rather caveat—all changes have an impact or cost. While you should always be ready to accept change, it is not something done lightly or without judging the impact on cost or schedule and then getting buy-in, approval or consensus.
Your thoughts on measuring software project success and failure are invited via comments.
