How to cancel a failing project gracefully


A couple weeks ago (August 21, 2011 to be exact), I was watching a “60 Minutes” interview on TV about whistle blowing and a NSA employee’s possible trial for espionage. Interesting stuff, I guess. However, what caught my attention was that the charges revolved around a software development project. (Here’s the full story, if you are interested: U.S. v. Whistleblower Tom Drake.)

The gist of the story began when the NSA needed greater capability to sort through the terabytes of data it collected. Internally there was a tool, believed by the whistle blowers, to meet the NSA’s requirements. However, the Director decided to requisition develop of a new tool, called Trailblazer, that did everything they could imagine needing to be done. Millions of cost overrun dollars later, the project failed and the former head of NSA, Lt. Gen. Hayden told congress, “A lot of the failure in the Trailblazer Program was in the fact that we were trying to overachieve. And that a lot of the failure was that we were trying to do too much all at once.”

Been there. Done that. Got the project coffee mug!

Even when you are project manager for less grand goals than saving freedom and democracy, there comes a time in the life of many projects when the failure handwriting is on the wall. How can you tell when it is time and what is the best way to end a failing project?  Back in 2009, I wrote a post on Spotting a Failing project that was a short piece on the subject. In the current economic environment I have once again been asked to re-visit the subject by several of my clients.  I am not sure I have a nice precise answer to either of these questions, but I will share my thoughts on this often overlooked subject. 

Signs of a Project in Failure Mode

Recurrent cost and schedule over-runs may signal that a project is doomed to failure. Disruptive technologies or changes in the business case for a project can (and should) lead to project cancellation. In addition, customers can lose funding, especially in a recession or on government contracts subject to priority changes. All of these situations provide legitimate reasons to stop working on a software project.

I would like to add a few of my own observations – I have borrowed a couple, some tongue in cheek, from Code Squeeze’s “101 Ways to Know Your Software Project Is Doomed.

Bruce’s top ten signs that your project may be a failure:

  1. The schedule changes every day
  2. No one has heard from nor can they find the project sponsor
  3. The weekly status report is always the same – it never changes
  4. Project resources (staff) are constantly changed out without notice
  5. No one can find the actual requirements document
  6. Every bug is prioritized as Critical and every feature is prioritized as Trivial
  7. Project cost estimates magically match the budget – all the time
  8. The phrase ‘It works on my machine’ is heard more than once a day
  9. All code reviews have been moved to a week before product launch
  10. None of the project team leads shows up for the status meetings; they send a proxy

For those of you working on Government projects who want some insightful information on why government software development contracts often fail, I highly recommend the Clutter IT Journal article, “How do the turkey projects get wings …” by Payson Hall. I also wrote a post about how bad estimating on government projects can lead to failures. The hard part is defining that your project is failing – sometimes you have to re-think what success would be.

How to Terminate a Failing Software Project with Grace

Grace means gaining concurrence from the project’s key stakeholders that stopping an irrecoverably failing project is the best action for all concerned. Or, to quote an old saying, “Don’t throw good money after bad.”

As a project manager, you created a risk management plan during initial project development that included trigger conditions to be monitored to indicate when the project’s success was threatened. I recommend contacting your PMO, portfolio manager or supervisor as soon as the trigger condition occurs. Tell them your plans to investigate the problem and give them a deadline to report results and recommendations. As part of your investigation, talk with your lead technologist in an effort to identify any products from the effort that can be salvaged as deliverables. Do not be offended if managers that are more senior suggest calling in a consultant to get a second opinion or to assist in generating options to project cancellation.

When alerting clients to a potential project failure, it is better to have a face-to-face meeting than to send a report or wait for the next scheduled review. I understand that no one likes to be the messenger bringing bad news. However, if you have done your due diligence in assessing the situation, the sooner you gain concurrence and understanding from customers, the sooner you can move on.

When explaining the situation to customers, present the problem in non-technical language, offer your conclusions and recommendations. If you have a recovery plan, present the options along with their associated cost and schedule implications. If you identified salvageable elements from the project, describe how those can be polished and delivered.

Assuming that you are successful in convincing management, clients and sponsors that it is time to pull the plug, you should still push to perform some key tasks.  Before closing the project, create documentation and lessons learned for inclusion in the organization’s knowledge management system.  I am certain that the organization will not want to repeat the same mistakes again!

Please share your thoughts on how to spot and cancel a failing project.

User Acceptance Testing – Dealing with Clever Users

I have never been particularly fond of the ubiquitous phrase, “where the rubber meets the road.” However, if ever there were a situation when that phrase applies, it is user acceptance testing (UAT). After all the planning, designing, meetings, reviews and internal testing, there is one final act before the buyer (or internal client) accepts the software or solution. Yes, your team and you have worked months to perfect it but the real gate for saying it is ready is when it is tested by real users on their system.

Bad User Acceptance Testing Strategies
Waiting until the end of development to think about user testing: NO-Start early. Plan the user testing from the beginning of the project. Build user testing into the schedule. Do NOT fall victim to the strategy of, “we’ll rely on the results of system testing and skip user testing to catch up with a delayed schedule.”  Those of us with lots of years experience can tell you that you build in testing and most surely with actual users if you want the system to work in the production environment.

Not allowing plenty of time for user testing: In a report from Sentry, they found that (and I paraphrase from their report: “assume that you have 120 test items such as producing a specific report or scanning 10 batches of 30 documents each into a database or generating an export file of 600,000 records. Further, assume it takes about 3 hours to setup each test cycle. Do the math. The time to setup, test, collect data and analyze approximates 360 hours.)  So how many projects build that into their schedule?  The ones that do I will bet find lots of real world issues and work them out before deployment.

Your project’s testing requirements will differ from any other projects because your product and situation are unique. However, the point is that meaningful and useful user testing takes time and planning.  You might even do something novel like include users up front in planning what testing and acceptance criteria they will use!

Here’s another failing user testing strategy from our friend, Dilbert.

OK- so enough bad strategies, how about a few tips on what to do right:

Good Advice on User Acceptance Testing:

  1. Include user-testing requirements in the original proposal, plan, cost and schedule. Use input from your business analyst or project team – that include customer representatives – to generate typical scenarios and scripts for user testing.
  2. Create a detailed plan of what the UAT is testing. You do not want a bug report that says, “The system failed to turn straw into gold.” User testing should be done based on agreed requirements, not everything a user may want the system to do.  In other words, bound the scope of the testing to be in line with the solution.
  3. Select test subjects (let’s call them users) with a range of experience and diversity of backgrounds following pre-defined scripts to get from task beginning to task complete.  Do not pick all junior or all experienced users, a mix is the best.
  4. Train your users (test subjects) how to report problems. “User Acceptance Testing War Stories – Writing a Perfect Bug” suggests that testers write down the steps they followed that led to failure. Then wait a few hours and try to follow the steps as they wrote them to see if the bug appears again. You are teaching testers how to find bugs and report them with sufficient detail to developers to repair.  Use some online tool if possible – it makes it easier.  Even a SharePoint list can be used to track the items that come out of the UAT.
  5. Maintain detailed reports of testing including procedures and outcomes. Communicate findings with the team, management and clients. Depending on the timing of the UAT, you may want to submit status reports daily or weekly. Besides a cursory pass/fail, add some analytical information about causes or commonalities.

Remember that user testing is about making sure the system does what the customer claimed was needed to help them do their job. If the specifications and earlier plans, all agreed to by the parties involved, resulted in a product that does not support user’s needs, document the revised needs and begin the process of negotiating the follow on contract.

Best Practices and Lessons Learned:
This is one of my favorite topics and I have made almost every mistake from cutting out user testing to letting users test things not included in the original scope and requirements.  A couple of Lessons learned that I hope you can avoid:

  1. Get the client/stakeholders involved at the beginning to define what the testing will need to be.  It is like having the students help write the test – they can’t say it was a bad test.
  2. Balance the formality of the UAT with the scope and length of the project and solution.  I have seen too much formality for a small system acceptance and not enough rigor for a large scale solution being deployed for thousands of users.
  3. Don’t start out user testing or UAT with negative points – in other words don’t sound like a lawyer!  If you have done everything right, including getting the users to help up front, then this is an exciting time where they get to try out the system before everyone else.  Kind of like test driving the new model car before everyone else.

So what are your tips or experiences in UAT?  I am sure others would love to hve you share.

Thanks for sharing.

The Lazy Project Manager's Blog

The Home of Productive Laziness Thoughts

Thoughts, experience, tips and tricks on issues affecting managers and project management

A Girl's Guide to Project Management

Project Management musings for one and all

LeadingAnswers: Leadership and Agile Project Management Blog

Thoughts, experience, tips and tricks on issues affecting managers and project management

Project Management Hut

Thoughts, experience, tips and tricks on issues affecting managers and project management

Herding Cats

Thoughts, experience, tips and tricks on issues affecting managers and project management


Pushing the Edges Out ...


Just another site