Anyone who has been around projects long enough has had the experience of a troubled, or at least challenged project. As a problem project guy, I've been through my fair share of challenged projects. In the last few years since becoming an Agile enthusiast, I've been using Agile to recover troubled projects. Mostly it works. (Hint: it doesn't work if you violate the Agile Principles and Values). This is the third in the series on applying Agile to recover troubled projects.
The Problem Project
This particular project was actually a program of 5 projects that was underway for 7 months and hadn't delivered. The program was most likely oversold by the vendor, and underscoped and understaffed by my client. It was also being driven as a technology project by the software vendor.
The most apparent sign of trouble to outsiders was the pattern of missed deadlines. This was in spite of being reported as almost done, which was the other sign of trouble. The status reporting was of the "good news only" variety.
For various reasons, nothing was ever "done done", though lots of things were in process and some were reported as almost done. When one particular piece of work was stalled, additional threads would be started in an attempt to get something done. It felt like a good strategy - start more things in the hopes that more things would finish. The results are just the opposite. It was like sending more cars down a freeway that had a traffic jam - it actually made things worse.
The technology was being driven by the software vendor with little true engagement with the business. The end users didn't understand the software and had little input to the requirements effort that was led by the vendor. It was little surprise then that much of the planned software was determined to be unnecessary when it was reviewed and actually understood by the business.
After 7 months, the project team was overwhelmed with the volume of work. The organization's culture worked against the team; 'do whatever it takes', and never say it cannot be done were the norms. So there was a sense of resolve, of just hunkering down and brute forcing the project over the line. Even as they fell farther and farther behind, and deadlines looked more hopeless, the team responded by worked harder and harder. And they threw more people at the problem. Unfortunately this approach doesn't scale up, nor is it sustainable for more than a short period of time.
As we began planning for the Agile transition, what immediately became clear was the incredibly inefficient processes in place. Data conversion was probably the most complex challenge, and the data conversion process involved more handoff's than the Olympic Torch Relay. (*Technically this is not true, the London Olympics Torch Relay involved 8,000 handoffs). Data was gathered and entered into templates by the local team, then sent to the vendor office on the West Coast where it was entered into other spreadsheets. Those spreadsheets came back to the local team with questions that had to be answered. Then they were sent to an offshore team for entry into another tool for conversion. The offshore team, lacking clear business rules, simply logged the issues and sent the data back to the local team for resolution. This created long queues and waiting time between the various handoffs. Work piled up and rework became common. Because of the number of people involved, and the lack of clear process, conversions were done over and over again using multiple interim deliverables. Process changes would render the interim work out of date and the team would start over with fresh data. The majority of the communications took place via email, with very few face to face meetings.
Various people were trying to fix the broken processes but the reality was, everyone was working so hard that they were fatigued, and no one wanted to stop and think about the potential futility of their actions. This is a very frequent occurence on problem projects, especially death march progress. The team members get tired, overwhelmed and they throw themselves into the work, losing all objective perceptions of progress and reality. It is actually quite similar to what happens in wartime. The phrase "The Fog of War", coined by Prussian Military Strategist Carl von Clausewitz, speaks to this concept:
“War is an area of uncertainty; three quarters of the things on which all action in War is based on are lying in a fog of uncertainty to a greater or lesser extent.”
- Carl von Clausewitz
The Agile Transition
So we were engaged by the project sponsor to transition the project to Agile. The sponsor was not initially a fan of Agile, but I suspect the mounting project costs and repeated delays and lack of predictable end dates left her desperate enough to try it. With her securely on board, we trained 5 teams and worked with them to create the Scrum artifacts like prioritized product backlogs. We installed Product Owners from the business teams and put in place the strongest Scrum Masters available. The sponsor was directly engaged in the initial replanning sessions. She contributed to the team vision and values, and helped foster an attitude that this could be done.
The Agile transition wasn't immediate or pain free. The project had been underway for 7-8 months before the move to Agile. It took two or three iterations, lasting two weeks each, before there was some clarity on what work remained and when the project might finish. Those finish dates continued to firm up after four and five iterations.
The teams consistently used the typical Scrum processes like iteration planning, daily standups, demos and retrospectives. Each team had a task board and we used a common team room for those task boards and the daily standups. In addition, weekly steering committee meetings and two 'scrum of scrum' type meetings were used each week to esure or confirm alignment of the efforts and facilitate cross team communications.
Another tool that was helpful in aligning the teams was having common plans. A high-level release plan was used to create a framework for major milestones such as the go-live date. A more granular iteration plan showed how the deliverables from the 5 teams would be delivered over time. This iteration plan gave the leadership team the type of granular visibility and informed decision making that was needed. It was reviewed each week, and changed or re-prioritized when appropriate.
The Payoff - Progress!
The beautiful thing about Agile is that when you apply it properly, it just works. Work is prioritized, and teams begin delivering useful solutions early. Iteration progress is visible on the tasks boards. Business and technical teams work together daily. As my friend and experienced project manager Cam used to say, "Hey, this stuff isn't rocket surgery!"
Some of the immediate and important benefits of this Agile Transformation:
#1 - Progress Transparency - No longer was the status cloaked in % complete ambiguity. There was a defined backlog of work that was not done, and the teams completed some of it every iteration. You could immediately see the progress of any team in any iteration just by looking at their task board.
#2 - Role and Task Transparency - Progress transparency extended down into the tasks of the backlog so that it was clear what each individual was working on and responsible for within every iteration (see color coding above). As part of the Scrum framework, team members developed detailed tasks to support the completion of the user stories, assigned them, and tracked them to completion. Daily standups were used to track process at a detailed level. The end of iteration demos served to prevent the teams from going too far off the rails.
#3 - Predictability - With common release and iteration plans, and an average velocity for the teams, it became easier to determine the ability to deliver critical functionality. Not only was there visibility to the plan, the teams were able to be realistic in their commitments, and deliver against their plan with some level of certainty.
#4 - Team Focus - A critical aspect of Agile is to focus the team's effort like a laser beam. Chasing a lot of work at the same time is counter productive, though it is often an attractive way of working. After the transition to Agile, each team scoped what could be done in the next two weeks and just took that much on. By prioritizing, and focusing on just one or two items critical to the go-live, the teams were able to eliminate a lot of unnecessary work, and speed things up considerably. "One is better than none" was an early theme of the re-planning effort. (Focus is also a key Lean principle, though we will save that for another post.)
“One is better than none.”
#5 - Building the Right Solution - The use of the backlog that was prioritized by the engaged business stakeholders meant that the next important thing was clear. It also meant that we were building the right solution.
#6 - Collaboration and Working to Common Objectives - By aligning the 5 teams to the release and iteration plans, it became much easier to get teams focused on hitting release dates. There was a sense of cooperation across the team. This was supported by the development of the success criteria and elevator pitch, which the project sponsor participated in creating.
Keys to Success:
Active Direct Leadership from the Business Sponsor - I've mentioned the business sponsor and her direct involvement in the program. That was one of the biggest factors in the success. The sponsor participated in the development of the charter and vision. The sponsor held weekly steering committee meetings. The sponsor assigned the correct people to be part of the engagement, from across the business and technical teams. And the sponsor turned up at the end of iteration demos, asked good questions, and thanked everyone for their hard work. That type of active involvement is important. As one of my favorite leaders, Bill Hybels, said recently, one of the most important roles of a leader is to energize the team.
Full Agile Adoption - There was no wishy washiness about the Agile adoption and discipline. Driven by the business sponsors, everyone on the team embraced it. A few may have had their doubts about the efficacy, but that lasted only one or two iterations. Once it was clear that the program was healthy and all teams were delivering against their prioritized backlogs, agile criticism faded.
Getting the Right People On the Bus - One of the principles that Jim Collins taught in "Good to Great" was that leaders need to get the right people on the bus.
“...leaders of companies that go from good to great start not with “where” but with “who.” They start by getting the right people on the bus, the wrong people off the bus, and the right people in the right seats.”
- Jim Collins, Good to Great
This same principle applies to projects in trouble, and to Agile projects generally. In fact, one of the 12 Agile Principles speaks directly to this:
"Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done."
As the project transitioned to Agile, new leaders were put in place in various roles including the Scrum Masters and Product Owners, and other technical and business leaders. Conversely, it was also necessary to get some people off the bus, which is also pretty common for the recovery of troubled projects. I would argue that it is vital that the old guard be moved out of the picture to make space for new leadership, ideas, and ownership.
Agile Principles Most Relevant to Success:
To conclude this post on this successful project recovery with Agile, I'd like to highlight what I think were the most important Agile Principles to the success of the recovery.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
- Simplicity--the art of maximizing the amount of work not done--is essential.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
I'd love to get your comments, or hear your stories about project recovery or agile transformation.
Cheers!
Anthony
