Hope is Not a Risk Management Strategy

To many people, software projects are a lot like black magic. You are trying to build something that no one knows the contents of until it’s been built, in a way that responds to change, in some semblance of a reasonable timeframe and budget. Everything from the estimation process, to the status calls, to the change control, to the releases themselves vary so wildly that we actually applaud when a team “makes it”. “Woo!” we say, holding parties, and sending mass emails, “We did our job!”

Because of this, we for some reason forget that there are very real strategies for dealing with the inherent risks in a software project. And I don’t just mean the “x = Programmer’s Estimate; ProjectTime = x*2.3 + 42” trick.

Frequent Releases of Working Software

Since there are so many variables in getting a customer what they want, a wonderful strategy is to frequently release working software, letting the customer hold it in their hands, push all the buttons and generally see how wrong they were about what they wanted. And I’m not talking about a “beta” here – I’m talking about one week in, shipping something. Of course, there are two reasons why you don’t see this more often:

• Fear. “If the customers see the product, they will want changes! Nay! Demand them!” Or, “The customers will see it crash, and lose all hope in the product!” This can be overcome by, well, doing it more often. You want your users on your side, and I’d much rather find out after the first week that the main UI screen will never work because we have 80,000 users trained to a green screen, and they really need it to look just like that.
• Skill Deficiencies – I recently heard someone say that their project was going to take 6 months before anyone could test anything. I don’t even remember the reason why. 6 months of coding? Really? That’s a skill deficiency, and if you are a manager, shouldn’t stand for it. Yes, there will be rare times when something isn’t building, but that shouldn’t last more than a day, tops. Our current product is 4.5 million lines of code, and we have had nightly releases every night for the past 9 months. Releases that were in the hands of the business the next morning that they could play with.

Frequent, Open and Honest Communication

One of the things that can really trip up a team is the Lack of Trust (as documented in the wonderful book 5 Dysfunctions of a Team). The funny thing is, many teams adopting agile practices start with perhaps the worst practice of all – stand-ups. Effective stand-ups rely on a high level of trust and a willingness to put yourself out there, and to challenge others when they aren’t. A team who doesn’t have that typically finds stand-ups that end up not actually exposing impediments.

But it’s more than just stand-ups. People need to feel empowered to stop the line, and to report when things aren’t going well. For example, if you are using the Scrum methodology, and have iterations, the team commits at the beginning of the sprint of what they are going to accomplish. If they find they can’t accomplish it, and don’t raise whatever is causing them to not get there until the end, then there is a serious organizational impediment to communication that needs to be resolved.

Of course, communication is a two-way street. In addition to the teams feeling comfortable about talking, the organization – especially management – needs to be willing to listen, and to trust what they are hearing. And when the organization does run into trouble, to stay away from The Blame Game, and focus on feedback loops which actually improve the process.

Team members as well need to be willing to listen, both to what is being said to them, but also to the little clues that indicate something is wrong. Mitch Lacey has a great article about a team who messed up a demo, and found out that someone on the team knew it was going to happen – but it wasn’t raised due to a lack of trust and communication.

Strategic Directions

In any agile team, or any software team, one key tenant is a common goal. Without a clear strategic direction the team can rally behind, the work they are doing might be lost. On a smaller scale, this is why it is so vital to frame the work you are doing in the context of what you are delivering to a customer, either in the form of User Stories, or Minimal Marketable Features. Someone needs to be keeping their head up of how it ties together.

In fact, in larger scale agile adoptions, you typically do find it necessary to have a strategic team that helps organize the work. One must be cautious that they don’t turn into an ivory tower, but teams that do this well help weave that common thread which lets everything come together.

An example of this role done well is the Chief Product Engineer at Toyota. They know both the business and production sides, and are empowered to make decisions – but know how to keep everyone focused on the bigger goal.

Exit Strategies

Even if you are communicating, and releasing, and have a common goal, there still may be times when things are out of your grasp. For example, working with another team, or a third party. In these cases, having an exit strategy in mind can make the difference between an Ok release and a failure. These are some of the hardest things to talk about, because no one likes to admit that failure is possible. Worse, sometimes the exit strategies require actions to be taken before a “final deadline”, giving the impression that if we just kept working a little harder, we could make it.

In the article No Exit, Don Gray outlines beautifully the causes of this, and how to prevent it from happening. In general, anything that has a risk factor, which is vital to the release, should have an exit strategy. This can be everything from software components, to deployments, to configuration.

For example, at one organization, we deployed daily to our production servers. We had a highly visible web application, used by hundreds of thousands of people daily. No matter how much testing we did, we knew there was always a risk. So we had a strategy that with one button we could switch all traffic from 16 servers to 8. We’d then deploy to the servers which had no load, and test. If it seemed fine, we’d push another button, and everyone would be switched to the 8 servers we just deployed to. We’d let that run for a bit to make sure everything was fine, and if it was, we’d deploy on the other half, and balance back out the traffic. If anything failed, we could back the whole thing out.

Another strategy example was a deployment of real-time polling updates for a large county. This was in 2000, and it was vital that it go smoothly. We had tested the software extensively, and felt confident in it, but had a backup plan where two of us had a set of pages we could update by hand that the polling stations could call into us in case the automated system broke down. Luckily we were able to just sit at work until 1am playing cards because everything did go smoothly, but if it hadn’t, we were ready.

Unfortunately, we don’t do the same for systems we work on all the time. What if that component team can’t get their UI finished? What if the vendor can’t get the bug fix in time? It may not be vital early on, but the thought process behind an exit strategy should exist, and simply become more refined the closer we come to the risk event, instead of scrambling to make one appear when it is clear we aren’t going to make it.

Hope Isn’t a Risk Management Strategy

What all of this adds up to are clear ways you can begin to mitigate the risks of your projects. Sure, many other things will come up. A nd there are lots of great papers on the subject of Risk Management. But, what it comes down to is shipping frequently, communicating often, and having an exit strategy for when things go wrong. Hoping that everything will be fine will only get you a bad delivery – or a new job.

What About Faith in the Team as a Strategy

When I posed the title of this to Twitter earlier, someone asked about Faith being a Risk Management Strategy. Faith is different than hope, although they get mixed in. Faith is saying, “Based on what I see or know, I’m confident that what I’m being told is true”. It’s about trust. Hope is saying, “Even though it looks like things are horribly wrong, I’m sure we’ll make it”.

In other words, faith is based on results. When those results falter, one loses faith in the results, and ultimately in what they are being told. Hope is suspending reality to make things seem better than they are. I hope I win the lottery, but I don’t have faith that I will.

The problem is that we’ve all heard, “I have faith you’ll do it” where “it” is some impossible thing. Teams hear this all the time, “I know you said this will take you 6 months, but I told management 3, and I have faith you all can do it by then.” That’s not faith. Nor is it “motivation”. It’s mismanagement.

There are many ways to ship software, and many teams who have had successful releases by sheer will of force. But a responsible team will employ good risk management, communication and release strategies to ensure that no matter what change comes at them, they are ready for it.