Subscribe
Posted on June 3rd, 2010
Yesterday at the XP2010 conference in Trondheim, Norway I gave a talk on Growing and Fostering Software Craftsmanship. The session was recorded by the conference, but I also recorded using my Flip camera. I put my video online, as well as the slides, and I’m hoping to also be able to post the conference video once it is available. (UPDATE: The conference video is available here, though it is Silverlight only)
Thanks so much to everyone who came out!
Growing and Fostering Software Craftsmanship from Cory Foy on Vimeo.
Growing and Fostering Software Craftsmanship
View more presentations from Cory Foy.
Posted on November 12th, 2009
This week, I had the distinct pleasure of co-presenting with the ever lovable Mitch Lacey at the Agile Development Practices conference in Orlando. I also got the chance to have a great conversation with Alan Shalloway about the state of Scrum and the Scrum Alliance, and to run into Alistair Cockburn.
This got me thinking about the whole ecosphere that makes up agility in software today. There are so many angles one has to consider – from development practice changes, to team and project management changes, scaling up to organizational issues such as portfolio management, risk management, and avoiding local optima by focusing on the flow of value through the organization. And it we look at the practices and methodologies out there today, one could say that the cure to what blocks value in an organization is X-LACKS:
- X – Extreme Programming – This focuses primarily on the software development team, though with an important nod of the importance of customer involvement at all times
- L – Lean Software Development – Lean has been around for a very long time in the manufacturing world, and had a strong applicability to software organizations.
- A – The Agile Manifesto, the founding document outlining the values and principles behind the agile movements.
- C – The Crystal Methodologies are a family of methodologies which provide a sense of what tradeoffs and restrictions have to come into play as the size, scope and criticality of the agile project increases.
- K – Kanban is part of the Lean principles, and has twisted to have a split meaning. The Kanban board is a specific artifact, similar to the Information Radiator from XP or the Sprint Backlog from Scrum, but with an important twist – statuses are limited to the number of items in a column at a time. This is called limited WIP (Work-in-progress). This concept of limited WIP is applied at the broader sense to the team under the same term of Kanban.
- S – Scrum. Ah yes. Scrum. Three Artifacts, Three Rituals, Three Roles. One of the great love/hate stories in the agile world.
Just like ex-lax is made up of multiple ingredients, each with a specific purpose, so goes the above list. People like to make blanket statements like “Kanban is better than Scrum” or “You still need XP in Lean”. But pay attention. The people you want to listen to will provide the context like “…when you are adopting a large-scale organizational transformation” or “…when you are working with a company whose primary function is software” – and that context is what is necessary to know if the medicine you are using is appropriate.
The agile practices aren’t snake oil. They really do work. And the information they expose can make it seem like you’d rather stay sick. But if you address it with the right combination, and with the right leadership, vision and values, you’ll end up with an organization magnitudes better than what you could have ever achieved before.
Posted on September 10th, 2009
When you think about the hardest part of Scrum, it isn’t the daily stand-ups. Nor the sprint backlog. Not even the demos at the end. The hardest part – and the oft missed part – is “Inspect and Adapt”. Inspecting and Adapting is at the heart of Jurgen’s ScrumBut post. One tool that teams have for this inspection is retrospectives. Not that this is new – there are lots of tools and names similar efforts go by – After Action Review/Report, Hotwash, and the ever popular, “What the hell is wrong with you people?”
To help counter this, Henrik’s Scrum Checklist helpfully includes three subelements to think about for retrospectives:
- Results in Concrete improvement proposals
- Some proposals actually get implemented
- Whole team and PO participates
This also can be tied back to developing SMART results – Specific, Measurable, Attainable, Relevant and Time-bound. And a sixth element – assignable. Someone should take responsibility for implementing or leading the implementation of the solution – and that person shouldn’t be the manager or ScrumMaster.
But is the end of a sprint the only time we can learn from what has happened? Of course not. In fact, we often laud performance review processes for that reason – they aren’t done until the end of the reporting period, and by that point the learning opportunities may have already been lost. Obviously with shorter iterations the mean time to inspection is lower, but it doesn’t mean we can’t become more lean in our approaches to introspection.
Short, yet oh so sweet
The first thing is to try having them every week if your iteration length is greater than a week. You may have to change the format up some, but the shorter cycle times may allow things to be caught much earlier. Like standups, you can try them at the end of the week, or kick off the week with a retrospective. Or, if things are being tied too heavily, put it in the middle. Adjust and tweak to find what works well for you.
One of the more interesting approaches I heard was from someone at KaizenConf last year. They don’t have scheduled retrospectives. What they do is have a spot on their big, visible wall for items that need to be brought up to the team. When it hits a certain limit (1 or 2 usually) they have a retrospective – on the spot. I like that because if people aren’t feeling comfortable calling something out, they can still post it on the wall, and effect the discussion through that means.
Out of sight, out of mind
Yet, even with all of these, I still find it important to have slightly more formal retrospectives on a regular basis with teams. After all, part of the exercise of retrospectives is discovering something that everyone only had a small piece of. And these little pieces can grow and become infected if left completely unchecked.
But yet, even with all of the acknowledged benefits, teams end up not checking this item on the list. Why is that?
- Haven’t cleared the 5 dysfunctions hurdle – Retrospectives require a high level of trust and a culture of openness. For many teams, that just isn’t the case.
- You have a poor facilitator – Effective retrospectives require a strong facilitator. There are lots of antipatterns when it comes to facilitation, and any of us who have ever been in that role can probably relate to one of the antipatterns on that list.
- The team isn’t empowered to make the necessary changes – This can be heartbreaking when the team identifies the very things keeping them from doing a great job – and can’t fix it. In some ways it’s horrendous – they now very much understand what is causing them pain, and know the fix, but can’t apply it
Luckily there are steps. Esther Derby and Diana Larsen have written a great book called Agile Retrospectives which covers a variety of games and skills one can use for holding effective retrospectives.
It’s still smelly
Perhaps the most interesting argument against scheduled retrospectives is on Doc List’s blog post Are retrospectives an antipattern?. In one of the comments, Scott Bellware says:
Scheduled retrospectives are a strong indicator that a team’s management isn’t sufficiently perceptive enough to know exactly when actions like facilitation, drawing out information, deciding how to move forward are needed based on observable phenomena.
And there’s the crux. For high-performing teams, who are adept at inspecting and adapting, modifying retrospectives to be part of a larger value stream becomes second nature, especially when they have a management team highly tuned into the team. But without that, teams need the ability to build those skills in inspecting and adapting, and thus scheduling retrospectives is important.
So if you are a team who is struggling with your retrospectives and want to modify them, ask yourself if you could put a place on your wall for post-its, and then hold meetings as needed. If so, then tweak away. If not, reach out and find someone who can help you facilitate your retrospectives. And begin to build the insights you need to move away from schedules and rigor towards a culture of learning, insight and reflection.
Posted on September 10th, 2009
Ah, the Sprint Backlog. While not quite as visible as the Daily Scrum, the Sprint Backlog makes up the core trifecta of Scrum. And what developer wouldn’t love it? Every two weeks (or four weeks, or whatever iteration length you have) we’ll get together with the product owner. They’ll tell us what is most important, we’ll tell them what we can get done, and we’ll add that to a list. During the iteration that list is all we know as a development team, so no more of this, “I have to have you work on this other thing now!” business.
To a team working in utter chaos, it sounds like the perfect club to wield against a management unit that can’t make up their mind. “Sorry, we’re doing Scrum, and it says you have to leave us alone over the next two weeks.” What they don’t realize is that something that is a weapon can be used by both sides – and when the team doesn’t finish all of the stories they signed up for, management will use that same club against them. But if we look closer at why we have the Sprint Backlog, and why we have to have the Planning Meetings, we can understand better when we can tweak them to fit our needs.
In traditional software project management, the question is, “Can you all get this stuff done in this amount of time?” Because developers are a) optimistic and b) really bad at guessing, the standard answer will be “Absolutely!”. And because management and/or the customers don’t know what they want there will inevitably be a point when everyone realizes they aren’t going to make it, leading to what is known as a Death March.
But the goal with Scrum is instead to say, “Here is how much work we can get done in a timeboxed period. Let’s deliver the most value we can in that time.” So the sprint planning games become the place where the stories are reestimated, reprioritized, and signed up for to create the Sprint Backlog. And because we know the approximate velocity of the team, the release planning becomes a matter of looking at the story estimates, looking at the velocity, and then adding up stories until the time we want to release.
(In theory, of course. One of the better practices I saw with this was to have all of the stories estimated for the release, then add two cards that were each 10% of the total value – one for changes, and one for refactoring or other technical debt. That team generally hit within a week of their estimated date on 6-month projects)
Consistency is the secret sauce
Inherent in developing the Sprint Backlog is that the team can consistently deliver the same velocity iteration after iteration. Using a technique known as Yesterday’s Weather, teams can do a pretty good job of signing up and delivering the stories during the planning meeting.
The interesting thing is that the onus of this process is the consistency of the development teams. While that may make good fodder for mathematical calculations and modeling of release planning, it isn’t always the best for the developers. If we’ve signed up for two stories – one a “4″ and one a “2″, then all of a sudden we find ourselves having to do some planning to make sure that we break these down enough to get them finished. In other words, the stories don’t naturally flow through the team.
But what if they did? What if the consistency wasn’t in the teams, but in the stories themselves?
“Look honey, a real, live Business Analyst!”
If we begin to look at the flow of items through the system known as our development process, we see something interesting. In this chart, we see that even though the story sizes don’t appear to have too much variability, the number of actual hours to complete each one varies wildly. What we really want to do is setup the backlog in a way that smoothes these bumps to reduce the variability in the tasks.
To do that, you begin by throwing away estimates. Instead, your card is your estimate. So if it is written on an index card, it should be the same approximate size as the other index cards. Let’s say a story on an index card only goes on there if the team knows they can finish it in two days. If the product backlog has items that are bigger, then you have a whole group known as business analysts who can evaluate, research and find out just how to break that into smaller items.
When this variability is reduced, then the question is no longer, “Here’s how much we can do in two weeks. Fill us up”. It simply becomes a matter of, “Ok, we’re done! What’s the next prioritized story?” As that begins to happen, the need for a “sprint backlog” begins to go away – the sprint backlog is simply, “What’s next?”
Sounds great, Bob! What else do our contestants get?
There’s an important point here. To get to this, you have to be at a maturity level to allow for several things:
- Recognition of flow through the system – You’ve taken the time to do value stream maps to figure out where the bottlenecks are.
- Invested customers – You have customers and business folk who are willing to work with you to get that level of detail
- Strong definition of done, done – because the consistency comes out of small stories, there is no room for “mini-waterfall” iterations. The team has to be very good at delivering running, tested features
Without these things, then the Sprint Planning Meetings and Sprint Backlog serve a very important goal – to stabilize and checkpoint the development process. So as you begin modifying the planning game and the backlog ask yourself – are we modifying this because we’ve analyzed the flow of our system and found that we need a different optimization stream? Or is it simply because your team can’t even meet two or four week commitments?
Posted on September 7th, 2009
<< Timeboxed Iterations | Series Home | Daily Scrum >>
One of the realities of many corporate environments is that you work with a group of people called a “team” but a variety of factors prevent you from actually sitting together. At Alistair Cockburn’s keynote presentation (PPT) at Agile 2009, he showed an interesting slide about the costs of a team not being colocated.
Colocation (and it’s more effective twin, pair programming) aren’t actually Scrum practices per se. Pair Programming is an Extreme Programming practice, and Jeff Sutherland has written several articles on working as a distributed team.
But colocation is a vital practice, so how can you know when it is OK not to be colocated and not doing pair programming?
I miss you already
Since most teams end up forced into non-colocated situations, let’s talk about the organizational maturity necessary to work effectively as a distributed team. When a team is not physically in the same office, there are several important things you lose:
- Facial and Body Cues
- HIgh impact communication
- Incidental conversation
- Low impact help
- Instant reflection
Let’s start with a non-colocated team. Let’s say we have part of the development team in Florida, part in South America, part in Pittsburgh, and a product owner in New York. The team discovers that the story they are working on has a disconnect between what is being done and what they think should be done. The team lead in Florida sends an email to the BA in Pittsburgh. She schedules an urgent meeting for an hour later with the whole team and sets up a conference bridge and a LiveMeeting. The PO is already on another call and can’t join. The South American team has some minor glitches trying to connect, but after about 10 minutes everyone has dialed in.
The BA starts with the issue – the very issue the lead in South America had been trying to bring up. He rolls his eyes to his team down there, who chuckles with the phone on mute. They finally discover that the way the Florida and South America teams have been coding this one feature are using different approaches after the Florida dev lead shares a Visio diagram he put together. It is decided that they will move forward with the South American way, and that the lead down there will work with on of the devs in Florida.
After the meeting, the dev lead in South America schedules a meeting with the dev in Florida. After some technical glitches, they finally share their screen. In describing the issue, they have to keep modifying and sharing back and forth a Visio document. Finally the Florida developer understands, and is able to share that with the team.
For some reason, we consider that a good way of working, and don’t realize how much waste is truly involved in that process. Let’s look at that same scenario with the team colocated.
No, connect this square to this circle
We find ourselves with the same team, with the same issue, except all members sit together in the same office. During their stand-up, they discover a disconnect in how one pair is completing the story from how a second pair is working on it. The BA rolls a whiteboard over, and they sketch out the disconnect. One of the team members rolls his eyes, and the BA catches it and asks him about it. The team member explains this has been a common issue, but ignored.
The team quickly discovers where the disconnect is, and the two pairs swap so they can show each other what they’ve been doing and get back to the common way. The BA and team lead bring the common issue up to the entire team during the Scrum of Scrums, and it’s quickly communicated and resolved.
But the rest of us live in the real world
Unfortunately, team colocation can sometimes be out of the control of the team members. Offshoring, outsourcing, remote employees and other aspects can lead to less-than-ideal situations. So when can a team be OK with not sitting together?
- Never – This might be a strong word, but the team should never be content that they aren’t able to colocate. With that out of the way…
- Open Communication – We need to be past many of the 5 Dysfunctions to know that when we have an issue, we can bring it up
- Excellent Tool Support – Conference Bridges, webcams, screen sharing, Skype, remote pairing tools and electronic whiteboards (or digital cameras for regular whiteboards) are all vital tools for trying to keep communication open.
- Team Agreements – Perhaps the best rule that I’ve seen in place on dislocated teams is to always use the webcam first, then phone, then email as a last resort. I’ve also seen teams keep open webcam links between two sites, with voice going to help increase the incidental communication factor
- One Team / Team Identity – Perhaps the worst situation is when you have two teams acting as a “team” but who have different values, different metrics, and different goals. Offshore / Outsourced personnel should have all of the logins, access and notification that someone working right in the office does. In other words – you should act as much as possible as one team
So while colocation may not be a stated principle of Scrum, it is vital for hyper-productive teams. While there are strategies you can take to be highly productive even with a dislocated team (see Jeff’s paper for ideals of how) in general, you have to have a mature, functional, high-communication team – and you may as well just colocate them.
Posted on September 7th, 2009
Perhaps the very first and most widely “adapted” practice in the Scrum world is that of the timeboxed iteration. Henrik’s Scrum Checklist identifies four subareas of concern:
- Iteration Length 4 weeks or less
- Always end on time
- Team not disrupted or controlled by outsiders
- Team usually delivers what they committed to
The original guidance in Scrum was 30 day sprints, but current versions of the Scrum Guide now recommend teams start out at two week sprints. However, even this isn’t enough for some teams – while some struggle to achieve working software in 6 or 8 weeks, others release working software every day. So how does a team know when to modify timeboxed iterations?
Here lie dragons
It is perhaps easier to start with the antipattern of modifying timeboxes. Unfortunately, these are pretty easy to spot – the team fails to meet their commitment during the scheduled timebox. This causes spillover into the planning game and to the next iteration with the team signing up for even more than they did the first iteration.
This usually continues for several iterations until either the iterations are “extended” (“This iteration is now 6 weeks long”), made longer (“We can’t do 4 week iterations, so let’s make them 8 weeks long”), or dropped all together. At this point the team is now without any guidance at all as to when things are going to be done. Long term release planning now becomes a complete guess as the team’s velocity is wholly unknown, and typically the entire premise on which agile was introduced becomes unravelled.
Perhaps surprisingly, timeboxed iterations are not to blame here. Indeed, it is the lack of self-reflection from the team – often based on immaturity as a working unit – which led them to spiral into a pit of disrepair. Let’s look at the team making two different decisions to alter timeboxed iterations which result in a much happier outcome.
If it doesn’t fit, it’s too big
Let’s say the team encounters the end of their first four-week sprint and finds that they did not make their commitment. And they over commit the second sprint signing up for more than they had in their first sprint plus the additional overwork to be done. By the third sprint, they seem hopelessly destine to be another agile failure. However, the team members, recognizing a bad thing happening, quickly gather the team. They decided on the tactic that if you can’t finish stories during a sprint – your sprint is too long. They decide to see what happens when they drop to 2-week sprints.
They discover that they need to break their stories down more, become more targeted in how they test, how they develop, how they release. Suddenly, what they couldn’t do in 6 weeks, they are regularly making happen in 2 weeks.
Or perhaps it doesn’t belong at all
Let’s look back at our team again. They are hopelessly stuck on their third sprint. However, instead of shortening the iterations, they decide to find out what is causing them not to meet their commitments. They discover that while the work flows through development just fine, it gets hung up during testing.
To solve this, the team decides to establish a Kanban board to visualize the workflow state of the work in progress. Additionally, because of the amount of items caught in testing, they decide to put a Work-in-Progress limit of 4 on the test column. That means that if there are 4 items in testing, no others can be moved to testing until the items are cleared. So the developers can’t move things along, encouraging them to jump in and help clear the backlog.
After several iterations of this, with the team applying the theory of constraints to find impediments, they discover that the “timebox” concept for flowing work doesn’t fit with their model. They have become so adept at it that each story is now approximately the same size, so estimation becomes a simple act of card counting.
Which team are you?
You’ll notice that the key outcome here is dependent on two factors. First – the maturity of the team and their ability to self-reflect to look for solutions (and try new ones). Second – the context of the work coming in. For our last team, if the business couldn’t feed stories into the pipeline that were about the same size, then other tactics might have to be put into place to allow for estimation and sizing.
So when you are considering dropping timeboxed iterations, ask yourself – are we dropping them because they aren’t of value? Or because we are afraid to face the real issues causing us from delivering value to our customers?