Lean from the Trenches

Lean from the Trenches is a case-study of using Kanban for building a large software project for the Swedish Police. It started with a development team of ten people and was later scaled to more than 60 people.

The book consists of two parts: the case study and a discussion of some techniques. I found the first part very interesting as it provides a glimpse into how Kanban can be used in a real large project. With about 100 pages and a lot of images it was rather short and I wish the author discussed some topics more in-depth. The second part, about 40 pages, felt like spacing material, especially the "Agile and Lean in a Nutshell" chapter.

My notes

How We Work

About the Project

Each release does carry a fixed cost. But the release is the moment of truth – the only time that we really learn about how our product fits the user's needs! The longer we wait between releases, the more bugs and incorrect assumptions we will embed in the code. Also, with smaller and more frequent releases, the pain and risk for each release is reduced.

One of the most important characteristics of a Lean process is that it keeps evolving. Sometimes we find better solutions. Sometimes a seemingly great solution yesterday causes a new problem today. Sometimes our environment and circumstances change, forcing us to adapt.

The key to minimizing risk in large projects is to find a way to "slice the elephant", that is, find a way to release the system in small increments instead of saving up for a big-bang release at the end. Ideally, each increment should independently add value to the users and knowledge to the teams.

Structuring the Teams

One of the key challenges in software projects is how to organize people into decently sized teams and then how to coordinate between multiple teams.

In the past, the teams were organized by specialty. [...] Teams tended to communicate with other teams through documents rather than talking, and they tended to blame problems on each other. Teams also tended to focus on getting their part of the work done instead of the whole product. [...] The level of collaboration improved dramatically as we evolved to a more Scrum-like structure, with cross-functional teams of analysts, testers, and developers sitting together. We didn't go "all the way", though; we kept some analysts and testers outside of the feature teams so they could focus on the "big picture" instead of individual features.

Attending the Daily Cocktail Party

Each meeting is timeboxed to fifteen minutes, each meeting has a core set of participants who show every day, and each meeting is public, so anybody can visit any meeting if they want to learn what is going on or have something to contribute.

The Project Board

This seems to be a general pattern: I see many Kanban teams that gradually discover (or sometimes rediscover) the value of many of the Scrum practices. In fact, sometimes Kanban teams start doing Kanban because they didn't like Scrum and then later discover that Scrum was actually pretty good and their problems had been exposed by Scrum, not caused by it. Their real problem was that they had been doing Scrum too much "by the book" instead of inspecting and adapting it to their context.

The project board is probably the single most important communication artifact in the project. It provides a high-level picture of what is going on in the project and illustrates flow and bottlenecks in real time.

Scaling the Kanban Boards

The speed of a project is largely determined by how well everyone understands what's going on. If everyone knows where we are right now and where we're going, it's much easier for everyone to move in the same direction.

Tracking the High-Level Goal

People are more likely to focus on the high-level goal if they know what it is. I know, that sounds rather obvious. But yet in many organizations I've worked with, managers think that everyone knows the high-level goal, and then it turns out that each person has a different answer when we ask them what that goal is.

If people can agree on a goal that they believe in, this has an immensely positive effect on self-organization and collaboration. Conversely, if people don't understand the goal or don't believe the goal is achievable, they will unconsciously disassociate themselves from the business goal and focus on personal goals such as "have fun coding" or "just get my part of the work done and go home".

Defining Ready and Done

-

Handling Tech Stories

Tech stories are things that need to get done but that are uninteresting to the customer, such as upgrading a database, cleaning out unused code, refactoring a messy design, or catching up on test automation for old features.

Remember to celebrate releases – even when you get good at it and they're not as exciting anymore.

Handling Bugs

The Kanban system helped us see that we needed to do system test continuously (well, regularly at least), instead of saving it until the end. The test team resisted this initially, since system test takes time and it felt inefficient to do it more than once in the release cycle. But that is an illusion. It may seem more effective to do test only at the end, but if we include bug-fixing time in the equation, it is significantly less effective.

Before we moved to Kanban, we had hundreds of issues in the bug tracker. Now we have a hard limit of thirty. If a bug is found, the first question is "Is this a blocker?" Blocker in this case means "The feature won't be releasable with this bug" or "This bug is more important to fix than building additional features". Write it on a pink sticky note and fix it now, like any other impediment. Don't put it in a queue. If the bug is not a blocker, however, we have a decision to make: "Is this bug more important than any of the other top thirty bugs in the bug tracker?" If so, then that other bug is removed from the top thirty list to make room for this one. If not, then we ignore the new bug. That way, the bug tracker continuously keeps us focused on the most important bugs and doesn't become an administrative burden.

If you do cause-effect analysis on your bugs, you'll find that bugs aren't really a problem: they are a symptom. Bugs in your product are a symptom of bugs in your process. If you focus on fixing your process, you'll dramatically reduce the number of new bugs in your product.

Continuously Improving the Process

If everyone knows where we are going and where we are right now and if we have the right communication forums in place, then chances are people will self-organize to move in the right direction and continuously figure out ways of getting there faster.

Noticing and celebrating improvement is important to fuel further improvement.

[...] Kanban doesn't provide many specific rules. It is up to you to decide things like how many boards to use on a project. That's the beauty (and pain) of Kanban – it's flexible, and you figure things out as you go along.

Managing Work in Progress

In any kind of workflow it is useful to distinguish work states and wait states. When the mail carrier is carrying an envelope to your mailbox, that envelope is in a work state (because something is happening to it). When it is in your mailbox, the envelope is in a wait state (because nothing is happening to it). The reason we need mailboxes is because you aren't always there to receive an envelope when the mail carrier arrives. Your mailbox is a buffer – a place for things to hang around in while waiting for the next step in the process.

The purpose of WIP limits is to avoid too much multitasking and overloading a downstream process. If the testers have too much work to do, we don't want developers to keep building new features and adding to their workload – instead, they should focus on helping test. WIP limits act as an alert signal to highlight the problem before it gets out of hand.

[...] the essence of WIP limits: focus on finishing things rather than starting things!

Capturing and Using Process Metrics

In theory, theory and practice are the same. In practice, they are not.

Planning the Sprint and Release

-

How We Do Version Control

Find out how long it takes to change one single line of code and get it into production. That may well be the most important metric in the project!

Why We Use Only Physical Kanban Boards

Most of those electronic tools can automagically generate all kinds of detailed statistics, be backed up, be accessed from outside the building, have different views, and so on. Why don't we use something like that instead? One of the main reasons is evolution. Our board has changed structure many times. It took a couple of months before it started stabilizing.

The second reason we use a physical board: collaboration. [...] If we had an electronic Kanban board, we could use a projector to display it on the wall. But we would lose most of the interaction, the part where people pick up a card from the wall and wave it while talking about it, or where people write stuff on the cards or move them around during the meeting. People would most likely update the board while sitting at their desk – which is more convenient but less collaborative.

What We Learned

Get everyone together and agree on what "perfect" means in your context. What would a "perfect" process, organization, and work environment look like? [...] Perfection is a direction, not a place! Having a clearly defined direction makes it easier to focus and evaluate your improvement efforts.

Don't look for perfect solutions. It's probably not worth the wait, and you'll probably get it wrong anyway. Instead, look for small incremental improvements, and think of them as experiments. An experiment may or may not lead to the intended improvement, but it should always generate insights that can be used to design the next experiment.

A great process isn't designed; it is evolved. So, the important thing isn't your process; the important thing is your process for improving your process.

Instead of asking "Why did we fail? Who screwed up?" ask "What did we learn, and what will we try next?"

The only real failure is the failure to learn from failure.

Most people like change; they just don't like to be changed. So, don't make any change without first involving the people who will be affected by it.

People are much more likely to accept a change if it was their own idea!

A Closer Look at the Techniques

Agile and Lean in a Nutshell

Broadly speaking, Lean and Agile are two sets of highly compatible values and principles that outline how to succeed with product development. Scrum, XP, and Kanban are three concrete ways of putting these principles into practice.

Reducing the Test Automation Backlog

-

Sizing the Backlog with Planning Poker

-

Cause-Effect Diagrams

The key to effective problem solving is first to make sure you understand the problem you are trying to solve – why it needs to be solved, how you'll know when you've solved it, and what the root cause of it is.

Symptoms often show up in one place, while the cause of the problem is somewhere else. If you "solve" the symptom without digging deeper, it's highly likely that the problem will just reappear later in a different shape.