Geek Philosophy

All of this Kanban reading I’ve been doing has been great, and as I’ve mentioned before I’ve started implementing some of the techniques and metric tracking. However, after meeting with my manager (himself an experienced Agile thinker), I’ve realized that in some ways I’m trying to solve problems I don’t have, while neglecting some of my bigger issues.

Limited WIP is of course one of the key techniques / philosophies of Kanban (and lean in general). It’s not important in and of itself, but instead because out of it comes increased collaboration, reduced cycle time, identification of bottlenecks, and all of that other great stuff. We haven’t really reaped any benefit from this (yet) besides limiting the backlog, which has fostered cross-business-unit collaboration, but it’s only been 3 weeks or so and we were a fairly disciplined team before that. Some of the metrics I’ve been tracking (such as how long bugs have been in the system before we ship them, and how long items are blocked for) will definitely come in handy to measure the team’s effectiveness and customer responsiveness.

That said, we’ve never really had much of a problem with Work in Progress, at least at the task level. After talking with my manager, we realized that we had another problem, one that was discussed by Johanna Rothman at the recent Agile Vancouver conference – far too many items in our project portfolio! We have 8 people on the team, and had limited our active WIP to 12 (to allow for some blockages), but it turns out that we were working on 7 different, mostly independent projects.  Most of these projects were rather small, and the diffusion of resources was mostly due to the fact that there was no obvious parallelization for most of the projects, but still – that’s almost 1 independent project per person!

Due to the nature of my team (maintaining 3 products, and integrating these products once monthly with software from elsewhere in the company), it’s unlikely that we’ll ever get down to a project WIP limit of one. After we clean up the current mess, we’re going to try 1+1 – a maximum of one project on the go at any given time, with the exception of these small monthly integrations. With the slack time, we can clean up some of our (ample) technical debt. Part of the metrics I’ve been tracking is the percentage of our work spent on “failure load”, a combination of technical debt and missed requirements.

This will likely start in earnest in the new year, but I’m already excited about the results. I’m actually surprised that I let it get this bad – we’d been effectively doing a 1+1 project WIP limit for most of the past year and a half – but I think by formalizing the process a bit more (not making it heavier though!) we can keep ourselves to good habits.

(This post is a continuation. Please read part 1 and part 2 first.)

A year or so had passed since I first started implementing the tracking system I’ve been discussing. However, in August I started work on a larger project that had to go through the company’s formal project tracking system. The project was took about 3 months (we’re just finishing it off), and we have done no releases since – that’s an eternity for a team like mine that’s used to shipping production code very two weeks. With the project coming to a close, I was looking forward to moving back to a process that encouraged more continuous flow.

I had first thought about Kanban after hearing a talk at the Agile Vancouver mini-conference, Lean Development for Lean Times. I’d also recently finished reading David Anderson’s excellent book Agile Management for Software Engineering, which emphasized the importance of short cycle times and delivering value to the customer quickly. After corresponding with David about this book, I mentioned Kanban to him and was delighted to find that he was working on book on the subject. I was able to obtain a draft of this book to review, and the ideas presented within further cemented my ambition to try Kanban with the team.

Another thing worth noting is that my team had grown, from 5 engineers under me to 8. We’d also taken on maintenance duties for yet another product, bringing the total up to 4. This increase in responsibility caused me to spend more time thinking about our development process, since more was at stake.

Limiting Work In Progress

I’ve mentioned before that the only real difference between what I was doing before and Kanban is the presence of formal Work In Progress (WIP limits). What makes WIP limits so important? As it turns out, this minor change can have major effect on the effectiveness and responsiveness of the team.

Little’s Law, a finding of queuing theory, shows that the cycle time of an item in a system is directly proportional to the number of items in the system. Thus, to reduce cycle time, it is necessary to reduce WIP. This is intuitively obvious – I can finish reading a book faster if I read it all the way through than if I read some, then read the paper, then read another book, then come back to the first book – but by limiting WIP both stakeholders and developers are made explicitly aware of the implications of trying to do too much at once.

The benefits of limiting the size of the backlog have already been discussed in part, but by having a hard cap on the size of the backlog, stakeholders can clearly see that by choosing a certain work item other work items are not being addressed. This leads to increased collaboration between different business units, and forces everyone to “see the whole” (to use lean terminology) – individuals no longer argue for local optimizations (i.e. pushing their agenda through to the detriment of others), but instead come to an agreement about what would be best for the business as a whole.

Limiting the size of the backlog also helps stabilize lead time. Expediting items (“drop what you’re doing and work on this!”) is not conducive to flow, since it almost always involves context switching. Being able to provide stakeholders with the average cycle time of items is of great benefit – it can help make scheduling decisions, decide on relative priorities, and give visibility to other departments.

Limiting WIP in the “active” states (analysis, design, testing, customer acceptance) can help identify and deal with bottlenecks. For instance, if test is a bottleneck, which will be visible when test starts to run up against its WIP limit, we can use techniques from the Five Focusing Steps to alleviate the problem, such as ensuring that test is never idle by having a buffer of work always ready for them. Since the rate of flow through your system is limited by the bottleneck, improving the performance of the bottleneck will directly lead to increased throughput and reduced average cycle time.

I don’t yet have any numbers or metrics for my team showing improvement – I only implemented Kanban a week ago. As soon as I have a large enough set of data draw some conclusions from, I will post my findings.

Any questions so far? I’m understanding all of this, but I haven’t had a chance to explain it to anyone, and I always find that explaining / teaching really helps me solidify my knowledge.

… because this is only 1/6th of the way into my moustache:

From Movember

Remember, it’s easy to donate and help fund prostate cancer research!

(This post is a continuation. Please read part 1 first.)

In my last post I briefly discussed the “kanban-lite” system that I had been using over the past year or so. I adopted this system for several reasons:

• As a team working on multiple projects, there were often multiple stakeholders with different goals who wanted some of my team’s time

Each of these stakeholders had valid reasons for their selections of issues that we should tackle. By getting together and having a weekly meeting to see what was really pressing, I could ensure that I was working towards what was best for the company, not just trying to placate whoever was yelling the loudest. In lean terms, this is known as “seeing the whole”.

• My team was known as the “Fast Track” team, since our primary mandate was to respond rapidly to customer reported defects and small features that could close deals

Project teams at Sophos often undertake projects that are anywhere between 3 and 24 months in length. My releases are more on the order of two weeks. Since I’m releasing software so frequently, I wanted a system that encouraged flow and ensured that we were still delivering high-quality releases. Having a visible board where each item’s state was clearly displayed made it very obvious when we should finish and polish in-progress stories instead of starting new items.

• I’d been reading quite a bit about Lean Software Development, which encourages flow and the elimination of waste (another term for non-value added activities)

A key technique of Lean is value stream mapping – identifying where and when value is added to your product, and pointing out areas that can be removed since they don’t add value. The visual card board is an easy way to show at least part of the value stream (the value stream really begins when a customer files a defect or a product manager thinks of a new feature), and I had thought about (but not gotten around to) recording the amount of time issues spend in the various states.

So, you may ask, why would I switch to Kanban if my system was already working? I’ll discuss that in the next post.

The month of November is known for a lot of things – snowfall, leaves falling, and presidential elections. Around my office, it’s known as Movember – a time when the guys can get together, grow horrible moustaches, and raise a bunch of money and awareness for men’s health issues, particularly prostate cancer.

I have never attempted to become a Mo Bro and grow a moustache – until now! Below is my day 1 photo, clean shaven (well, clean shaven as of last night). Expect it to get much, much worse.

From Movember

So, how can you help with men’s health issues? By donating to my moustache campaign, of course! All of the funds go to Prostate Cancer Canada, and qualify for tax deductions – you can’t afford not to donate! I will be updating photos ever few days throughout the campaign so you can view my horrible, horrible progress.

Thanks in advance!

Last week at work I transitioned my team to a Kanban system (warning, pdf). I’ve been thinking of doing this for a while, and in fact a lot of the process that I had already put in place was very similar to Kanban, but a couple of recent developments made the time seem right to take the plunge and fully implement the system.

First of all, what is Kanban? Kanban is a term that comes from lean manufacturing that means “visual card”.  The basic idea of Kanban is to have a system that facilitates flow and pull-based scheduling by limited Work In Progress (WIP).  In software engineering, this manifests as a visual process control system such as a whiteboard / cork board with story cards on it, or an electronic version of such a board. This board shows what state these cards are in, such as “backlog”, “in development”, “in test”, “customer acceptance”, and “waiting to ship”. So far, this sounds like pretty basic fare for an agile team.

The real key with Kanban is the limited WIP. All states of the board should have a WIP limit. For active states such as development or test, this intuitively makes sense, as it’s generally agreed that multitasking is not optimal. For a state such as “waiting to ship”, limiting the work in progress ensures that code is always shipped to customers in a timely manner. For a state such as “backlog”, limiting WIP (as opposed to considering the backlog to be the entire collection of defects, suggestions, and new features for the product) makes it possible to stabilize average cycle time (the average time it takes for an item to move from “backlog” to “complete”) and greatly eases prioritization.

As I said before, this is very similar to a system that I had set up for the team over the past year or so.  I would informally limit work in progress (by gathering a list of relatively high priority items in the backlog, encouraging team members to finish off in-progress stories before starting new ones, and shipping frequently). I encouraged a “pull” system instead of a “command and control” style approach, where team members would have a clear idea of relative prioritization and thus be able to choose the next task to work on when they’ve finished their own. I had a card wall that showed the various states that items were in.

What lead me to create a system that was similar to Kanban before I had heard about Kanban? I’ll talk about that in my next post in the series.

This post is a slightly edited version of an email I sent today in response to the question “Do you know any publications on performance loss due to switching from project to project?” I thought I gave a good answer, and I’ve wanted to write more about software engineering recently, so here it is.

It really depends on what is meant by “performance loss” and what the nature of the “switching” is.

The worst case scenario (while still remaining plausible) would be switching back and forth between 2 (or more!) projects in some sort of phased SDLC – that is, doing all of the analysis for project 1, then project 2, followed by all of the design for project 1, then 2, then coding & 2 and testing 1 & 2 and so on until you’ve shipped both projects. This may seem absurd to some people, and would definitely be a major process failure for an independent team, but in some cases this might be necessary; say, if specialist resources like designers and testers are a constraint, or if there are external dependencies. The obvious problem here is the greatly increased lead time for both of the projects. If the projects are of roughly the same size, the lead time has at the very least doubled (and that’s discounting any cost at all for context switching). This is a “Bad Thing” (there should be ample references in the lean literature about the benefit of decreased lead time), even if the scenario is not as blatantly bad as the one I just laid out.

For “performance loss”, it really depends on what’s being measured. If you’re thinking about lead time (which is directly related to throughput¹, and thus revenue – in other words, you should always be thinking about lead time), switching (or expediting, a different manifestation of switching) will negatively affect performance. Tom Demarco argues in Slack that the real penalties for task / project switching are often hidden because of how people measure efficiency (number of hours worked / busy people as opposed to value delivered to customers, i.e. throughput).

As for actual loss due to context switching, which was possibly the original intent of the question, I think this may have been partially covered by Brooks in The Mythical Man Month, and according to a blog post by Jeff Atwood of Coding Horror this topic is discussed in Gerald Weinberg’s Quality Software Managemnt: Systems Thinking. I haven’t read this book, but the title is evocative of Peter Senge’s The Fifth Discipline: The Art and Practice of Learning Organization (which helped pioneer systems thinking). I haven’t had a chance to read that either (it’s on my to-do list!), but it’s pretty well regarded from what I can tell.

So, does anyone out there have some other references to exactly how switching from project to project can cause trouble? I’m looking for papers, books, studies and the like, not just restating of lean or agile principles, so you can’t just say “context switching is bad because is slows people down” – show me the study!

¹Agile Management for Software Engineering, David J. Anderson [2003]

I’ve actually had a chance to play some games in the last little while (being sick has its benefits!), so I thought I’d go over some recent highlights.

Guitar Hero: Metallica

The formula is pretty simple – a Guitar Hero game featuring songs from Metallica and other hard / metal-rock bands. Guitar Hero (and its various incarnations) have been discussed to death, but I will say that this game was an awesome workout for the hands. Trying to play along with these songs, even on a fake plastic guitar, has given me a new appreciation for Metallica, and before I had beaten the game (which I managed to do on expert guitar!), I had taken out my old electric guitar that’s been gathering dust since high school. Some new strings and a guitar -> USB connector later, I’m playing Metallica (very poorly) in real life – now that’s an inspiring game!

Lost Odyssey

I’ve seen mixed reviews on this traditionally-styled JRPG, but I really quite enjoyed it. The art is beautiful, probably one of the prettiest games I’ve seen, and the characters, while being a bit cliche, develop nicely. The battle system has quite a bit of depth, combining elemental matching with real time events, and at almost every point in the game you would encounter challenging enemies that could be defeated with the right choices. Probably the most unique thing about this game was the various dream segments, where the lead character remembers bits of his forgotten past through short stories presented as text. Although the presentation of these dreams are simple, the music, text styling, and content of the stories were compelling – a mix of Zen koans, depressing tragedies, and funny anecdotes. The plot is long and engaging (40+ hours of gameplay, and there were plenty of unfinished sidequests), and the ended was pretty satisfying. A great game, especially for the story-driven RPG lovers out there.

Fable

This game was pretty hyped when it came out several years ago for the original Xbox, so I was excited to finally get around to playing it. Unfortunately, while it had some fun parts, it didn’t live up to its high expectations. The music was decent, the world was well-imagined, and the graphics were okay for their time, but the much-touted alignment / cause-and-effect system left a lot to be desired. There were some neat aspects to it – your character’s appearance changed based on how strong, weak, good or evil you were, and people reacted to you differently as well – but a lot of the interactions seemed superfluous. So, my character can drink beer, get drink, then throw up – what does that gain me? Why would I bother to do that except to see the animation? Apparently eating too much food made your character fat, but my character ate a lot of food throughout the adventure (food resulted in healing) and I never gained a pound – too much running I guess. The entire marriage / dating / sex subsystem was equally lacking – sure, it was there, but there was no motivation at all for me to take part in it. The plot was also a bit thin, a “previously unknown super bad guy is now the last boss” type of thing, but the game ended up being good enough that I am looking forward to trying out Fable II for the Xbox 360.

As the winter comes I’ll have time to play a few more games (currently looking at Jade Empire, Gears of War, Tales of Vesperia, and a few others), so I may continue to write about them. At the very least it’s getting me writing

… at least it is in Vancouver. I can’t really complain, since only a couple of months ago people were complaining how we weren’t really getting summer weather, so instead of talking about the Vancouver heat I’ll talk about a few other areas of my life that are heating up (yes, a lame segue, but really it’s too hot to think of anything more clever).

Yesterday, the team I manage at work grew by 60%, going from 5 members to 8 (not counting myself). We are now wholly responsible for 4 complete products, 1 internal and 3 external, and with any luck we’ll be the most efficient team in the company in terms or revenue per person. That said, efficiency isn’t what I’m striving for – while the team growth and priority shifts are causing me a bit more work in the short term, I think we’ll be able to maintain a comfortable pace that allows for slack.

Last night I went to Vij’s for Jen’s birthday, and the food was as fantastic as the last time that I was there. We didn’t have any cricket bread this time, but the dishes we did have were perfectly spiced for the weather – we were all sweating a little bit and our mouths were tingling, but nothing was too hot. If anyone out there is a fan of Indian food and hasn’t yet been to Vij’s, hit up Jen’s review for a strong argument for you to go as soon as possible.

I’ve played relatively few team sports in my life, at least by choice (I’ve tried to forget about junior high gym class). I did 6 years of martial arts throughout high school, which I found very enjoyable, but I never was one to pass a ball, shoot a basket, or score a touchdown. That’s changed this summer, as I’ve been playing Ultimate with a team consisting mostly of workmates. While I’m relatively fit (not fat, not athletic, was biking to work a few times a week before moving), my lack of disc-throwing experience combined with an almost complete absence of team play made the first few outings quite difficult. Over the summer, however, I feel like I’ve improved to a level that I’ve never before reached in a team sport. That’s not to say that I consider myself “good” at Ultimate, nor am I one of the better members on the team – I just feel that I’ve never had this level of team strategy combined with skill in the sport. I’m seeing plays materialize, I’m going to where the disc is going to be, and I’m actually getting some of the more interesting throws to go where I want them (hello, outside-in). It’s pretty fun, and I’ll very likely play another season next summer, but before that – playoffs! We don’t exactly feel a lot of pressure, but there’s still going to being a lot of excitement on the field, and given that we have a game this week, we’ll definitely be feeling the heat.

There, that’s three things that are (very) tangentially related to heat – increased workload, spicy food, and the pressure of playoffs. Hopefully if I keep thinking of other heat-related things, I can forget the fact that my daily commute to work is more like a pressure cooker than a bus ride. Stay cool out there!

Wow – pretty close to a month since I’ve posted last. It’s certainly not like I’m lacking things to write about. Here’s a brief recap of the past few weeks.

At home, I’ve moved, explored the new neighbourhood, planted some herbs on my new patio space, BBQ’d on the first BBQ I’ve ever owned (thanks, Jen!), and nearly ruined my wrists playing Guitar Hero: Metallica. The cat is having a lovely time running up and down the stairs (her first, I believe), it’s wonderful to have in-suite laundry again, and it’s just awesome getting out on the patio, even if it’s just to varnish a shelf. I have yet to name the bird that sings to me every morning on my walk to the bus loop, but that may get done this week.

The commute to work hasn’t been too bad – 45 minutes there, an hour back – and I’ve been playing some awesome emulated games on the way. At work, the team’s been doing really well, garnering a lot of internal and external attention with our efforts. I’ve moved to a kanban system for pull-based scheduling, which I’m hoping will streamline our work flow. This will be especially important early next month, as my team is set to grow by 66%.

Let’s see… I’m quite enjoying playing ultimate, Michael Jackson’s still dead, and I’m tired from a long day of around-the-house busywork. Hopefully I’ll get back to writing regularly sometime soon.