Geek Philosophy

For the past month or so I’ve been taking part in two free computer science courses offered online by Stanford – Machine Learning and Introduction to Databases. They’re both great, presenting material at about a second or third year undergraduate level (though not a full semester’s worth, and often a bit light on the math), with full video lectures and graded assignments. I would have also loved to do the class on Artificial Intelligence, but at the time the video lectures were not download-able, and I watch the the vast majority of the lectures during my commute. It’s too late to sign up for these courses, so I won’t spend too much time writing about them here – I plan to do a full write up of at least the Machine Learning course after it’s complete.

What may be of interest to my readers is that, starting in January, Stanford is offering several other courses with the same online format. Here are the ones that I’ve seen so far:

• Software Engineering for SaaS
• Human-Computer Interface
• Natural Language Processing
• Game Theory
• Probabilistic Graphic Models
• The Lean Launchpad

I certainly won’t have the free time to take all of them (my commute is not that long), but I’ve signed up for all of them to at least have a look at the material. If they’re structured like the courses I’m currently taking, completing the “advanced” track of any of these (which means doing quizzes, exams, and programming assignments) gets you a certificate of completion from the instructor. Hopefully Stanford will start working with Mozilla, specifically their Open Badges initiative, and continue to push the boundaries of online education and credentialing.

Are you taking any of the current courses? Do any of next term’s look interesting to you? Leave a comment!

I’ve recently spent a lot of time thinking about how to grow professionally as a software development manager. Since I was a programmer before becoming a manager, I first thought about how to transfer some of what I learned about professional development as a coder to my role as a manager.

Programmers have it easy

A lot of options

Let’s think about how a programmer progresses. Of course, there’s experience and time, the greatest of teachers. Even if you have a programmer solve a similar class of problems on a regular basis, if she’s keen she’ll come up with new and better ways of doing things. Add a piece of automation here, a new editor trick there, perhaps a refactor or a domain-specific language – there’s always some opportunity to learn something new.

Luckily, most programmers aren’t stuck solving the same problems over and over – they’re doing different things. This could mean a new programming language (or new techniques with the one you already use), a new platform, a new module or third-party API, or a new paradigm, such as incorporating more of a functional design style into otherwise object-oriented code. Programmers can also change their focus, for instance from user experience to scalability to embedded systems.

Additionally, professional programmers most often don’t have to do it alone. If you run into a database problem you don’t know how to fix, you can go look over the shoulder of the local SQL guru as they optimize your query. If you’re fortunate enough to be on a team where pair programming is the norm, you’ll be in almost constant learning mode. At a standup meeting, at an architecture design meeting, or at a security review, there’s a group of your peers (or superiors) solving technical problems.

All that said, the title of this section is a bit flippant – to really take advantage of these learning opportunities, a developer would ideally have the will to learn, the drive for self-improvement, and the hacker ethos of experimenting with programmable systems. It’s not that this learning comes for free, it’s just that it can be part of the job (or some on-the-side project, or an open-source effort, etc.).

What exactly would you say you do around here?

The pleasure's all on this side of the table, trust me.

So now, back to the land of management. What does a software development manager do, and how would one go about improving those skills?

Roughly speaking, you can break down what I do into four areas:

• People management. This includes conflict resolution, career development for my direct reports, and managing up.
• Project management. This involves task prioritization, risk assessment, project tracking, and scope management.
• Process / team management. This is broad, but in part consists of promoting technical and team best practices, and focusing on how the team does their work.
• Technical guidance. Although I’m not the most senior technical person on the team, I have substantial product experience and a higher-level context that proves useful during in-depth technical discussions.

Of course there is overlap. What is a team if not a group of people? Choosing continuous deployment could be process or project management, and encouraging pair programming is part team management and part technical guidance. The list goes on and on – I’ll stick with the above demarcations tfor now.

How do you grow?

There are several paths towards professional self-improvement, but I’ll lump them into three broad categories: Experience, Training, and Self-Study.

Experience in this case refers to professional experience, and covers most of what I talked about above with respect to programmer development. This is the knowledge that comes from trying new things, learning from mistakes (both yours and others’), and paying attention to what works in various situations, all as part of your day to day work life. There is always some sort of experiential learning to be had, should you choose to take advantage of it.

Training is another way to learn, and can be helpful in circumstances where there’s a specific skill set to be learned that is too broad to wait for experience to teach, too critical to be learned through trial an error, or too time-sensitive to be learned through self-study. School and certification courses are obvious forms of training, but I also count mentoring (external or within your company) and conferences under this heading.

Self-study is a broad term. and I use it here to refer to everything from reading blog posts and books, to podcasts and forums or user groups, to side projects and open source contributions. Self-study is the most accessible form of learning for most people in this profession, since it is plentiful and often free, but it is often the least taken advantage of.

Naturally, like the overlap in the management responsibilities above, the distinctions between these categories are not hard and fast. Would MIT Open Course Ware count as training, or self-study? Work experience often has aspects of self-study, even though it may not self-directed. A conference may be training, but what about an unconference?

So what are you doing about it?

Aspiring to greatness

Now that I’ve laid out the loose definitions for the types of learning I might undertake, as well as the areas in which this learning could be applied, I’ll get into what and how I’m trying to learn.

People management is an area where I think there is a limitless amount of experiential learning to be had, and it’s one I’ve been focusing on more recently.  Even if I was managing the same people for 10 years, they would change, I would change, and the project would change, so there would be new challenges and opportunities. That said, I have been pursuing self-study in this area (mainly through blogs, books and podcasts), and the upcoming Much Ado About Agile conference has several talks and tutorials on the subject of people.

Project management is something I do as part of my job, but I’m not really a project manager. I have experience working within my company’s project framework, but there’s a lot of inertia to overcome to change anything since it’s a framework the entire company uses. This is an area I’m not putting a lot of effort into improving on my own time, since it’s not a primary interest and there’s relatively little I can do about it in my current position, but I’m always on the lookout for things that I can improve during my day to day tasks.

Process and team management is where I’ve spent a the most time learning, through experience, training, and self-study. I’ve followed the Kanban community, posting questions and comments, and even helped review a great book on the subject. I’ve read books by Mary Poppendieck, Tom DeMarco, Esther Derby, and more, and my to-read list just keeps growing. I’ve attended and spoken at Agile meetups and conferences. I’ve tried to be an Agile coach for my team (which can sometimes be difficult). Although this is where I’ve focused most of my efforts, I’m starting to back off a bit here in favour of People- and Technical-centric learning, primarily to round out my management style.

Finally, learning about technical management. Although over the years I’ve done some self-study on this, reading books on design patterns, lean and agile development practices, and architecture, I haven’t been as rigorous about this as I could have been. Part of it has been that I haven’t been coding at all (maybe a few lines every few weeks), a situation which I’ve started to correct. Picking up new technologies, even if I’m not using them at work or for any serious projects, helps broaden my perspectives when faced with the challenges I do need to solve at work, providing me with a bigger technical toolkit. I’m also getting more involved in security reviews at work, and have recently had some work-sponsored training by a security expert, and I hope to get some further training on the technical track of the upcoming conference.  While I don’t know if I’ll ever go back to primarily being a developer, I think staying sharp technically is an important part of being a development manager.

Your turn

Thanks for making it this far! Do you agree or disagree with my assessment of the learning opportunities for programmers or managers? Have anything to add? Let me know in the comments.

I recently attended an Agile Vancouver talk by Bill Wood of Ping Identity on the subject of “Metrics Driven Agile Development”. I’ve been interested in metrics for some time as a result of following the work of David Anderson, so I had looked forward to hearing from an experienced manager who actively uses metrics on an agile team.

Bill opened the talk with a few points about Ping’s philosophy towards metrics. He admitted that many metrics are not scientific measures and can be prone to inaccuracy, but they can still be useful to help ground a company that’s working in a rapidly changing environment. The use of metrics is simple – gather some metrics in as consistent a way as possible, monitor those metrics, and question change. In this case, change means for the better or for the worse. In this way, metrics can provide a good way to “Check” when implementing a Plan Do Check Act cycle of learning. Some of Ping’s other philosophies and practices reinforce the value of metrics, such as the idea of “Going Honest Early” – bringing up potentially problematic issues immediately – and the use of operation reviews, company-wide meetings in which the health of the engineering organization and its projects is discussed. These reviews, which I first read about in Agile Management for Software Engineering, provide valuable feedback to non-engineering facilities and are a practice that I’d love to see taken up in my office.

Next up was an Introduction of the classic “iron triangle” of project management, Quality, Time and Features. At Ping they fix time, committing to three month “trains”, or releases. If something is not ready in time due to quality issues, incomplete development, or any other reason, it’s not in – the rest of the completed features go ahead and ship. Lead time is established to be 3 months, with high confidence; in the last 5 years, there have only been 12 days in train delays. With time fixed, there is a tension between quality and features, with quality always winning. This is not to say that they gold plate – there is simply a zero defect release criteria. The concept of the “Iron Triangle” was elaborated into an “Iron Pentagon”, the additional two factors being resources and system.

Bill brought up several metrics that are used at Ping to observe many aspects of their development system. There were graphs of “utilization”, or how many team members were available to work on a project and how many were absent due to vacation, holidays, illness, etc. There was a slew of metrics related to quality, from the count of development-injected bugs to the average amount of time between finding and fixing a bug and the amount of time the test team spends on set up. Like all metrics, these require the gathering of data over time, and one of the metrics mentioned was a measure that I did have the data for, albeit not centralized – the number of various types of automated tests run by one of our products. A few hours of hacking data together, and a few minutes learning some jQuery and flot, and I get the following chart:

From here it’s easy to tell that 2008 was an awful year for unit tests, we greatly prefer unit tests to system tests, and that both have been growing over time. There are several more metrics that were mentioned that I’d like to take a look at, particularly the ones related to defects. I’ll post any graphs I make, and would love to hear stories from others who have sucessfully used interesting metrics to drive change in their development organization.

As an aside, I asked Bill after the talk about techniques used to loosely couple features such that they may be removed near project end without destabilizing others. He said that a variety of techniques were used, ranging from a preventative / planning approach – that is, simply not working of features at the same time that are likely to impact each other, making backout easy – to incorporating functionality “switches” in the product, which can be configured either at compile or run time to use either new or old code. These are techniques that I would love to start applying on my team, if there is ever a need to have strict adherence to a release schedule.

This past year was probably a turning point for me in terms of getting involved with a larger community on cutting edge issues, specifically the Kanban community. Although my team achieved great successes in the past year, shipping 37 different software releases and maintaining high quality standards, I can’t say that I’ve really seen too many benefits from Kanban thus far. Don’t get me wrong – I still think the method has a large number of merits. It’s really more that I’ve been barely doing Kanban.

The year is a continuum, of course, and I can’t say that I didn’t do anything with respect to Kanban all year. My knowledge about the team and the method improved throughout the year, culminating in a Kanban “relaunch” in the new year. That’s been going well, and my current approach is much more in line with what the method prescribes, but that’s a subject for a different post. Today I’ll reflect on what did and didn’t work in the past year, and why.

First, I’ll cover the highlights. Even the very limp version of Kanban we were doing afforded us some structure to build on and learn from. With a visualized workflow and explicit work in progress limits, the team engaged in conversations around how much parallelization was too much and what the appropriate size of work items was. The beginnings of explicit policies emerged, blocked items were noted and prioritized, and the team successfully delivered on their commitments.

Now for what didn’t work. Explicit policies were few and far between, which led to the rehashing of several topics as well as too much unnecessary management intervention (tie breaking, direction, clarification, etc.). Metrics about lead time, cycle time, and work in progress were not kept – I always thought there would be a better time to start, such as the beginning of the next project, and worried about missing important metrics or recording useless ones. The lack of classes of service also provided some discouragement for data-gathering, as two different tasks in progress could be two orders of magnitude apart in difficulty. Without such data, our Kanban system was being used primarily for task management instead of as a process of evolutionary change. Finally, with a dedicated, embedded customer / product owner on the team, and with an existing culture of openness, there was no cadence at all. Releases, retros, and queue replenishment were all ad hoc, which prevented any rhythm from forming.

Although our work system is still, and will always be, a work in progress, all of the above difficulties have been addressed. I now have a month’s worth of data, spread across 5 classes of service. We have a weekly cadence for queue replenishment and retrospectives; we’re in the middle of a large project, so we no release cadence right now.  Policies are explicit and are up for review during the retrospectives, or during the week if the policy is wrong or dangerous.

My next post will outline how the team’s system works, some problems we’re still trying to address, and what we’ve learned so far.

One of the roles that I see myself growing into as time passes is that of change agent. I don’t mind speaking up when I see problems, I’m dedicated to seeing problems solved even if it won’t happen immediately, and I don’t get discouraged if there is resistance or setbacks when implementing change. I’ve been lucky enough to be leading a team where I can experiment with new ideas and have a lot of control over how my team works.

However, when I look at professionals in my field who are interested in team and organizational improvement, they are almost always consultants. Does it have to be this way? I can see the benefit – moving from company to company, the rate of feedback is much faster, and you can learn what works and what doesn’t work much sooner than you could working within the same system. Also, there’s always this old quote from Deming:

“As a good rule, profound knowledge comes from the outside, and by invitation.  A system can not understand itself.”

Is it that companies refuse to accept major change initiated from within the organization? I’ve heard this repeated many times (though I’ve never seen any studies), so that could account for the lack of “famous” internal change agents. It could also be that people who are genuinely good at turning companies around demand too high a price to be a permanent employee. Alternatively, maybe there’s an inherent bias of contractors and consultants to be more vocal about their methods, previous clients, and successes, as it helps them gain new business.

All that said, I like working in a single company and have always thought that I’d hate being a consultant. That distaste is changing a bit – I think it’s mostly the gathering of clients / insecurity that I’d dislike, as opposed to the nature of the work – but I’m still not very interested in jumping ship and becoming an entrepreneur.  Of course, all of this may be due to inexperience. I’ll be the first to admit I that have relatively little experience in the industry.

For now, I’m still in a good place. I run a small team in a large company, and there seems to be some opportunity for change that I may be able to help out with, so I’m definitely not out of growth options in this area. I try to keep as up to date on modern methods as I can by reading books and forums, talking to other professionals, and airing my ideas publicly so they can either be torn down or validated. Still, it’s interesting to think of the future, and wonder if I’ll ever have the desire (not to mention capability!) to go solo.

More than any other conference I’ve been to, this year’s Much Ado About Agile conference has left me with a pile of books that I want to read. I’ve actually put off reading a few career-related books recently because I’ve felt that the information isn’t sticking, but I’m going to try out the SQR3 method (as mentioned in Declan Whelan’s presentation) to see if I can up the retention level.

Here’s the list of books that I wrote down as “must-haves” – if you were a presenter at the conference and don’t see your book here, it’s probably because I didn’t attend your session!

• Certain to Win: The Strategy of John Boyd, Applied to Business (Chet Richards)
• Teamwork Is an Individual Skill: Getting Your Work Done When Sharing Responsibility (Christopher Avery)
• Behind Closed Doors: Secrets of Great Management (Esther Derby, Johanna Rothman)
• Agile Retrospectives: Making Good Teams Great (Esther Derby, Diana Larsen, Ken Schwaber)
• Fearless Change: Patterns for Introducing New Ideas (Mary Lynn Mans, Linda Rising)
• The Fifth Discipline:The Art and Practice of the Learning Organization (Peter Senge)

The Fifth Discipline wasn’t written by a presenter, but was mentioned by several. It’s been on my “to-read” list for a while, but being called out as a wonderful reference on both systems thinking and organizational learning means that I’m going to actually suck it up and get a copy.

The first talk I attended in the afternoon of the second day of the Agile Vancouver conference was on a subject that I’m becoming increasingly interested in – team / organizational learning. Presented by Declan Whelan, “Learning is Key to Agile Success“, this was one of the talks that had me sold just based on the description. Peter Senge, social psychology, and the Theory of Constraints? I’m in.

Declan began the talk by stressing the need to create a safe environment for learning within a team or organization. Here, “safe” means “no (or few) negative consequences for certain amounts of failure”. An organization that does not tolerate failure will never push its boundaries and thus never grow and learn grow, as it will always be operating within its safe zone. He talked of the need to create a “safe to fail” environment (contrast with a “fail safe” environment), encouraging people to try new things and not fear personal repercussions from trying to go above and beyond.

Beginner’s mind was a topic in this talk, as it was in many others at the conference (I love it when Buddhist philosophy, another great interest of mine, overlaps with software!), and again Arlo Belshee’s concept of promiscuous pairing was discussed, this time in the context of learning as opposed to lining up with one’s natural cycles of alertness. There was also some discussion of the Satir change model, which came up when discussing the ways that groups (from families to teams to organizations) react when change happens. Peter Senge and Josh Seddon were invoked in the systems thinking context – you can’t change people, you need to try to change the system that they work within, then the change comes for free.

The Theory of Constrains was also mentioned, with the idea that learning is in fact the biggest bottleneck in information work. To illustrate, I’ll borrow an example from Christopher Avery. Think of the last project your team finished. Imagine the source code was completely lost, and business decided that yes, you need to redo that same project over again. No change in scope, just do the work over again. Most projects would see a dramatic reduction in effort the second time around, which makes sense – after all, you have all of your design decisions made, you’re familiar with the code, and you won’t make the same mistakes twice. You’ve learned, and this learning substantially speeds up the work. This is one of the reasons the Empire State Building was constructed at such an incredible rate (as discussed in Leading Lean Software) – the people involved had solved this problem before and learned.

Declan finished out the talk with several concrete examples of how to encourage learning, at a personal, team, and organizational level. Brown bags, open spaces, retrospectives, scheduled slack, practice fields, information radiators – there are many ways to increase the learning of your agile team. I’ve been toying with the idea of scheduled slack (for instance, having a task card on the board that amounts to ‘go learn something’), and I think it could be especially useful when combined with the idea of “practice fields” (areas of code where mistakes can be made with almost no consequence – personal projects, proof of concepts, technology demos, etc.).

The slides are online if you want to have a look, but like all good presenters there’s only so much you can get out of them .

It’s nearing the end of day one of the yearly Agile Vancouver conference, Much Ado About Agile. I’ve enjoyed the talks I’ve seen so far, and I’ve been taking extensive notes, but I thought I would try to quickly summarize some of the talks.

The first talk I saw was the keynote, delivered by Chet Richards (author of Certain to Win). The talk was titled “The Fundamental Secret of the Universe” – that secret being that if do the unexpected, and react faster than your opponent / enemy / competitor, you are certain to win. The talk focused on the findings of John Boyd, the OODA process (Observe, Orient, Decide, Act), and how in a variety of situations across human history the ability to react to happenings quickly has overcome superior technology, numbers, and head starts. I’d heard of OODA before, but outside of describing the acronym I’d never seen it described. The talk was quite good, and everyone really seemed to enjoy it. The mindset approach involved – get a few core concepts, and learn them very deeply – is something I would like to work on more, as I often worry (for lack of a better word) that my understanding of fundamental Agile and Lean principles is somehow “superficial”.

The second talk I attended at the conference was presented by Esther Derby and entitled “From Big Boss to Steward Leader – Becoming an Agile Manager”. This talk focused on how managers need to shift their thinking on a few fundamentals, such as decision boundaries and intervention strategies, in order to adapt to the way that teams work under the agile mindset. This was thought-provoking session, and I especially liked the idea of increasing “information overlap” (that is, making sure that developers know how the business makes money, and that higher-ups know the way the work is done). I was fortunate enough to snag a half-hour private meeting with Esther afterwards, where I got to ask some questions specific to my situation, and about topics that were complimentary to her talk. I’d been meaning to pick up her most recent book, Behind Closed Doors, and now I definitely will.

Tomorrow’s post – Linda Rising’s excellent talk on Caffeine, and possible other tidbits from Agile Vancouver!