Technology and process innovation

April 27, 2008
In an agile/lean software development team, discussion is invited, but coordinated.  I’ve noticed that people who are passionate about technology or process often feel friction if they don’t get a good hearing for their idea.  Equally, the technical/process leadership have to help everyone on a team achieve consistency.  Regardless, there should be no “status quo”…
I’ve been playing around with a pipeline approach to technology/process, where the team can
  • achieve consensus on the current state, and next steps
  • get rid of things that aren’t working
  • propose blue-sky ideas
  • get support for taking almost-working ideas through to completion
So, every iteration, we can build up a map of our technologies and practices, and then select and shine the elements that we think are important.
  • Deprecated – was useful once, but we should remove uses of this practice/tech when it is found
  • Definite – should be (and is being) used unless it is seriously mismatched to the problem at hand
  • Sound – this is a really good, proven idea/tech that isn’t being used yet (or only sporadically), but we think it should be adopted
  • Tentative – this would be really good, but it might have some issues that make it unsuitable
  • Radical – as a concept it solves some problems we’re having, but it may raise more issues than it solves
For instance, we might have the following map (example: for a Java based web site)
  • Deprecated – Struts, JDK 1.4
  • Definite – JDK 1.5, Struts2, Spring Dependency Injection, Maven, Hibernate, Continuous Integration, CVS
  • Sound – Test Driven development, Subversion, Freemarker
  • Tentative – BDD and acceptance criteria executed with Rspec and selenium, continuous pairing
  • Radical – Scala
  • Out – consensus is around not using this tech (e.g. .Net on a java project to name a simplistic example)
This is just a snapshot.  You can see a mix of practices and technologies here.
Each iteration (for some value of each), I would like the team to produce the following:
  • System guardians for each definite technology/practice
  • A safety-factor of 1-5 from each team member for every element on the map
  • A vote/score for each sound/tentative/radical idea on the map – this is the priority associated with adoption
  • Actions that can be taken to move prioritized items towards definite (for instance,  a 15 minute “topic of the day”)
In this way, there is a collaborative understanding of what works, what doesn’t, and what we are doing about it.  
Some thoughts around this:
  • I think it’s OK for a initially unpopular idea to stay up on the map, because it allows the proponent to feel included, and it stays in the collective memory for times when the idea is the right one.  
  • No definite process/technology is immune from deprecation – although part of the criteria for moving from sound to definite is to address issues of how to phase out the old process/tech
  • Items can be split – there are situations where a technology provides some benefits, but only if it is used in a particular way.  So, Spring DI may be definite, but Spring MVC may be out.
  • A lot of discussion around these things will happen in the retrospective anyway, but the innovation map should be persistent and displayed on the team room wall.
  • The “system guardian” pattern is great for identifying subject matter experts, and then making them able to move on to other areas.  Essentially, each system guardian is responsible for finding 2/3 other people and paring with them until they are also system guardians.  Once you have 4/5 names as guardians, the knowledge will propagate quickly.
The DebtStream Guards
In agile delivery, there is often a pattern of reserving a pair or two (depending on team size) for technology and process maintenance – the technical/process “debt” stream.
I’ve used this stream effectively to
  •  Simplify the build
  • “Proof of concept” 3rd party software integration
  • replace crufty code with a big restructuring
  • etc…
This team is rarely made up of the same people.  In fact the idea is often to seed a pair who will then roll into the main streams of development, spreading their experience, and backfill them.
Advertisements

What colour is your code?

February 14, 2007

Thanks to Jeremy Ruston for this – it started as a drunken conversation about 4 years ago, developed through a couple of years of casual use in a professional environment, and now I use it for a variety of reasons:

* Explaining why product companies are bad for business users with specific requirements
* Understanding emotional commitment to technology within IT departments
* Structuring code-base dependencies

Consider an [arbitrary] piece of code. It’s indoors, it’s well behaved, and you can be proud of it. What is it about that code that you like? Is it elegant? Is it clever? Does it enable something? How would you measure the value of code?

Well, one of my thought experiments runs like this: I’m going to give the code a colour. The colours I can choose from go like this:

  • Yellow – only a specific business customer needs this code
  • Pink – only a particular market (e.g. group of customers) would use this code
  • Black & White – this bit of code is valuable to a whole bunch of markets
  • Blue – this code is a piece of infrastructure that I cannot change.

What’s that? Not clear enough? Let’s try an example….

Yellow:

  • a configuration file in which I store the email address of the system administrator
  • a custom rule that defines specific discounts for a favoured partner
  • a complex sql query that generates a report for that one customer who has 70 call centre staff
  • that natty bit of boolean logic that decodes the snazzy protocol for controlling the new swiss-made robot that just arrived in your warehouse
  • A customer logo

Pink:

  • A model of products and pricing schemes that supports pricing by weight
  • An integration with a supplier’s order management system
  • Decoding rules for retail bar-codes
  • The URL for the equifax credit scoring service

Black&White:

  • Huffman encoding algorithms
  • A virtual machine for testing electronic circuits
  • Default velocity configuration file

Blue:

  • The J2EE specification
  • That mainframe messaging format
  • Oracle JDBC drivers

The model of code costs (pre-test driven) works out as thus: Some code valuable, and some is worthless:

Yellow code has no value – in fact it only really has a cost. As a product company, I want anything a specific customer requires to be cheap to create.

Pink code has some value – if I can open up multiple customers, offer them something common, I might start making a profit.

Black & White code – I have something that is truly unique, gives my company an edge, and that might enable me to sell my company – or to buy another company. I’m thinking mergers and acquisitions.

Blue code – It’s something I’m lumbered with – it has no real value in itself, except as an underpinning to other things. I might generate a large amount of value from using J2EE EJBs, or I might not. But I probably have to use JDBC to enable any value…

Note that colour coding has nothing to do with complexity. It is all about return on investment.

Cost on the other had has everything to do with complexity. If my team of 50 propeller-heads spend 10 years developing something, I damn well expect it to be Black & White. If they come out with something that only suits one of my customers, well – I’m going to have to give up my plan of early retirement in the Seychelles.

Conversely, something that is easy to create has no real value to my M&A agenda: good for you you’ve written a routine that will count the number of characters in a string! Come back when you deliver ME some value.

As a product company:

At the intersection of high cost and high value, a rational product company will create as much black & white code as possible. In a quid pro quo, a rational company will make adaptability of their application extremely “cheap” (in other words: configuration and scripts!) and hand it over to the customer, or start a services company.

The flawed model here is that, from the customer’s point of view, a vendor’s application is mostly Blue. And the skills required to customise even a simple product are high. It is unlikely that a vendor’s application will provide a competitive advantage (unless they are the only purchaser). From the product company’s point of view, they have no interest in making customer specific alterations. A trap, for the unwary.

Emotional attachment

A similar pattern happens in IT departments of medium sized organisations. The intellectual property of the IT department – the crown jewels, if you will – is usually some fiendishly complicated system that is difficult to change. Black & White. Since it encapsulates all that the business has done so far (for who can predict the future?), there is an immense amount of pride in the system. Rightly so.

From the business side of an organisation, what has been done, is just a cost. They aren’t going to sell their company on the back of some wonderful system for expressing business logic. They just want to express new business logic. Their “Black & White” is new business rules and new models of business. Brands and Market penetration techniques. The best Black & White of all is a large, growing, customer base. What is Black & White to IT becomes Blue to a business. Yellow and pink (business) code is often infeasibly expensive.

UIs are often the primary mechanism by which customers judge a business. When they become locked down by the IT department – making new channels difficult to create – there is tension.

As an aside, it is for this reason that I believe that multiple-user interfaces are required on many IT projects. If usability analysis and interface design is a scarce resouce, channel creation will be deemed Yellow by IT, and therefore under-invested.

Dependency analysis

When refactoring code, it is almost always a process from Yellow through Pink to Black & White. Agile developers don’t short-cut this route. They also try and keep things simple, leaving the costs of change quite low.

However, this mutation leads to the following property: Yellow code depends upon Pink code. Pink code depends upon Black & White code. Ask yourself – where does the “utils” package go in a project. It is Yellow, Pink or Black & White? If it’s more than one, split it up.

An example? If you have a UI which is designed solely for admin users, it is Yellow. The complex services that this admin UI uses will probably also be yellow, but there may also be some pink and black & white elements (for instance, that generic user manager is black & white, but each user’s vehicle registration number is probably pink). The (branded) UI that customers use is pink, but the logo itself is yellow.

If you see a yellow piece of code used in a pink piece of code, flag it up. Use colours as part of your vocabulary.