Developing Metrics for Development

Last month I wrote about my intention to try Cycle Time as a metric for software development. Back then it was still hypothetical, now it's real. Maybe it's still a bit early to draw any final conclusions, but it's already evident that I failed miserably in communicating the reasoning behind the change. And the criticism is well deserved.

I selected the average Cycle Time during past month as the metric for each team. In a way it is a good metric and gives indication about how quickly the value was realized after the work on a backlog item was started. But all backlog items aren't equal and also work type and intensity differ during the release cycle. In the beginning of the release cycle teams mostly work on the new development. Nearing the end the period emphasis is shifted more towards testing and bug fixing. Issue life cycle tends to be much shorter with bugs than when creating new functionality.

It is also questionable if one monthly average number can help in decision making, in essence serve as an actionable metric. You can see the trends in longer time intervals, not much more than that. Another angle is that a team can be really mature and produce outstanding results even though they don't split their Stories into very small parts. Their average deviation can be small and thus predictability high.

Then again I feel that the metric got a little unfair criticism when it was compared with Lead Time. In the end the company is interested in how the customers experience our products and services (Customer Experience = CX). But this includes many topics that are out of scope for software development. Like the company brand(s), how salesmen conduct their business and how the customers are served by the customer service. I think the software development part of the system (company) can greatly affect the User Experience (UX).

 

I would love to get the big picture correct. So personally I feel that the CX is more important than UX alone and Lead Time more important than Cycle Time. But in both cases the internal part is relevant component of the whole and deserves a metric of it's own. (It would be so cool if I could somehow bundle measuring UX and Cycle Time somehow... Probably there's no such easy connection. I think relationship between Lead Time and Customer Experience is much easier to show.)

Now the current bleeding edge is to try using the whole information provided by the JIRA Control Chart instead of simply one number. I haven't yet figured any cons, but on the pro side you get trend lines of where the performance is developing and standard deviations. Of course you can still identify the release cycle from this data, but you can draw other conclusions too. Increases in rolling average or standard deviation can signal trouble. And when the figures are followed regularly, the team can be aided in a timely fashion.

 

Currently I feel like I'm making too rushed decisions. I'd like to involve other people more and ask for their input already before I make an experiment. Now I mostly just measure the impact and make amendments, but I'd like to concentrate on this more. Let's see if the Santa Claus and New Year will bring a difference. In case this will be my last blog post this year, I already wish you all Merry Christmas and Happy New Year!