Software Development - More Than Coding

Think about software development. I'm assuming you have an image of someone writing code in your mind. Well software development is that, also. But it's also much, much more.

In my company I'm responsible for the software development in this more broader sense. When I think about software development, everything begins from the customers. Developing software is solving customers' problems. Selecting what problems to tackle as a company is a strategic choice. Strategy narrows down what you develop and helps the company focus. Probably it also states in which markets you want to compete in.

Strategy helps the company define it's portfolio. What customer problems we want to solve and what kind of solutions we offer? The solutions can be products or services. Strategy should guide you in this. If it doesn't, refine the strategy.

Let's now assume that in your offering/portfolio you have product(s). Each product should have a vision. What the product will be in the future? What customer problems will it solve when it will be in it's ultimate, final form. (This form will never see the daylight. Vision serves as a distant goal, something to strive for. It's an idealisation that will be even more magnificent when you get closer.)

Vision will help you define a roadmap. Where as vision can be hazy, the roadmap should be really concrete and tangible. I think of roadmap as a guidebook for the next steps toward the vision. If your strategy is planned for the next 5-10 years, roadmap could cover maybe 3 years. It's worth noting that it's still a high level plan and as for any agile plan, subject to changes.

In software development roadmap is implemented in releases of new software versions. They are usually developed in iterative and incremental fashion. Old functionality is modified and improved with new features and bugs and defects are fixed. Many times companies try to shorten their release cycle. The ultimate case would be Continuous Deployment where every change would be deployed to customers. But this is very much industry specific. In some cases the overhead of taking new software version into use is so big that customers don't want to do that often. Then it's practical to select a release cycle that suits both the company and the customers.

In agile development, software is usually created by small teams in an iterative fashion. The length of these iterations or sprints is usually limited to less than a month, a little bit depending on the selected methodology. Sprints are like mini-projects; they are planned, executed and in the end there are sessions for examining the results and for learning from the experience. ...and then cycle starts over again.

Daily work in sprints is carried out by the development team. Developers are experts in their field and select the most fitting methods for the implementation. They have the authority for carrying out daily decisions.

We got to the part where the coder writes source code. But as there are many layers in onion, there are many layers in software development. Writing the code is just one of them. Never the less, it's an important craft if the software is to be of any use and even more so if it is to be maintained.

From Forecasting to Cycle Time

I like measuring for the sake of improvement. I believe that measuring something helps to understand its nature. And by measuring you might learn if something is good or bad and going to positive or negative direction.

We have been using so called forecast hit-rate to measure how well the Scrum Teams are able to predict their velocity. It's the result of dividing planned number of Story Points with the actually realized number of Story Points for a Sprint. But in case the number of stories is small, there's easily large deviations when things don't perfectly as planned. And usually things don't go perfectly as planned.

Also this measure can result in some rather nasty dysfunctions. Teams start to favor keeping the forecast instead of trying to maximize the amount of customer value in a Sprint. Or even though some Story becomes obsolete it isn't dropped because that would flaw the statistics. This isn't nice.

We use Atlassian JIRA and JIRA Agile for our Backlogs and Scrum boards. (I guess I should start calling it JIRA Software soon.) It offers a handy Velocity Chart where one can follow these planned vs. completed results. This is essentially the source of our forecast hit-rate information.

But due to all the possible events that affect the Scrum Teams and the #NoEstimates movement that gains momentum also in our company, the data has a really high variance. Even after couple of years it shows no signs of stabilizing. (Of course it can argued that is a systemic dysfunction. As it is. But let's not go there.)

Another tempting alternative for this metric that I've been thinking about is Cycle Time. Average time that a Backlog item takes between the start of work until it is completed. Let's assume that such metric would be easily available and that there would be target on it or that management would follow the number. What kind of behavior would this drive?

My assumption is that it would make the Stories smaller. That teams would strive to get things done faster instead of having mammoth size stories. So In the end there would be a bigger number of smaller Stories and tasks that could be made in shorter time. I would like that. Also it would fit nicely with #NoEstimates.

Since this is still simply hypothetical, I don't have any results yet. But this is an experiment that I'm about to try.

Planning Releases

I believe I was just getting the hang of this Scrum thing and I thought I could do some decent Scrum Mastering. But now the things have changed slightly and I need to jump "over the fence". For a bit over a month I've been working as a Product Owner and a Release Train Engineer. I can see that it's actually a lot easier to identify missing Product Owner as an impediment than to actually be the Product Owner and try to live up to the requirements of the role. There just doesn't seem to be enough hours in the week to constantly keep the Backlog in order and think about all the details. But maybe that's exactly the point. No-one is able to know everything in the complex world of software development and that's why letting go and giving the team the space they need comes rather naturally.

But then something about planning Releases. In Scrum everything happens around the Scrum team. Team forecasts what they can achieve during a Sprint and that's it. Achieved velocity, "yesterday's weather", helps to forecast what can be expected in a longer run if the backlog has been refined into a decent degree. But there's usually no mentions about what to do when you have multiple teams, multiple products, a bit of legacy and all in all a more complex situation than just complex.

I think Scaled Agile Framework offers nice way to see this from a higher level. Scaling to the level beyond teams and Sprints, I see Releases as a good logical logical next step. Also it seems like a good idea to set some higher level goals (or Objectives) that can be then divided into Stories when the actual work starts. Of course there will be some uncertainty associated with these goals, but I think they should be treated more as commitments than just some initial guesses that will be reformulated right after the first Sprint. Still, I don't support big upfront planning or waterfall. I just think that a bigger enterprise can't live without ability to plan ahead for longer than just two weeks.

Anyway, the actual planning of the Release is a complex task on its own. As the Release Train Engineer I wanted to arrange a so called Pre-Planning where all the teams share their initial plans and everyone gets some indication about what is to be expected. Then there will be time to work the plans out until everyone (or at least most of people) are happy with them. Then we can have can simply agree that this is the plan that we try to implement. And if it needs to be amended along the way, so be it. But the train will take off and only the future will show what's going to happen. ;)

Scaling Scrum Beyond Teams

Our teams do Scrum. The lowest level of inspect-adapt-loop is the Daily Scrum. Above that are the Sprints that usually take one to four weeks. Most common is probably two weeks. This level is planned solely by the Scrum Team.

Our Sprints belong to Releases. We are currently satisfied with a pace of four releases per year. That means that a release comes every three months. With two week Sprints it means that we can fit six Sprint per one Release.

I know that in the most pure form of Scrum, teams produce each Sprint potentially releasable content. And there are probably no bugs. Unfortunately my version of reality contains a legacy product that has some known bugs. And since we have a long history and some dependencies, we really need to do some integration testing at the end of the Release cycle.

So we carry out an extensive Release testing at the end of each Release cycle. Currently we have reserved the whole third month of each Release for this, but the teams are free to use their best judgement on this. If they have everything tested in time, they can start new development already before the Release period is over.

The idea of four releases per year is actually taken from Scaled Agile Framework. We do not want to re-invent the wheel, so we rather use practices that someone else has found useful.

Release planning is carried out between Product Managers and Product Owners. Of course in the end the team needs to "accept" the plan, so we iterate until a mutual agreement is found.

Beyond the Release level are the Product Managers Roadmap and product Vision. And even above this is the Strategy. This whole setup is rather new, so I'll tell more once we have experimented a bit further.