Jogging Meeting

I have an impression that walking meetings are currently all the hype. Because everything should be always tested before judging, I decided to give it a spin. But why settle for a walk if you generally move faster?

So we decided to go for a jogging meeting with one of my team mates. I know he's an active runner and I too occasionally enjoy going for a run. And we have one-on-one meetings biweekly anyway.

When clock hit 2 PM we went to staff changing room and took the elevator down. Fortunately my team mate had been smarter than me and thought of a good route before hand. I had just thought about it, but didn't ever have time to do that. As one can guess, route length is an essential factor for keeping the meeting in schedule.

Weather was cold but sunny, practically no wind. A really nice running weather. Close to the office there was still some traffic, so it was not so easy to have a conversation. But after the first couple hundred meters we reached a calmer grounds and the conversation was easy.

Another factor to keep in mind is pace. Don't go too fast or you will be just huffing and puffing. With a decent pace you can talk, but still get some exercise.

During the jogging we discussed some memorable events from the past couple of weeks and how's the cooperation with other team members and rest of the organization. But since these one-one-one's are my substitute for development discussion, I wanted to also discuss future. For most this is a difficult topic (it is the same for me too), but I think it's also really important. I demand Product Managers to share their Roadmaps, because it means that we have some plans for the future and we won't be just hitting a wall after the current Release is finished. That's why I'd like people to also have plans for their future. Many times these plans can be beneficial from company's point of view also.

Skipping to the end: we reached the office safely, went to shower and got back to our computers. As an experience I think jogging meeting is definitely worth trying. Of course there are certain prerequisites, but if everything is matches and you feel like it, go for it! I'm pretty sure we'll try this again.

Facing the Inevitable

Layoffs, bankruptcies and people losing their jobs seems to be almost like a trend nowadays. For the previous generations it was possible to graduate, get a job and finally retire with a gold watch after working in the same company for all of your career. If someone now has something similar in mind it almost feels naive.

Of course there are always exceptions. And probably there's big variation between different companies. But I think everyone needs to admit that things happen faster than before. Fortunes are made and lost in mere days, maybe even faster. Robots handle transactions at the stock market. One of the big dilemmas seems to be that there's a lot more data available than before, but it seems increasingly difficult to distill the important messages from the background noise. Do we have time and wisdom to understand what we measure?

I have adopted some rules of thumb from what I've read. I don't anymore think that people will spend all their lives in the same company. For my own company and team I try to create best possible circumstances for them to work. Create a system that they can feel connected with and achieve something. Exchange their free time to a hopefully competitive salary (I don't have much power over this) and to tasks that are intellectually challenging, foster their creativity and when ever possible, can be identified as things that move the company towards some greater goal.

Another thing to consider is situation when company needs to let some people go. It always feels bad. But sometimes it is inevitable. I think about Nokia as an example. If you manufacture normal, non-smart phones and people don't buy them anymore, the situation is tough. There's no easy way to increase the sales. Maybe you can find a new market where people still would prefer these 'old school' phones, but that doesn't change the fact that globally the demand for them has gone down. And will not go up anymore in the foreseeable future. And if your organization consist mainly of experts in this field, management doesn't have a multitude of options. You can wait until your bank account is empty, but in the end the result will by inevitable and ugly.

In the end the choices are to go out of business or face the facts and adapt. Things that worked in the past do not work anymore and you need to come up with something new. You will need to develop new competencies and be bold enough to let go of the past.

For some this is a cruel message. For some it can be a wake up call. My best advice for everyone could be to keep challenging yourself and to learn new things. If you learned to do something 10 years ago and continue to do the same year after year... one day your services may not be needed anymore.

Stay curious and find out new things. Experiment. Stay agile and adapt to the changing environment. Learn everyday!

Build Length

Continuous Integration is maybe one of my favorite engineering practices. With the green/red build indication you always know that your code compiles. Or doesn't compile. This information is vital.

After compilation the next step is running automated tests. Best practice approach would be to run unit tests. Unfortunately it's not always so easy to write tests for single modules. Then the tests tend to become somewhat integration tests. But since in my case we haven't made very strict rules about these, I'll simply adopt the vocabulary from How Google Tests Software and simply call them Small and Medium tests.

After these shorter tests are run, the following round will include Large tests. These are end-to-end tests that tests the functionality in many realistic use cases or even integration between different products.

Going a bit more into details, the execution times for these tests one year ago was around 10 hours for all Small and Medium tests. They were executed using physical machines and the system running them was TeamCity. Or actually the tests were ran for both x86 and x64 bit programs. number of tests was around 5k for each bitness.

The Large end-to-end tests took also 8-12 hours, but they had not been automated. Also the visibility to these tests was rather poor. They were executed by the system team, but the results weren't transparently available. The best way to know whether the tests were passing or not was to ask from some team member.

The radical change happened when build machines were virtualized. Instead of having a few high clock rate physical machines we ended up in having a set of servers which we filled with Hyper-V virtual machines. Number of processors was set to 8. We also made some modifications to run the tests in parallel (they were previously all just run consecutively). CI system was moved from TeamCity to Atlassian Bamboo.

Build times (or build + running tests) dropped  significantly. One thing we also learned was that virtualization platform makes a big difference. With all the same settings VMware was almost 20% faster than Hyper-V. Good thing is that both can be scripted nicely using PowerShell. We created VMware build machines with 16 logical processors.

During the same effort we managed to get also our Large tests to run in parallel in our continuous integration system. This wasn't as straight forward, because the framework used fixed file names and the test runs interfered each other. But in the end all the problems were resolved and we got the tests running automatically.

After the changes our build + Small & Medium tests now take 30-45 minutes depending a bit on bitness and other factors. Large tests are executed in about 45 minutes. So, for any change done to the codebase we now get the following phases:

1. Changes are checked into continuous integration and compiled
2. Small and Medium tests are executed.
3. Large tests are executed.

In parallel with step 2 we have other builds that produce installer for manual testing. Build time is much shorter, so the test execution still dictates the build length.

Before our latest efforts the steps 1-3 would have taken almost 22 hours including some manual steps. Now they take around 1,5 h for any changes, things are fully automated and the results are transparently available for anyone anytime. Since nothing is ever enough we aim to go even further, but I think the current results are already worth a small celebration!

Tower of Hats

A few days ago I started thinking about all the different things I do at work. I have a main job, but in addition I do bunch of other things too. (This post is not about any Agile practices or management. If you are mostly interested in those posts, please feel free to stop reading now. But if you are interested to read how tall my pile of hats is, please continue.)

My main job is Manager, Software Releases. All the software that goes out goes through me. At least in theory. With 10+ teams it's sometimes a challenge. :)

I'm the Chief Scrum Master, owner of the Scrum framework (and how it's implemented in the company) and somewhat coach of our Scrum Masters. Fortunately I have another person who has recently started to assist me in this.

I'm the owner of the Release Process. How we package the software and use version control. This also means that I arrange release activities like planning, demo & retrospectives for all our products.

I'm the owner of Software Development Process and company wide Definition of Done. I try to come up with common conventions that allow us to have top notch code while still having creative freedom.

I'm the head of Release Team. We keep the infra structure in place and make sure build machines and version control work. Also we maintain other tools like JIRA and Confluence. I also pitch into this work, but I admit my team mates are more capable for doing that. And I'm actually really, really proud of that.

In addition to the supervisory duties I'm the Product Owner of the team. So I also get to prioritize the customer needs and bugs in the Backlog. With so many internal and external customers to serve I think it's a real handful.

I'm the owner of the Software Product Creation value chain. This means all the activities between having a customer need and actually filling those needs. SW Development and Release Processes are part of this value chain, but it also includes Issue & Configuration Management Processes and testing & development practices.

I'm a member of the Technology Unit's management team. We discuss unit's different matters with other Product Owners and country managers. Mostly the bigger decisions concerning the unit are done here. Actually I'm also the secretary of these meetings.

I'm an Internal Auditor. This isn't really a big burden since we don't have internal audits but couple of times a year. But it's interesting and offers a chance to examine other parts of the organization than just R&D.

During last strategy update I worked as the internal strategy facilitator. Mostly this meant making sure all work groups progressed. In a way it was really really educational. I learned tons of new things about the company and opened my eyes more to the business and long term goals.

I'm the substitute for Technology Unit head & also for the Quality Manager. Usually these things don't require any actions from me because the actual people are mostly present.

I'm a project manager for a Tekes funded research project DD-SCALE. Mostly I interact with researchers from Helsinki Univeristy or attend steering group meetings with all participants.

I think I'm still the vice shop steward. Haven't done anything regarding this in ages.

I'm a member of the company's leisure team. We arrange different activities for the employees in Finland.

I'm rather interested in the social media. I actually created the twitter account for the company and have been rooting for more external communication. How can the customers know about the cool new stuff if we don't let them know about them?

There. If I counted correctly, it's about 10+ hats or so. At least my head shouldn't get cold. Sometimes it just gets a bit dizzy for all the different responsibilities. But maybe this explains why I like to simply call myself the Jack of All Trades

Things I do on my spare time are out of scope. And too numerous to fit here. ;)

Loops of Learning

I'm currently reading a book about Organizational Patterns of Agile Software Development. In addition to the interesting patterns I was especially attracted to the chapter about anthropological foundations. The book also opened my eyes to see that processes alone are not enough. I've written about this before, but my view was reinforced once more.

Continuous improvement with processes usually only deal with Single-Loop Learning. We inspect the results of our immediate reactions and then make corrections. Usually this is ok for a team that is building software in iterations. It's about following the rules. But when we go a bit higher, it maybe isn't enough anymore.

The second level, Double-Loop Learning was introduced to me by the Lean Startup book. In that method we still make small modifications as often as possible to learn fast, but we also can choose to either pivot or persevere. Do we want to keep on chasing the selected goal or should we select another target? Already this felt to me like something really awesome. Someone has even described the ideas of Lean Startup as possessing super powers. (I wouldn't maybe go that far, but it's a good book.) On this second level we create more insights about our actions. It can be also characterized as Systems Thinking.

But I wasn't aware, at least consciously, of the next level: Triple-Loop Learning until now. Jurgen Appelo writes about it in his blog post and Thorsten Gragert's wiki site offers a nice a summary including the below figure. The 3rd level deals with principles. In organizational context I'd associate it with company values and learning about learning.

Figure adopted from www.thorsten.org

By digging a bit more I found this article about Learning organizations. Interesting concept. They have the following five main features:

• systems thinking
• personal mastery
• mental models
• shared vision
• team learning

I think I need to give this a lot more thought. And try to lead my organization further on this path. Maybe I will coin a term #BeyondAgile. :)

And finally, as a somewhat sidestep, I'll share a some tips I have spotted from many successful organizations:

1. Dream big
2. Hire the best people (who share your dream)
3. Stay out of their way
4. Share the success

Scales and Emergent Properties

It is fascinating how things look different depending on the distance. From space Earth looks like a blue ball with some green stuff here and there. By zooming closer one can see the continents and mountains and some of the biggest man-made structures like highways. Going closer still and taking a helicopter view the individual houses are yet indistinguishable, but cities have forms.

Humans can't usually see the forest for the trees. Trees are closer to our scale and we usually look at them from the ground up, not from top down. Also we and all the organic material around us consists of small cells. Many of them have a specific purposes (like muscle or brain cells). They are visible with a microscope.

If you could magnify even more, you would see that even the cells aren't the ultimate building blocks. They could be further divided into atoms which could be further divided into quarks (and maybe even further, but let's stop here.)

Many of these levels have their own forces and interactions. Quarks can feel the strong interaction (or nuclear force). In their world, gravity or electromagnetism isn't very interesting. On the other hand, interactions and events on the cosmic scale take so long and distances are so big that compared to them the human lifespan is only a drop in an ocean.

The same fascinating scale related properties exist also in the corporate world. Individual people, teams, units and companies work on different scales. After saying this I do admit that some exceptional individuals might match the output of a whole team or even a small company. But I'd claim that there are such efforts that simply aren't doable without the appropriate size. And yes, on the other hand bigger scale usually makes things more stiff and slower.

I think a team has a lot better chances to really finish a software feature. And I mean the whole deal: thinking about the usability, implementation, documentation, testing, optimizing, refactoring and packaging everything into an appealing form. And if we go a bit further and think about productization of an idea, I think a company is better off than a team or an individual. Sales and marketing are essential activities. But such capabilities aren't often found in the standard developer skill set. For a company to be successful, both skills are needed. But individual people can be highly successful in their roles with only one of these skills.

No-one can do everything. By working together we can make great things happen. With the joint effort of brilliant minds we have been able to reach the space and descend into the deepest ocean depths. If we could work together as a planet who knows where we would end up... But currently I'm happy to follow the road ahead with my company. :)

Release Planning in One Day

In this blog I have shared experiences of our Release Planning Days. Of course the first one was most memorable since back then everything was new and exciting. Second attempt proved that it wasn't pure luck, but that the concept was actually working. I documented also the third one, but after that I thought that it maybe isn't that exciting anymore.

One unfortunate thing to mention is that the day was a public holiday in one of our offices. But quite a few of our colleagues from that site were willing to come to work on that day and get another day free in exchange. I call that going the extra mile.

But maybe I can tell about the most recent attempt, since it was again a bit different from the previous times. For our latest Release Planning I had given some of my colleagues a few chores to fill in advance. Here's the list:

• Portfolio Backlog is in presentable state
• Product Managers have discussed their the Roadmaps with teams working directly with them
• Product Managers have made an initial Launch Plan for their products
• Product Owners have crafted Draft Release Plans for their teams

Out of these, the first task went to our Chief Portfolio Officer who is like the Grand Vizier of the Portfolio level. Second and third tasks went to Product Managers (Program level) and the fourth one to the Product Owners (Team level). 

Launch Plan is a term that isn't originally from SAFe (as far as I know). It communicates the intended usage of the Release if and when it would be successful. Previously we had identified a waste of extra waiting time when the Release was ready, but we had not decided what to do with it. The accompanied business needs are in most cases clear already when the work on the release starts. That's why it is practical to craft also a plan about the usage. And if something is left our the Release scope, the plan can be adjusted.

So, in essence we skipped the first Team Breakout. Also the organization of the presentations was modified. In our past Release Plannings all Product Managers have presented their Roadmaps in a row. But since we had the Draft Release Plans already available at the beginning of the event, I wanted to form logical entities of the Roadmaps and associated Release Plans. This also helped in focusing on one subject at a time.

The morning part of the day was heavy. That was clear already beforehand. But we didn't come up with any better alternatives. At least grouping the presentations in the new way was seen as an improvement.

All in all things went pretty much as expected. During the Team Breakout some plans were modified but most of them were almost the same as during the draft phase. The new, more compact way of conducting the day seemed to work well and we will probably do it like this also next time.