DISCLAIMER: This post appeared originally in SpiderOak’s Engineering Blog (CC BY-NC-ND 4.0).

Whenever the word automation appears in conversation, I always remember this story of a build engineer that took the word “automation” to the next level.

There are times where you want to be really needed at your job; at the very least, it gives you a sensation of stability that has nothing to do with whatever your environment is.

This should be just a stage. Stability should come from the environment, and your goal should be to not be needed in as many things as possible, so that you can focus on new things.

This is an over simplification, but if you see a software development company as a machine or a factory, you want to make the machine as resilient as possible to “problems”. When parts of the machine become inactive, the throughput is affected – that’s unavoidable – but you want to make sure the whole thing doesn’t stop working.

This post is about exactly that: how to make the factory more resilient.

Planning

Planning is the starting point in delivering. Planning properly is a key step in delivering consistently.

At SpiderOak we’ve adopted agile methodologies; we really like its evolving nature. Requirements change, the people involved in the process change, so it’s important to have a process that evolves along with the products and teams.

So as we did made this transition, people took their places and settled into the new ride. At some point we were comfortable saying that the new process works. So that’s it? We’re done? You know the answer is “no” because this post continues below. First of all, you’re never done because of the nature of the agile process. On top of that, we have a new problem to face: this process is lead by specific people, so what happens when these people are on vacation or out sick? Well… that’s when you learn that you really don’t know something until you have experience doing it.

Humans by nature don’t pay attention to 100% of what happens around them; brains filter information that “useless”. So a process that we thought was really simple and that people had gone through many times was suddenly not that clear when somebody new played the role of “scrum master”.

Responsibility

So after a sprint planning session that didn’t work out as well as it could’ve, we decided to write a “scrum master script”, so that people could play the part by basically just reading from it as a last resort. While was writing it, we realized that we were creating a document about responsibility.

The goal of the scrum master is to run the meeting in the most productive (and efficient) way. This means that other parts, not just that of the scrum master, need to be played just right.

The simple script turned out to be a more interesting document about what makes the whole thing move smoothly. This, tied with rotating scrum masters (intended to increase team engagement), makes up our current process.

Below is a slightly edited version of the actual document.

Scrum Master

Goals

The goal of a scrum master is to guide the sprint planning in the most efficient way making sure the team is operating at the capacity it needs to operate.

The team along with the scrum master need to be aware of the metrics around past performance to potentially tweak how the current sprint is being planned, maybe increasing or decreasing the capacity for the team either long term, or short term depending on circumstances such as team size, vacations, risk, trainees, etc.

The team leads need to work with their teams to make sure the sprint planning starts with enough preparation that virtually no discussion over what tickets to grab is necessary. Team wide planning discussions are welcome and encouraged. Inter team dependency discussions are encouraged. Individual planning should be treated separately as it’s something to figure out between the team lead and each individual contributor.

Sprint planning meetings are attended not only by the development team but by people that are not on the team doing active development, but are impacted in some way by the progress of the team. They need to leave the meeting with a clear understanding of what the team is progressing towards. This is the main reason for entering the meeting as an individual contributor with a clear understanding of the personal goals for the sprint.

Retrospective

This is the most important point of the meeting. The goal is to evolve the process to suit the team best, understanding what worked and what didn’t is key to do this.

Note that the goal is to focus on improvement as opposed to blame, so the way to phrase the questions should always be towards this objective: prefer saying “what could’ve been done better?” rather than “what went wrong?”. Focus first on what went well, then what could’ve been done better.

The scrum master needs to read the previous retrospective (at least) to make sure that things that could’ve gone better were addressed, or make sure that the team is still aware of them. Any action items that is not checked from the previous retrospective should be addressed as part of this conversation.

Closing the current sprint

Before closing the current sprint it’s important to go down the list of tickets and what status they are on. The ideal sprint will have all tickets closed. A more real world sprint will have some tickets that are in review, some in progress and maybe some not even started.

Guiding a conversation about the items that are still in the air is important. At this point is where external dependencies might appear, problems with estimation, amount of points per developer, etc.

Not all developers operate at the same capacity, eg: a developer new to the project will probably complete less story points than one that already knows the project. Making sure to raise these situations and adapt points as time goes by is a joint responsibility between the scrum master and the team lead.

Tickets that were in the air and are moved to the next sprint should go through one or more of the following changes:

• No changes (only if the ticket wasn’t started at all).
• Point reduction: if some work was done, the complexity needs to decrease.
• Task break down: if the complexity does not decrease, then the task is not well designed and should be broken down into smaller tasks (this needs to happen before the call in the best case, after the call in the worse case, involving the team lead and the individual contributor working on the task, but it shouldn’t delay the meeting).

Once everything is clarified, the sprint can be closed.

Starting a new sprint

Ideally, developers should add their own tasks to the sprint. Alternately, the scrum master can go through each person in the team making sure everyone has added tasks.

At the end of it, the scrum master should compare the points in the current sprint against the ones in the previous sprint and the against the math for the team. Ideally all the numbers will be similar, but if they differ in a significant way, this should be analyzed.

Once the team wide number makes sense, each individual’s points needs to be quickly recapped to make sure the overall math is uneven. If it’s uneven, it should be explicitly addressed.

The sprint should be started once the distributed work makes sense for everyone.

Extra work

Sprints plans rarely work exactly as they are intended. The goal is always to reduce the distance between plan and reality, which will give us the ultimate goal of predictability.

Here are the possible situations at the end of the sprint, and how they should be handled:

• Planned work was not finished: this needs to be analyzed in the retrospective to understand why it happened. If it happens somewhat consistently for a team or an individual, capacity for sprint planning may need to be lowered, or there may be a problem of under-estimating effort.
• Planned work gets done before the sprint is finished: new tasks should be taken out of the backlog, which should be ordered by priority. This should be noted in the retrospective, and if the behavior is consistent, planned story points should be increased. This work should not be added to the current sprint.
• New/Interruptive work appears in the middle of the sprint (eg: new dependency on a ticket, or a high priority new task that preempts others): should be added to the sprint. The result by the end of the sprint of adding these tasks should fall in the above options, and should be handled accordingly.

That looks like a lot of structure!

Our workflow is indeed structured. How structured in some cases depends on which team. This is what works for us, it doesn’t necessarily work for every team. There’s no magic bullet here.

SpiderOak is a company that started as a really loosely structured company. This looked amazing in many ways, but it turned out to have a lot of drawbacks for us.

Having a clear process for how development works has given us a stable structure that remains standing and operating no matter whether there are earthquakes around it or not. This in turn gives us more predictability in development output, and that makes conversations with product, business, design, sales, and every other department a lot easier.