Running a Quarterly Review Process

Most people inherit the team they are on. They join a team that already has a standup, an on-call rotation, a backlog meeting, a set of Slack channels, and a dozen other small rules about how work gets done.

Sometimes everyone understands why those rules exist. But often the people who created them have left, new people have joined, and the team continues following a process that no one would choose if they were starting over.

To address this, once a quarter I get the team together and ask them to reconsider how they work. We keep what is useful, change what no longer fits, and remove what we no longer need.

I call this a quarterly team process review.

Why quarterly

I don’t like doing process reviews every two weeks. Two weeks usually isn’t enough time to decide whether a change is actually working, and constantly adjusting the process creates a lot of thrash.

Consider a four-person team that changes its on-call rotation from one week to two. After the first two weeks, the person who just spent two weeks on call may hate the new schedule. Everyone else, who just got a longer break from the pager, may love it. That isn’t a particularly useful evaluation of the change.

A quarter gives several people a chance to experience the new rotation. It also gives the team time to see how the change behaves during both quiet and busy weeks.

At the same time, it is important to repeat the review every quarter. If people believe this is their one chance to change the process, they will be reluctant to try anything unless it solves every possible edge case. Knowing that we will revisit the decision in a few months makes it safer to experiment. We can try a partial solution, live with it for a while, and change it again if it doesn’t work.

Do it in person

I prefer to schedule these reviews during an on-site. It is possible to run one remotely, but I get much better participation when everyone is in the same room. People respond more naturally to one another, and it is easier to tell when the group wants to spend more time on a subject.

The setup is intentionally simple. I need a room, a wall or whiteboard, Post-it notes, and markers. This meeting is about the team talking to itself, so the tools should stay out of the way.

Start by getting everyone talking

I start with a deck of open-question cards. Everyone takes a card and we go around the room answering them. Someone can answer the question on their card, or they can answer someone else’s question if it is more interesting.

The questions themselves aren’t important. The point is to get everyone talking before asking them to challenge a process or disagree with one another.

Passing is always allowed. I have had team members tell me they would rather not share, and I respect that. Someone may not be in a state where they want to talk much that day. Giving them a clear way to pass keeps the warm-up voluntary and prevents an exercise intended to create engagement from turning into a demand for personal disclosure.

Explain why the review exists

After the warm-up, I give roughly the same explanation every quarter:

Most of us inherit the team we are on. We join a team with existing ceremonies, communication patterns, and culture. Every person who joins changes that culture, and every person who leaves changes it too, or at least should.

Teams often continue practices created by people who are no longer there. The current team may not understand why a ceremony exists and in some cases it may no longer be needed. The people have changed, the work has changed, or the problem the process solved has disappeared.

This review is a chance to take ownership of the team. We have a few requirements. We need to get work done and we need to communicate our progress. Beyond that, almost anything about how we operate is open for discussion and change.

The team belongs to its members. It does not belong to the manager. My role in this meeting is mostly to facilitate, clarify constraints, and help the group make decisions it can live with for the next quarter.

I repeat this explanation because people forget, but also because the team changes. Someone may have joined since the last review, someone else may have left, and everyone has another quarter of experience with the current process.

The last point also changes how I participate. I may have preferences, especially around on-call, but I try to step back. A manager cannot dominate the discussion and then call the result team ownership.

Give examples that encourage bigger changes

At this point I give examples of things other teams have discussed. These aren’t recommendations, and they aren’t a list the team needs to work through. They are intended to show that the review can go beyond changing the team logo or making some other cosmetic change.

Slack channels

Many teams have a backroom Slack channel for weekend pictures, jokes, and informal conversation. Is that channel public or private? Who should be in it?

I’ve seen backroom channels with fifty people in them because former team members, occasional collaborators, and even customers were added but never removed. Eventually the current team no longer knows what kind of room they are in.

The team may decide that only current members should be in the backroom. They may also decide that pull-request notifications belong in a separate channel from the funny weekend pictures. These sound like small choices, but they directly affect how the team communicates.

On-call rotations

Teams can reconsider the length of the on-call rotation, the relationship between the primary and secondary, and what work an on-call engineer should attempt. I have suggestions about all of these, but I still want the team to reason through the tradeoffs.

I once worked with a four-person team that changed its on-call shift from one week to two. Two weeks was more brutal, but it also meant a longer break before each person went back on call. The team preferred to rip the Band-Aid off and then have more time away from the pager. Another team might make the opposite choice. The point is to make the choice intentionally instead of treating the inherited schedule as inevitable.

Backlog work and handoffs

A team may decide that the primary on-call engineer shouldn’t plan to make progress on an OKR project. Instead, that person can work through small backlog items that are easier to stop and restart when an interruption arrives. Maybe the secondary scrubs the backlog each week. Maybe the team changes what information is passed from one on-call shift to the next, or decides it no longer needs a formal handoff at all.

These choices are connected. Changing the on-call rotation may change how backlog work is handled, which may change the handoff. The review lets the team look at the whole operating system instead of adjusting one ceremony in isolation.

Write down the topics

Once the purpose of the meeting is clear, I hand out the Post-it notes. I give everyone ten to fifteen minutes, or longer if the room needs it, to write down processes and ceremonies they want to discuss. When they are ready, they put the notes on the board.

Starting with quiet writing gives everyone a chance to form their own ideas before the loudest or quickest person in the room sets the agenda.

Group similar notes

I look for notes with enough overlap to discuss together. Before I combine any of them, I ask the people in the room whether the grouping makes sense.

This is important because two notes that look identical to me may have a meaningful difference to the people who wrote them. It is their agenda, so I don’t erase that distinction just because combining the notes would be more convenient.

Vote on what to discuss

Everyone gets a limited number of votes. For a team of around twelve people, I might give each person four. They vote by using a marker to draw a dot, an X, or really any mark next to a subject. They can spread their votes across several subjects or put all four on one thing they care about deeply. I trust people to keep track of their own votes; we don’t need physical dots to control the process.

Before voting, I explain that we will not discuss a topic that gets no marks. I used to work through every note, but this meant spending time on subjects that no one in the room actually cared about. If nobody is willing to spend a vote on a topic, it isn’t important enough for this review.

After the vote, I count the marks and start with the most heavily voted group. From there we work downward while we still have time and interest.

The vote doesn’t decide the answer. It decides which questions get the team’s attention first.

Record the decisions

At the end, I collect the process changes, write them up, and send them to the team. The notes should be specific enough that someone who missed the meeting can understand what changed. We then work under those decisions for the next quarter.

When the next review arrives, we start again. I don’t assume that the team still agrees with the previous decisions. The people may have changed, the work may have changed, and now we have evidence about how the process behaved in practice.

A side benefit of a scheduled review

There is also a side benefit to having a known time for process complaints.

During the quarter, when someone raises a non-urgent problem with a ceremony or team rule, I can say, “That’s a good topic for the quarterly process review.” The concern now has a destination and a date. We don’t need to change the wheels on the car while it is moving every time someone encounters ordinary friction.

This doesn’t mean ignoring a process that is hurting people or preventing the team from working. Some things should change immediately. But most process questions benefit from a little evidence and a room where the whole team can participate.

Remove what is no longer useful

The review also gives the team permission to remove things. Processes tend to accumulate because adding something feels safer than removing it. Once a quarter, the team can ask a simple question: if this ceremony didn’t already exist, would we create it now?

If the answer is no, and nobody can explain the value it provides, we should probably stop doing it.

A feeling of ownership

The most important result of the review isn’t any particular process change. It is the stronger feeling of ownership the team develops over how it works.

When people help create the process, they are more likely to follow and support it. They understand why it exists and have regular opportunities to change it. It no longer feels like they are doing process for the sake of process.

I find this is true even when someone’s preference is overruled by the group. They may not get the outcome they wanted, but they had a chance to make their case and be heard. They leave with a sense that the team has opted to do this rather than my manager is making me do this.

That is a powerful difference.