The title above represents one of the most common questions we got whenever we introduced 2 types of scoring (progress and score).
In actuality, it's not mandatory to have both. Based on our observation, however, teams that implemented both types of scoring often had a more comprehensive appreciation of what they have accomplished. In addition, having these data points are valuable to identify the lessons learned one might have from the previous OKR cadence.
Let's take a deeper look.
Team A – one type of scoring
One of the teams we coached only used one type of scoring. They called it score; using the range 0 - 10. To define it, there are 2 elements of measurement they used:
- Completion – was the Key Result achieved or not?
- Effort – how much effort did she allocate to achieve the KR?
For a while, it all worked fine. It was easier for them to do OKR Review because they only needed to give 1 score.
However, over time, the team lead noticed something was not right. Her team members, on average, scored very high. The average score per person was around 8 or 9 (out of 10). But, what made her scratch her head was the fact that the team OKR was nowhere near completion.
The problem, she realized, was that her team members had difficulty in balancing completion and effort; and unknowingly, they leaned toward the latter more than the former.
As a result, the effort was seen as more important than the completion of the KR itself. The individual achievement could no longer be a measurement of team success.
Team B - two types of scoring
Another team was more opened toward the idea of having 2 types of scoring, even though it took time to convince the whole team to adopt this approach. There were 2 elements of scoring they'd need to maintain:
- Progress – which measures the completion of the KR. In other words, measuring the actual outcome against the expected outcome. The range is in percentage.
- Score – which measures the effort and the quality of the actual outcome itself. The range is 0 - 10.
It required a lot of practice to distinguish the 2 elements above properly. It was not easy, but over time, the team found a different appreciation toward scoring.
They realized that progress and score are not directly proportional. One KR could have high progress, but a low score. And vice versa.
This was not uncommon. John Doerr, the author of Measure What Matters, called this extenuating circumstance.
The analogy is like this. Let's say this is your KR: "Action items and the timeline for the marketing campaign with Client X are received by the end of the week." When Friday came, you didn't do much but miraculously Client X sent the action items and timeline to you. Thus, your progress would be 100%. But, your score wouldn't be 10, because you didn't put much effort into it (you could have done better).
Let's look at another example. Your KR: "5 interviews with the shortlisted candidate for Sales Manager." You have secured all the interviews but 2 cancelled last minute. 1 out of 3 candidates that you interviewed showed high potential. Thus, your progress would be 60% (= 3/5), but your score would be higher, because of 2 reasons:
You put a lot of effort into securing the interviews. 2 candidates cancelled last minute was something outside of your control.
You were satisfied with the quality of the actual outcome itself because 1 candidate was really promising.
To recap: having one type of scoring is not wrong, albeit much harder because everyone in the team must balance the progress and effort for each KR. The team lead must pick up the signal if any of the team members show any tendencies of favoring the effort more than progress.
The other alternative is breaking down the scoring into 2 elements, which is more ideal because it recognizes both progress and effort; giving more visibility on how individual achievement contributes to team OKR, while at the same time appreciating the individual effort. This practice helps to give clarity in regards to what went well, what didn't go well, and on which area improvement should happen.
Want to find out more:
1/ If you're interested in articles like this, you might want to check out our writings on Medium. It covers a diverse topics of Product Management, People Operations, and everything in between.