In this article, we already covered the difference between progress and score. Progress is the objective data point; in which we measure the actual outcome against the expected outcome. In other words, it's about completion.
The score, on the other hand, measures the effort and the quality of the actual outcome. In John Doerr's words, this is the "goal-setter's thoughtful, subjective judgment".
For some people, implementing the score can be a challenge because of the subjectivity. How would you know if you were a 6 instead of a 7? What would differentiate an 8 from a 9?
The subjectivity also opens the possibility of the score being misused. There were a few teams in which all the members scored highly (in the range of 8 or 9) consistently. They could not produce satisfying reasoning when the team asked for it.
All of these lead to a question: should the score be standardized? In other words, should we create a tiering of numbers (from 1 to 10); each having its own definition?
A standardized score
In theory, a standardized score sounds good. It provides a clear set of standards for everyone to follow. It should be easier for everyone to adopt.
However, a standardized score has its downsides. A regulated score can be seen as rigid; it doesn’t leave much room for the individual to appreciate the subtleties regarding the process or the result.
If progress is about completion and it directly correlates with the achievement of team or division OKR (when set up properly), then score is about the appreciation to the individual; to acknowledge the effort they put and the quality of the outcome.
Thus, the score is never meant to be objective like the progress. The appreciation that one gives to herself might be different than the one her peers give to himself. And it should be fine. This practice will enrich the learning process in OKR: you'll learn the characteristics and the standard that your peers set for themselves.
The standardized score also doesn't completely prevent one from abusing the score. If a team member insisted they already gave maximum effort all the time, then her score would always be 3 or 4. Anyone who has the intention to misuse the score will almost likely to find a way to do just that, whether the score is regulated or not. (It signals "a people issue", which is a more pressing matter to be solved than the scoring itself)
An unstandardized score
By looking at the arguments above (that we gathered from real feedback from some of the teams), it seems clear that the unstandardized score might be the better alternative. For one, it doesn't require you to spend time and effort to create the tiering.
In the beginning, scoring oneself might be tricky. People might be guessing whether they score themselves right. However, as time goes by, they should be more adept at this, especially after knowing that their score can be challenged by their peers and their manager. During OKR Review, after one explains her score, the other team members are encouraged to give constructive feedback. As a result, group-learning will happen. When the group-learning is fostered properly, it should help all members to understand how to give the proper score to themselves.
The group-learning will also reveal the standard of the group. You'll know the standard that is set by your peers and your manager. Subconsciously, you should be motivated to adjust to that standard over time.
If the group-learning doesn't happen, or if some of the team members disregard the group-learning and choose to misuse the score even, then the manager has one option left: a calibration process.
By looking at the pattern of the progress and score over several cadences, it would become clear that something is going on. The analysis of comparing the progress and score should be communicated to the respective individual. The goal is not to assign blame, but to raise awareness of the unhealthy trend that can affect the overall group.
The next step would be to formulate action items that are agreed upon by the manager to help the individual do better.
Want to find out more:
1/ If you're interested in articles like this, you might want to check out our writings on Medium. It covers a diverse topics of Product Management, People Operations, and everything in between.