A shortlist is only as good as the criteria that filter it. Defining them before the search is what turns a list of names into a decision you can defend.
Many searches start with a job description and end with a list of candidates who “fit.” The problem is that nobody agreed beforehand on what fitting means. Each person in the process evaluates with their own yardstick, and the shortlist ends up reflecting opinions instead of requirements.
What a role scorecard is
A scorecard is a short document that translates the role into expected outcomes and observable competencies. It doesn't describe the ideal person: it describes what they will need to achieve in their first months and what evidence would tell us they can do it.
- The mission of the role in one or two sentences.
- The concrete outcomes expected within a defined time frame.
- The technical and behavioral competencies those outcomes require.
- For each competency, what evidence counts and how it is scored.
Why it improves the shortlist
When criteria exist before the search, the search becomes more precise. Search terms come from the scorecard, not from intuition, and every candidate is compared against the same standard. That cuts noise: you stop receiving profiles that resemble the role in title but not in the outcomes they will have to deliver.
It also changes the conversation with the hiring manager. Instead of debating whether someone “looks good,” you discuss what evidence they bring for each criterion. Disagreements become useful, because they point to a poorly defined criterion or to missing evidence.
How to start
You don't need a complex template. Sit down with the person who owns the role before the opening is published and answer three questions: what has to be solved within six months, what someone who achieves it knows how to do, and how we would verify it in an interview. With those answers in writing, the shortlist stops being a list of names and becomes a recommendation you can explain.
The scorecard doesn't replace the judgment of whoever decides. It makes that judgment visible, and that is what lets you improve it from one search to the next.