Good shortlisting criteria are written down before the first application is opened, split into essential and desirable requirements, linked to the real duties of the job, and applied to every candidate with the same scorecard. To shortlist candidates, you check each applicant against the essential criteria, use the desirable ones to narrow a large pool, rank the remaining candidates by score, and record the reason for every decision. This guide explains each step, with a worked scorecard you can adapt.

Key takeaways

• Shortlisting ranks candidates who already meet the minimum requirements. Screening is the earlier pass that removes those who do not.

• Define criteria before advertising the role, separate essential from desirable, and state what evidence counts for each.

• Score every applicant on the same anchored scale, set a minimum for each essential criterion, and record notes next to every score.

• A shortlist of roughly four to six candidates is typical, but interview capacity and the strength of the pool should set the number.

• Software can handle the repetitive matching and record-keeping, while people keep the judgment on borderline and context-heavy cases.

Table of Contents

• What shortlisting means and how it differs from screening

• The criteria to define before you read a single CV

• Build a scorecard so every candidate is judged the same way

• How to shortlist candidates from a large applicant pool

• Keeping shortlisting fair, consistent and defensible

• How many candidates to put on the shortlist

• Where software helps, and where it should not decide alone

• Our perspective: criteria first, tools second

• Shortlist faster with Recruitify

• FAQ

• Sources

What shortlisting means and how it differs from screening

Shortlisting is the step that decides which applicants are invited to interview or testing. The London School of Economics describes it as assessing applications against essential and desirable criteria and putting forward those who meet them. The Labour Relations Agency in Northern Ireland frames it similarly: applicants who meet the criteria in the person specification move on to interview or tests.

The terms are often used loosely, so it helps to separate them:

The Victorian Public Sector Commission in Australia draws the same line: screening determines which applicants meet the minimum key selection criteria, while shortlisting ranks applicants for progression. A recent employer guide makes the practical point that shortlisting compares those who passed screening using more detailed criteria.

In a small pool, the two steps can merge into one read-through. With hundreds of applications, keeping them apart saves time: screening is quick and binary, and shortlisting is slower and comparative.

The criteria to define before you read a single CV

Criteria written after reading applications tend to describe whoever impressed you first. The LSE guide advises agreeing in advance which competencies from the person specification will be assessed, basing initial decisions on the essential criteria, and using desirable ones only to narrow a large pool of suitable people.

Essential and desirable criteria

Essential criteria are the requirements without which the person cannot do the job from day one. Desirable criteria are the extras that separate two qualified candidates. The University of Illinois Chicago recommends keeping required and preferred qualifications clearly apart and linking each one directly to job duties.

A practical test for every line on your list: if a candidate lacked this, would the work genuinely fail? If not, it is desirable, not essential. Long essential lists quietly exclude capable people and produce thin shortlists.

Evidence, not just quantities

“Five years of experience” is easy to count and weak as a predictor. The same UIC guidance suggests defining what acceptable evidence looks like beyond simple quantities such as years. For a recruiter, that means writing down, for each criterion, what a strong answer would show on a CV, in a portfolio, in a screening answer or in a work sample.

Utah State University adds guidance on screening questions: they should relate directly to the position, explain clearly what information is wanted, allow transferable experience, and be asked of all applicants in the same way.

Common criteria categories

The exact list depends on the role, but most shortlisting frameworks draw from the same categories:

1. Technical or professional skills: the tools, methods or certifications the work requires.

2. Relevant experience: the kind of problems solved and the scale or setting, not just the duration.

3. Qualifications: only where the role or a regulator genuinely requires them.

4. Behavioral competencies: collaboration, ownership, communication, shown through examples.

5. Practical constraints: location or work authorization, availability, and compensation range where relevant to eligibility.

Agencies placing contractors usually add commercial criteria such as rate expectations and notice period, since a perfect technical fit who cannot start or cannot meet the client’s budget is not a usable shortlist entry.

Build a scorecard so every candidate is judged the same way

The most dependable way to apply criteria consistently is a common scorecard. The Victorian guidance recommends rating every applicant against the same criteria with anchored descriptions, setting minimum thresholds for the essential criteria, and ranking eligible candidates by total or weighted results where appropriate. Evidence and notes are recorded, and panel members compare scores and resolve differences using rules agreed beforehand.

Its shortlisting-plan example uses a 1-to-5 scale, a minimum rating of 3 out of 5 on each criterion and a total threshold of 12 out of 20, with candidates who meet every minimum proceeding. You do not have to copy those numbers, but the structure is worth copying: a scale, a floor per criterion, and a total.

An illustrative scorecard

The example below is a hypothetical scorecard for a senior backend engineer role, built to show the mechanics. The weights and anchors are illustrative, not a standard.

Two rules make this work. First, an essential criterion scoring below 3 removes the candidate from the shortlist regardless of the total. Second, the desirable rows only break ties among those who pass, which is exactly the role the LSE guide gives them.

Why anchored descriptions matter

Without descriptions for what a 1, 3 and 5 look like, two reviewers will use the same scale differently, and one person’s “4” becomes another’s “3”. Writing the anchors before review forces the team to agree what strong looks like and gives you something concrete to point to when a hiring manager asks why a candidate was left out.

How to shortlist candidates from a large applicant pool

Guidance from the LSE and Cumberland Council in the UK points to the same sequence. Cumberland recommends setting aside sufficient time, working from the job description and person specification, reviewing evidence against essential and desirable criteria, and discussing scores with panel members for consistency. Put together, the process looks like this:

1. Agree the criteria and the scorecard. Do it with the hiring manager and any panel members before the role is advertised, and confirm which criteria are essential.

2. Screen against the essentials. Remove applicants who clearly fail a minimum requirement, and note the reason in one line.

3. Score the remaining applicants independently. Each reviewer scores alone, using the same scale, with notes tied to evidence in the application.

4. Compare scores and resolve gaps. Where reviewers differ by two or more points, discuss the evidence rather than averaging blindly.

5. Apply the desirable criteria to narrow. Use them only if more candidates pass than you can interview.

6. Rank and cut to the planned number. Invite the top-rated candidates up to interview capacity.

7. Record the decision. Keep scores, comments and reasons for rejection in your recruitment system.

Step 3 is where volume hurts. Reading 300 applications carefully is slow, and fatigue pushes reviewers toward shortcuts such as favoring familiar employers. Many teams therefore let software do the first-pass matching and keep humans for the scoring that needs judgment. We return to that below.

Keeping shortlisting fair, consistent and defensible

Shortlisting is where unconscious bias does the most damage, because decisions are fast and rarely examined. Several sources point to the same defenses.

• Fix the criteria early. UIC recommends developing the screening tool before advertising, discussing what demonstrates strength, and applying the matrix rigorously at each stage to limit cognitive and structural bias.

• Use more than one pair of eyes. The LSE recommends that all panel members contribute where possible, with majority agreement on the final list, and that scores and comments be recorded in the recruitment system.

• Evaluate individually, then compare. Yale advises assessing each candidate on their own before a side-by-side comparison, and warns that switching repeatedly between the two modes can introduce recency bias.

• Judge behavior and performance. The same guide advises focusing on actions and performance of essential job tasks rather than likeability or personality.

• Prefer the specification to intuition. The Open University argues that systematic use of job-specification criteria supports choosing the strongest candidates and documenting why others were rejected.

The documentation point matters beyond fairness. If a rejected applicant, a client or an auditor asks why someone was not progressed, a short written reason tied to a criterion is a far better answer than a recollection. Processing candidate data also carries privacy obligations in many jurisdictions, so keep notes factual and limited to job-related evidence.

How many candidates to put on the shortlist

There is no universal number. The Open University notes that it is common to shortlist up to six applicants per position, with the figure reflecting interview time and the strength of the field. WorkWise guidance describes four to six interviewees as typically suitable, depending on the role, and recommends using desirable criteria when many applicants meet every essential one.

The Victorian guide gives a simple rule for the oversupplied case: if many candidates meet the minimum, return to your planned interview capacity and invite the top-rated. Five interview slots means the top five by rating.

For agency recruiters, the client sets a second constraint. A client submission usually works better with a small number of strong, well-presented profiles than a long list that makes the client do the sorting. Agree the expected number in the intake call, and say what “top” means in terms of the scorecard.

After the list is final, hold the same standard in the next stage. Interview questions should map back to the criteria, so that what you learned in shortlisting is tested rather than repeated.

Where software helps, and where it should not decide alone

The weak point of a manual process is not the criteria but their consistent application across hundreds of CVs. Three tasks suit automation:

• Extracting structured data from PDFs, Word files, scans and photos, so every candidate is compared on the same fields.

• Matching against requirements and producing a ranked list with a visible score for each criterion.

• Keeping the record: scores, notes, consent and decisions in one place.

The decisions that should stay with people are the borderline cases, unusual career paths and anything involving context a CV does not show. Scoring is a way to prioritize attention, not to replace the recruiter’s final call. We discuss this balance in more detail in our article on AI scoring that keeps the human touch, and on how a screening agent turns high application volume into a manageable queue.

If you are choosing a system, check that it can hold your scorecard, not just a keyword filter. Our overview of applicant tracking system features lists what to look for, and the piece on why an ATS alone is not a recruitment system explains the gap between storing applications and running a process.

Our perspective: criteria first, tools second

We build recruitment software, and the most common mistake we see is buying automation before the criteria exist. A matching engine can only be as good as the definition it is given. If the essential list is vague, the ranking will be vague too, only faster.

For IT recruitment in particular, we think keyword matching is the weakest form of shortlisting. A candidate who lists one framework often understands its neighbors, and a candidate who lists ten buzzwords may understand none. That is why we designed our matching to read relationships inside a tech stack and to give a percentage match score against the project requirements, so the recruiter starts from a ranked list and can see why each person ranks where they do.

We also hold that every automated score should be explainable and reviewable. The recruiter should be able to open a profile, see which criteria drove the result, and override it. Consent management and a full audit trail support the same goal as the documentation advice above: a decision you can explain later.

Our practical advice is to write the scorecard first, run it by hand on ten applications to test the anchors, and only then automate. If the manual run produces disagreement between reviewers, the criteria need work, not the software.

- Recruitify Team

Shortlist faster with Recruitify

Recruitify combines an ATS, a CRM and contracting tools in one platform, with AI matching and scoring, a CV parser that builds structured profiles from PDFs, DOCX files, scans and photos, and duplicate detection. Recruitment projects let you keep each role’s candidates, stages and notes together, so the shortlist and its reasoning stay in one place. You can see how AI recruitment features work or explore recruitment projects.

Published plans are the HR Team plan at €61 per user per month and the Agency plan at €67 per user per month, with Enterprise available on request. Details are on the pricing page.

FAQ

What is the difference between a longlist and a shortlist?

A longlist holds everyone who passes the first check, often a larger group that is scored against each selection criterion. A shortlist is the smaller group taken forward to interview once those scores are compared. The LSE guide describes using a longlist to score candidates when the field is large.

Should desirable criteria ever remove a candidate?

Not on their own. Essential criteria decide eligibility, and desirable criteria help you choose among eligible candidates when there are more than you can interview. If a missing desirable skill would really block the person from doing the job, it belongs on the essential list instead.

How long should shortlisting take?

It depends on the volume, but allow real time in the plan rather than squeezing it between other work, as the Cumberland guidance advises. Independent scoring by each reviewer, followed by a short comparison meeting, is usually the slowest part. Automated first-pass matching can shorten the reading, not the comparison.

Can AI scoring be used without introducing bias?

No tool removes bias by itself, because a score reflects the criteria and data it is given. Use clearly job-related criteria, review the factors behind each score, keep a person responsible for the decision, and record the reasons. Checking results regularly for patterns that look unfair is also sensible.

What should I tell candidates who are not shortlisted?

Keep the message brief and respectful, and base any feedback on the documented criteria, not on impressions. Having a written reason tied to a criterion makes this easy, and it protects you if the decision is questioned later.

Sources

• A Guide to Shortlisting, London School of Economics

• Best practice recruitment selection methodology and tools, Victorian Public Sector Commission

• Interviews, Labour Relations Agency

• Understanding the difference between screening and shortlisting candidates, Internwise

• Developing Screening Criteria: Step-by-Step Guidance, University of Illinois Chicago

• Writing Effective Screening Questions, Utah State University

• Evaluate Candidates to Make a Final Selection, Yale

• Recruitment and selection, The Open University

• Effective shortlisting guidance, Cumberland Council

• Guidance for shortlisting and longlisting candidates, WorkWise

Recommended

• Why hiring managers no longer trust a “good shortlist.”

• Recruiters are no longer searching for candidates. They’re seeking signals, which means true alignment.

• AI Scoring in Recruitment: The smarter way to screen talent without losing the human touch

• Blind CV & Talent Pools | Pitch candidates safely