Do reviewers score sourcing samples consistently?

# Do reviewers score sourcing samples consistently?

Published November 6, 2026

Question and scope

This file review asks what the sourcing sample scoring consistency record can show for a single recruiting team. It is a local method, not a sector rate or a hiring standard. The result applies to one population and one observation window. A recruiting lead can reproduce the check from the definition. The evidence standard separates an observed gap from a hiring judgment.

Population and unit

Track one sourcing work sample from the first signal to the verified close. State the opening event, the closing evidence, and the pause rule before counting. A bare tally is not a measurement if the pipeline changes size during the window. List each exclusion so a second reviewer can rebuild the sample. A rejection note cites the specific evidence gap and the reviewer who closed it.

Definitions and measures

Before opening the queue, agree on what complete means for sourcing sample scoring consistency. Name the required fields, the acceptable source records, and the rule for a blank value. Report the eligible count, the exclusions, the sample size, and the unresolved items. A rate without a denominator and a window is not a finding. The panel owner checks the calendar and the interview date before confirming the slot.

Evidence standard

For each sampled record, capture the source, the time observed, the field supported, the reviewer, and the recruiting owner. Missing and confirmed-negative values stay apart in the report. Candidate detail remains in the approved system and notes use a reference. Collect only evidence the protocol can defend. The panel owner checks the calendar and the interview date before confirming the slot.

Sampling and collection

Select routine, exception, and reopened records across the window. If a definition changes, preserve the earlier observation and write down the date. Example: Two reviewers score the same sample against different criteria. A pipeline export is the starting point, not a list assembled after the result is known. A rejection note cites the specific evidence gap and the reviewer who closed it.

Analysis and reporting

Report what was seen before what it might mean. Show the oldest open record and the missing fields, not only completion rate. Name the next decision and the recruiting owner who must make it. Put the sample size beside every rate in the sourcing sample scoring consistency report. Candidate consent and communication preferences stay visible in the record.

Limits and sources of error

Recruiting and recordkeeping guidance gives general framing but cannot answer this sample or a specific hiring question. A complete file is not proof that a hiring decision was correct. The method measures evidence quality, not candidate quality. Limits belong beside the result. Any missing candidate evidence is marked missing until the source arrives.

Review cadence

Agree a fixed review date. End with one process change, one owner, and one check date. A repeated defect should change the template, the source, or the rule before coaching. Reuse the same sample rule each time and attach the review note to the change. Candidate consent and communication preferences stay visible in the record.