Structured interviews: the highest-return change most teams never make

What "structured" actually means

Not "we have a list of topics". Four specific things, all of them at once:

1. The same core questions, in the same order, for every candidate for that role.
2. Defined probes — the follow-ups you are allowed to use, decided in advance, so one candidate is not helped along while another is left to flounder.
3. A rubric written before you meet anyone, with scoring bands and anchor examples, including what a mediocre answer looks like.
4. Independent scoring: each interviewer records their score before any discussion.

Remove any one and you lose most of the benefit. The most commonly skipped is the fourth, and it is the cheapest to fix.

Why unstructured interviews feel so much better than they are

An unstructured interview is enormously satisfying. You follow your curiosity, the conversation flows, and you come away with a strong sense of the person. That feeling is the problem: the confidence it produces is not tracking accuracy.

Three mechanisms do most of the damage.

You ask different people different questions. So you never compared them. You compared your impression of candidate A answering question X to your impression of candidate B answering question Y.

Rapport gets scored as competence. Conversational ease is a real skill for some jobs and irrelevant for most. It is also unevenly distributed by culture, first language, neurotype, and how many interviews someone has sat this month.

The first few minutes set the rest. An early impression shapes which follow-ups you choose, how generously you interpret an ambiguous answer, and what you remember afterwards. You then experience the rest of the interview as confirmation.

None of this makes interviewers bad at their jobs. It makes unstructured interviews a poor instrument, in the same way that a badly built assessment can be reliable and still measure nothing — see does your assessment predict anything.

How to build one

Step 1 — Derive questions from the actual work. Take three to five skills from the job description and write one or two questions each. Two formats carry most of the weight:

• Behavioural — "Tell me about a time you shipped something that broke in production. What did you do in the first hour?" Past behaviour, specific instance, not a hypothetical.
• Situational — "You're on call, a payment provider is timing out on 30% of calls, and finance needs today's reconciliation. Walk me through the first thirty minutes." Judgement under realistic constraints.

Avoid brainteasers and questions about the candidate rather than the work. "Where do you see yourself in five years" measures how well someone has rehearsed answering it.

Step 2 — Write the rubric before you see a candidate. For each question, three or four bands with a concrete anchor:

Q: Production incident, first hour. 1 — Weak. Jumps to a fix with no diagnosis; no mention of communication or blast radius. 2 — Adequate. Diagnoses before fixing; mentions rollback; communication is an afterthought. 3 — Strong. Stabilises first, defines blast radius, communicates to a named stakeholder, then diagnoses. Distinguishes mitigation from root cause. 4 — Exceptional. All of the above, plus what they changed afterwards so the class of failure could not recur silently.

Writing band 2 is the hard part and the valuable part. Bands 1 and 4 are easy; nearly every real candidate lands between them.

Step 3 — Define the probes. For each question, list two or three permitted follow-ups. This stops the accidental generosity that makes candidates incomparable — helping the candidate you already like, and letting the one you do not run out of road.

Step 4 — Score independently, then discuss. Everyone submits scores before the debrief. This single change removes the anchoring effect where the most senior or most confident voice speaking first sets the room's baseline. Then discuss the disagreements, which is where the useful information is. Agreement needs no meeting.

Step 5 — Look at your final stage honestly. Whatever your last gate is, that is your real selection criterion. If the process is four structured rounds followed by an unscripted founder chat with veto power, then your selection method is an unscripted founder chat. Either give it a rubric or remove its veto.

What structure costs, honestly

It is worth naming the trade-offs, because pretending there are none is why adoption stalls:

• Upfront work. A rubric per role, written once, revised occasionally. Real, and it amortises.
• It feels rigid at first. Interviewers who pride themselves on reading people experience the constraint as a loss of skill. Some of that loss is the point.
• Repetition. Asking the same questions twenty times is boring. Boring and accurate beats interesting and arbitrary.
• It exposes disagreement. Independent scoring reveals that your team does not actually agree on what "good" means. Uncomfortable, and much better known than unknown.

What you get

Comparability, first — the ability to say candidate A scored higher than candidate B on the same evidence. Then a defensible record: what was asked, how it was scored, who decided. Then better feedback, because a rubric score converts directly into something specific to tell a candidate, which is the difference between useful feedback and a form letter. And finally, reduced exposure to the drift where irrelevant characteristics leak into a decision nobody documented.

How CalHire structures it by default

The text interview on CalHire is generated per session from the skills set for the role, scored against consistent criteria, and it runs under the integrity layer. Two properties matter here:

• It is uniform by construction. Every candidate for a role is assessed on the same skill dimensions, so the comparability does not depend on an interviewer remembering to be consistent.
• The interview never sees who the candidate is. Personal information is stripped before any model is involved, so the score cannot absorb identity signals. The interview is text-only; video is human-only.

The interview contributes to a composite score alongside the skills test and role-fit, with weights the employer sets per role. AI recommends and explains, and a person makes and is recorded for every decision — nothing is auto-rejected. See the features page for the full picture.

If you change one thing in your hiring process this quarter, make it independent scoring before the debrief. It costs nothing and it is the closest thing to free accuracy available.