You cannot reliably detect AI-generated text, and detectors produce false accusations that fall hardest on non-native English speakers. The workable response is to change what you assess so that AI assistance stops being decisive, gather integrity signals that route to a human rather than an automatic verdict, and decide explicitly where AI use is legitimate.
- AI-text detectors are not accurate enough to accuse anyone. Treating their output as evidence is the real risk.
- False positives concentrate on non-native English writers, which converts a detector into a discrimination problem.
- Unique-per-session generation removes the shared answer key, which is where most leakage actually comes from.
- Integrity signals should produce a risk score for a human to assess — never an automatic fail.
- Decide where AI use is legitimate and say so. Unstated rules are unenforceable and unfair.
Start from the honest position
You cannot reliably tell whether a block of text was written with AI assistance.
Detectors exist and they are marketed confidently, but the underlying task is genuinely hard: fluent, well-structured prose is what both a good writer and a language model produce. Lightly edited model output routinely passes. And the false positives are not random — they cluster on writing that is grammatically correct, plainly structured and slightly formulaic, which is an excellent description of competent English from someone who learned it as a second language.
That combination makes detectors worse than useless for hiring decisions. A false negative costs you a weak signal. A false positive is an accusation of dishonesty against a candidate who did nothing wrong, concentrated on a group you are legally obliged not to disadvantage. If you are running a detector as a gate, you have built an adverse-impact problem and called it integrity.
So the question is not "how do we catch them". It is "what should we be measuring, given that this tool exists?"
The three real problems, separated
Lumping these together is why the topic feels unsolvable. They have different answers.
1. The answer key problem. Everyone gets the same assessment, so answers circulate. This predates AI by decades and is by far the largest source of leakage. It is also the most solvable.
2. The unaided-reasoning problem. You want to know whether this person can reason through something, and an assistant is doing the reasoning.
3. The impersonation problem. The person doing the assessment is not the person applying. Genuinely different — see fake candidates and proxy interviews.
Fixing the answer key problem
Generate each assessment uniquely per session. If no two candidates receive the same items, there is nothing to share, and a leaked answer is worth nothing to the next person. This single change removes most of the value of the whole grey market in assessment answers.
Detect duplicates rather than detecting AI. You cannot tell whether one answer was AI-assisted. You can tell that four unrelated candidates submitted the same distinctive answer. Cross-candidate similarity is a far stronger signal than any stylistic classifier, and it does not require guessing at authorship.
Stop reusing last year's questions. Public question banks are public.
Fixing the unaided-reasoning problem
The trick is to design tasks where assistance is available but not decisive. A model can produce a plausible answer; it cannot supply judgement about your specific context, and it cannot defend a choice under follow-up.
Formats that hold up:
- Critique, not generate. Give the candidate a flawed design or a broken implementation and ask what is wrong and why. Models produce confident, generic critiques; the specific, prioritised ones are hard to fake.
- Trade-off questions with no clean answer. "Which of these two approaches, and what breaks first?" Judgement about consequences, not recall.
- Context the model does not have. Constraints specific to a scenario, where the generic best-practice answer is wrong.
- Follow-up on their own submission. The cheapest and most effective integrity check in existence: ask someone to explain a decision they supposedly made. Depth is very difficult to improvise.
- Iteration under changing requirements. "Now the third-party API is rate-limited to 10 requests a second. What changes?"
Notice that all of these also make the assessment better — they measure judgement rather than recall. That is not a coincidence. Assessments vulnerable to AI are usually the ones that were measuring recall-of-a-known-answer, which was never very predictive anyway.
Signals worth collecting, and how to use them
There is a legitimate middle path between "no integrity measures" and surveillance. The principle: measure the session, not the person's body or their home.
Reasonable signals:
- Response latency patterns — a long pause followed by a large, polished block of text is a different behavioural pattern from continuous composition
- Typing cadence — composition looks different from transcription
- Paste behaviour — whether content arrived as a unit
- Cross-candidate similarity — as above, the strongest of the set
- Consistency between assessment and follow-up — the candidate should be able to discuss what they submitted
- Identity assurance at session start — confirming who is sitting the assessment, once, rather than monitoring them throughout
Three rules for using them:
- Combine them. Any single signal has an innocent explanation. Someone who thinks, then writes fast, is not cheating.
- Route risk to a human. A risk score should open a review, never issue a verdict. An automated integrity fail is an automated adverse decision about a person, with all the oversight obligations that implies — and it will be wrong sometimes.
- Give the candidate a route. If a risk flag affects someone, they should be able to respond. A fifteen-minute conversation resolves most flags in both directions.
Decide your AI policy and publish it
This is the part most teams skip, and it is the part candidates most resent.
For a growing number of roles, using AI well is the job. An assessment that forbids it is measuring compliance with an artificial constraint. For those roles, consider assessing the thing you actually care about: can they use the tool and catch it when it is wrong? "Here is a model-generated implementation with two subtle bugs — find them and explain the fix" is a superb question that would have been impossible to write three years ago.
For assessments genuinely about unaided reasoning, say so plainly, explain why, and design so that assistance is not decisive.
What is indefensible is leaving the rule unstated, letting candidates guess, and then penalising the ones who guessed differently.
How CalHire's integrity layer works
CalHire's approach is deliberately built around the constraints above:
- Unique-per-session generation. Assessments are AI-generated per session, so there is no shared answer key to circulate.
- Behavioural and originality signals, not surveillance. Response latency, typing cadence, LLM-likeness and originality signals combine into a risk score. We explicitly do not use webcam snapshot proctoring — see why.
- Identity assurance at the point of assessment, with liveness checks, rather than continuous monitoring.
- Risk routes to a human, never an auto-fail. This is an architectural guarantee: no code path can auto-reject a candidate. A flagged session becomes a human review.
- Text-only AI interview. Video is human-only, and only after a consented reveal.
- The AI never sees who the candidate is, so an integrity assessment cannot be coloured by identity.
The integrity layer is described on the features page.
The uncomfortable conclusion: AI did not create an integrity crisis so much as expose which assessments were only ever measuring whether someone had seen the answer. Those needed replacing regardless. The ones that measure judgement are largely fine — and they were the better assessments all along.
Frequently asked questions
- Can you detect if a candidate used ChatGPT?
- Not reliably from the text alone. AI-text classifiers produce both false negatives — lightly edited output often passes — and false positives, and the false positives cluster on writing that is grammatically plain or formulaic, which describes a great deal of competent non-native English. No current detector is accurate enough to justify rejecting a candidate on its output.
- Why are AI detectors risky rather than just imperfect?
- Because a false positive is an accusation of dishonesty. If your false positives fall disproportionately on candidates who learned English as a second language, you have built a screen that removes a protected group at a different rate for a reason unrelated to job ability — which is an adverse-impact problem as well as an ethical one.
- Should candidates be allowed to use AI in an assessment?
- Decide deliberately and state it. For many roles, using AI well is now part of the work, and an assessment that pretends otherwise measures compliance with an artificial rule. For assessments genuinely about unaided reasoning, say that explicitly and design the task so assistance is not decisive. What you cannot do is leave the rule unstated and then penalise people for guessing wrong.
- What signals are legitimate to collect?
- Signals about the session rather than the person’s body: response latency patterns, typing cadence, whether content appears pasted, whether the same answer appears across unrelated candidates, and consistency between an assessment and a follow-up conversation. These are proportionate, do not require surveillance of someone’s home, and should feed a human review rather than an automated verdict.