Capability
AI governance, bias auditing and the audit trail
Regulators are no longer asking whether your hiring tools are fair. They are asking to see the working. This is the part of the platform that produces it.
AI hiring governance means being able to show how an automated system influenced a hiring decision, whether it produced different outcomes across groups, and who the human was that decided. CalHire computes adverse-impact ratios against the four-fifths rule, writes every score, weight and decision to a hash-chained audit trail, gives talent a route to appeal a score, and handles data-subject requests through a dedicated console. The evidence is produced continuously rather than assembled when someone asks.
Last reviewed
The short version
- Adverse-impact ratios are computed against the four-fifths rule, on disclosed cohorts of at least 30, with groups under 5 reported but never flagged.
- Every score, weight, flag, human decision and identity disclosure lands on a hash-chained audit trail.
- Talent can appeal a score. A person reviews it and the outcome is recorded.
- Access, correction, export and erasure requests are handled through a console, with the audit record kept as required.
What changed, and why the burden moved
For most of the history of hiring software, fairness was a claim a vendor made in a brochure. That period is ending. It is ending in three jurisdictions at once, and the shape of the obligation is broadly consistent across them: know what your system does, check whether it treats groups differently, tell people it is being used, and keep a human in the decision.
The EU AI Act classifies systems used for recruitment and applicant evaluation as high risk, with obligations covering risk management, data governance, logging, transparency and human oversight. New York City requires an annual independent bias audit for automated employment decision tools, published results, and notice to applicants. Colorado requires developers and deployers of high-risk AI systems to exercise reasonable care against algorithmic discrimination and to notify the Attorney General on discovering it. Underneath all of it, in the United States, the four-fifths rule from the Uniform Guidelines has been the working test for adverse impact since 1978.
The practical consequence is that evidence has to exist before it is requested. A bias audit you can only run by exporting a year of data into a spreadsheet is an audit you will do once, badly, under time pressure.
How adverse impact is actually computed
The maths is simple. The care is in knowing when a number is large enough to mean anything.
- The four-fifths rule
- Compare each group’s selection rate to the highest group’s selection rate. If the ratio falls below 0.8, that is an adverse-impact signal requiring investigation. It is a screening heuristic from the Uniform Guidelines, not a legal verdict, and both regulators and courts treat it that way.
- A minimum disclosed cohort of 30
- A category is only analysed once at least 30 people have disclosed it. The threshold applies to the disclosed total rather than the whole applicant pool, because undisclosed applicants cannot be assigned to a group and should not silently inflate a denominator.
- A minimum group size of 5
- Groups smaller than five are reported but never flagged. In a group of three, one selection swings the ratio from 0 to 1. Flagging that would generate noise indistinguishable from a finding.
- Disclosure is voluntary, and separate
- Demographic data is collected for measurement only, on an opt-in basis, and is not available to any scoring or matching path. You cannot measure disparate outcomes without collecting the categories, and you must not let the measurement data leak into the decision.
What the audit trail records
Scores and weights
Every component score and the exact weighting in force when it was computed. A score without its weights cannot be reconstructed, and an unreconstructable score is not evidence.
The human who decided
Advance, decline, override and appeal outcomes are attributed to a named actor with a timestamp and a role. "The system decided" is never an available answer.
Identity disclosures
Every reveal request, consent, decline and disclosure, visible to both the company and the talent it concerns.
Integrity signals
What was flagged, at what confidence, on what evidence, and what a reviewer concluded. Including the signals excluded from scoring for fairness.
Hash chaining
Records are chained so that altering an earlier entry invalidates everything after it. An audit trail that can be quietly edited is a log, not evidence.
Exportable evidence
Compliance exports produce the record set an auditor or regulator asks for, without an engineer writing a query against production.
What talent can do about a score
- 1
They are told a score exists
Report cards state where they placed and against what. A talent who does not know they were scored cannot meaningfully contest it, which is the gap most transparency rules are aimed at.
- 2
They appeal
A first-class flow, not a support ticket. The appeal states what they believe was wrong.
- 3
A person reviews it
Not the model that produced the score. The reviewer sees the answers, the rubric, and any integrity signals on the session.
- 4
The outcome is recorded either way
Upheld or declined, the appeal and its resolution join the audit trail. A successful appeal updates the score and the ranking that depended on it.
The primary sources, not our summary of them
Everything above is our reading of these documents. Where they and this page disagree, they are right. This is general information and not legal advice.
- Regulation (EU) 2024/1689 — the Artificial Intelligence ActOfficial Journal of the European Union
- Automated Employment Decision Tools (Local Law 144)NYC Department of Consumer and Worker Protection
- Employment Tests and Selection ProceduresU.S. Equal Employment Opportunity Commission
- 29 CFR Part 1607 — Uniform Guidelines on Employee Selection ProceduresU.S. Government Publishing Office
- Senate Bill 24-205 — Consumer Protections for Artificial IntelligenceColorado General Assembly
- Regulation (EU) 2016/679 — General Data Protection RegulationOfficial Journal of the European Union
What this does not do
It does not make you compliant. It produces the evidence and the controls that compliance requires, and it removes the excuse that the data was never captured. Whether your overall process satisfies a specific regulator in a specific jurisdiction is a question for your counsel, and nothing on this page is legal advice.
An in-platform adverse-impact analysis is not an independent bias audit. New York City requires the audit to be performed by an independent auditor. Our role is to make that engagement cheap and fast by having the data ready in the right shape; we cannot be the independent party.
A four-fifths ratio above 0.8 does not prove your process is fair. It means one screening heuristic did not fire. Intersectional effects, small groups below the flagging floor, and disparities in who applies at all can all sit comfortably behind a passing ratio.
The audit trail records what happened in CalHire. Decisions made in a hallway, a private message or a spreadsheet are invisible to it, and a process that routes its real decisions outside the platform will produce a clean trail that documents very little.
Questions people actually ask
Does CalHire count as an automated employment decision tool under NYC Local Law 144?
Who performs the bias audit?
How do you compute selection rates without discriminating?
What happens if an adverse-impact ratio falls below 0.8?
Can we export everything for an auditor or a regulator?
How are data-subject requests handled?
Related
Where this connects to the rest of the platform.
Composite scoring and blind ranking
A job score you set the weights for, defaulting to 35 percent vetted score, 40 percent the job’s own interview and 25 percent match. It orders a list. It never makes a decision.
Assessment integrity without surveillance
Page, paste, timing and identity signals, each with PII-free evidence. No video recording, no verdicts, and environmental flags are excluded from scoring where accommodations apply.
Enterprise controls
SAML single sign-on, job-scoped permissions, departments and approvals, custom domains, white-labelling and data residency, with tenant isolation enforced in the database.
Read the reasoning
The evidence and the argument behind what is on this page.
What an AI bias audit actually measures
A bias audit is an outcome analysis, not a code review. What impact ratios are, what an audit cannot tell you, and how to read a summary you have been handed.
ReadThe four-fifths rule: how to test your hiring for adverse impact
The four-fifths rule is a rule of thumb, not a legal safe harbour. How to calculate it, where it misleads, and how to find the stage causing the disparity.
ReadThe EU AI Act and hiring: what recruitment teams are responsible for
AI used to recruit or evaluate candidates is high-risk under the EU AI Act. What that means for employers, what providers owe you, and what to ask vendors.
ReadNYC Local Law 144: a practical compliance checklist
Local Law 144 requires an annual independent bias audit, a published summary, and advance notice to candidates. What each obligation means operationally.
ReadWhat "meaningful human oversight" actually requires
A reviewer who rubber-stamps 200 rankings is not oversight. What regulators mean by meaningful human involvement, and how to build a process that holds up.
ReadGDPR and candidate data: what recruiting teams get wrong
Consent is usually the wrong lawful basis for recruitment. What to rely on instead, how long to keep applications, and the rules on automated decisions.
ReadSee a verified pipeline for one of your roles
Post a role free and review anonymous, skill-ranked talent. No card, no sales call to get started.