calhire
Capability

AI governance, bias auditing and the audit trail

Regulators are no longer asking whether your hiring tools are fair. They are asking to see the working. This is the part of the platform that produces it.

AI hiring governance means being able to show how an automated system influenced a hiring decision, whether it produced different outcomes across groups, and who the human was that decided. CalHire computes adverse-impact ratios against the four-fifths rule, writes every score, weight and decision to a hash-chained audit trail, gives candidates a route to appeal a score, and handles data-subject requests through a dedicated console. The evidence is produced continuously rather than assembled when someone asks.

Last reviewed

The short version

  • Adverse-impact ratios are computed against the four-fifths rule, on disclosed cohorts of at least 30, with groups under 5 reported but never flagged.
  • Every score, weight, flag, human decision and identity disclosure lands on a hash-chained audit trail.
  • Candidates can appeal a score. A person reviews it and the outcome is recorded.
  • Access, correction, export and erasure requests are handled through a console, with the audit record kept as required.

What changed, and why the burden moved

For most of the history of hiring software, fairness was a claim a vendor made in a brochure. That period is ending. It is ending in three jurisdictions at once, and the shape of the obligation is broadly consistent across them: know what your system does, check whether it treats groups differently, tell people it is being used, and keep a human in the decision.

The EU AI Act classifies systems used for recruitment and candidate evaluation as high risk, with obligations covering risk management, data governance, logging, transparency and human oversight. New York City requires an annual independent bias audit for automated employment decision tools, published results, and notice to candidates. Colorado requires developers and deployers of high-risk AI systems to exercise reasonable care against algorithmic discrimination and to notify the Attorney General on discovering it. Underneath all of it, in the United States, the four-fifths rule from the Uniform Guidelines has been the working test for adverse impact since 1978.

The practical consequence is that evidence has to exist before it is requested. A bias audit you can only run by exporting a year of data into a spreadsheet is an audit you will do once, badly, under time pressure.

How adverse impact is actually computed

The maths is simple. The care is in knowing when a number is large enough to mean anything.

The four-fifths rule
Compare each group’s selection rate to the highest group’s selection rate. If the ratio falls below 0.8, that is an adverse-impact signal requiring investigation. It is a screening heuristic from the Uniform Guidelines, not a legal verdict, and both regulators and courts treat it that way.
A minimum disclosed cohort of 30
A category is only analysed once at least 30 people have disclosed it. The threshold applies to the disclosed total rather than the whole applicant pool, because undisclosed candidates cannot be assigned to a group and should not silently inflate a denominator.
A minimum group size of 5
Groups smaller than five are reported but never flagged. In a group of three, one selection swings the ratio from 0 to 1. Flagging that would generate noise indistinguishable from a finding.
Disclosure is voluntary, and separate
Demographic data is collected for measurement only, on an opt-in basis, and is not available to any scoring or matching path. You cannot measure disparate outcomes without collecting the categories, and you must not let the measurement data leak into the decision.

What the audit trail records

  • Scores and weights

    Every component score and the exact weighting in force when it was computed. A score without its weights cannot be reconstructed, and an unreconstructable score is not evidence.

  • The human who decided

    Advance, decline, override and appeal outcomes are attributed to a named actor with a timestamp and a role. "The system decided" is never an available answer.

  • Identity disclosures

    Every reveal request, consent, decline and disclosure, visible to both the employer and the candidate it concerns.

  • Integrity signals

    What was flagged, at what confidence, on what evidence, and what a reviewer concluded. Including the signals excluded from scoring for fairness.

  • Hash chaining

    Records are chained so that altering an earlier entry invalidates everything after it. An audit trail that can be quietly edited is a log, not evidence.

  • Exportable evidence

    Compliance exports produce the record set an auditor or regulator asks for, without an engineer writing a query against production.

What a candidate can do about a score

  1. 1

    They are told a score exists

    Report cards state where they placed and against what. A candidate who does not know they were scored cannot meaningfully contest it, which is the gap most transparency rules are aimed at.

  2. 2

    They appeal

    A first-class flow, not a support ticket. The appeal states what they believe was wrong.

  3. 3

    A person reviews it

    Not the model that produced the score. The reviewer sees the answers, the rubric, and any integrity signals on the session.

  4. 4

    The outcome is recorded either way

    Upheld or declined, the appeal and its resolution join the audit trail. A successful appeal updates the score and the ranking that depended on it.

The primary sources, not our summary of them

Everything above is our reading of these documents. Where they and this page disagree, they are right. This is general information and not legal advice.

What this does not do

It does not make you compliant. It produces the evidence and the controls that compliance requires, and it removes the excuse that the data was never captured. Whether your overall process satisfies a specific regulator in a specific jurisdiction is a question for your counsel, and nothing on this page is legal advice.

An in-platform adverse-impact analysis is not an independent bias audit. New York City requires the audit to be performed by an independent auditor. Our role is to make that engagement cheap and fast by having the data ready in the right shape; we cannot be the independent party.

A four-fifths ratio above 0.8 does not prove your process is fair. It means one screening heuristic did not fire. Intersectional effects, small groups below the flagging floor, and disparities in who applies at all can all sit comfortably behind a passing ratio.

The audit trail records what happened in CalHire. Decisions made in a hallway, a private message or a spreadsheet are invisible to it, and a process that routes its real decisions outside the platform will produce a clean trail that documents very little.

Questions people actually ask

Does CalHire count as an automated employment decision tool under NYC Local Law 144?
Local Law 144 turns on whether a tool substantially assists or replaces discretionary decision-making. A scoring system that ranks candidates can fall within that, which is why we build for it rather than argue about it: candidate notice, an independent bias audit, and published results are all supported. Whether your specific configuration is in scope is a determination for your counsel.
Who performs the bias audit?
An independent third party you engage. The platform produces the data in the shape an auditor needs and tracks the engagement, including flagging when a published audit is coming up for renewal. We do not audit ourselves.
How do you compute selection rates without discriminating?
Demographic categories are collected only with the candidate’s consent, held separately, and are unavailable to any scoring or matching path. They exist to measure outcomes after the fact. Using them as an input is the exact harm the measurement is meant to detect.
What happens if an adverse-impact ratio falls below 0.8?
It is surfaced as a signal for investigation, not an automatic change to anyone’s process. The usual next questions are which stage produced the disparity, whether the cohort is large enough to be meaningful, and whether the requirement driving it is genuinely necessary for the role.
Can we export everything for an auditor or a regulator?
Yes. Compliance exports cover the audit trail, adverse-impact analyses and decision records, with the dated evidence files an auditor expects.
How are data-subject requests handled?
Through a dedicated privacy console covering access, correction, export and erasure, with defined timelines. Records that must be retained for audit purposes are retained, and the reason is stated rather than left implicit.

Related

Where this connects to the rest of the platform.

  • Composite scoring and blind ranking

    A composite score you set the weights for, defaulting to 40 percent test, 35 percent interview and 25 percent role fit. It orders a list. It never makes a decision.

  • Assessment integrity without surveillance

    Four families of signal, each with a confidence level and PII-free evidence. No webcam, no verdicts, and environmental flags are excluded from scoring where accommodations apply.

  • Enterprise controls

    SAML single sign-on, job-scoped permissions, departments and approvals, custom domains, white-labelling and data residency, with tenant isolation enforced in the database.

See a verified pipeline for one of your roles

Post a role free and review anonymous, skill-ranked candidates. No card, no sales call to get started.

Start hiring freeTalk to the team