Assessment Vendor Due Diligence: Why a Bias Badge Will Not Shield You

Assessment vendor due diligence is the work of confirming that a hiring tool you did not build will hold up when somebody challenges a rejection. Most buyers skip it, because the vendor arrives with a bias audit certificate, a fairness white paper and a confident sales engineer. None of that moves legal responsibility onto the vendor. Under Title VII, the employer that uses a selection procedure answers for its effects, even when a third party designed it, hosts it and scores it.

That is the position the US Equal Employment Opportunity Commission set out in its May 18, 2023 technical assistance on software, algorithms and artificial intelligence used in employment selection procedures: if an employer administers a selection procedure, it may be responsible under Title VII even where an outside vendor developed the test. The same document notes that an employer can be responsible for the acts of an agent it authorizes to screen on its behalf.

Key Takeaways

• The employer using an assessment or AI hiring tool carries the Title VII exposure for its effects, even when a vendor built, hosted and scored it. A bias-tested claim is a sales representation, not a defense.
• A vendor audit run on pooled client data may not cover you. Under the New York City rules implementing Local Law 144, an employer may rely on an audit built from other employers' historical data only if its own data went into that audit, or it has never used the tool.
• Ask for the technical manual, validation evidence for roles like yours, the reference population and impact ratios stage by stage. One overall pass rate can conceal a failing step.
• Colorado's amended automated decision-making law voids contract provisions that shift a party's own liability for algorithmic discrimination, for consequential decisions made on or after January 1, 2027. Run the four-fifths calculation on your own applicant flow before go-live and annually.

Responsibility Does Not Move With the Invoice

Buying a tool buys capability, not cover. The Uniform Guidelines on Employee Selection Procedures (29 CFR Part 1607) attach their obligations to the user of a selection procedure: the party that records applicant flow, shows job-relatedness and considers less discriminatory alternatives. A vendor has no equivalent duty there. It has a contract with you.

State and city rules follow the same logic. New York City's Local Law 144 places the bias audit, the published results summary and the candidate notice on the employer or employment agency, not on the software company. Colorado's automated decision-making law splits fault between developer and user but keeps the user answerable for how it configured and applied the tool. The party that made the hiring decision is the party on the hook. For the duty framework rather than the buying process, read AI Vendor Accountability Recruitment: Your Legal Duty; this article is the procurement instrument underneath it.

What a Bias Audit Certificate Actually Proves

A bias audit certificate tells you something about a model as tested, on somebody's data, at a point in time. It rarely tells you what you need. Three gaps recur in every vendor liability hiring tool conversation.

Population. The audit was probably run on the vendor's aggregate client base. An impact ratio computed on 400,000 pooled candidates says little about the 900 who applied to your warehouse reqs in Ohio.

Configuration. Vendors ship defaults; buyers change them. If you raised a cut score, reweighted a competency or turned on a video module, you are not using the audited product. Ask whether the audit covers the configuration you intend to run.

Stage. A pooled audit usually reports one end-to-end figure, and that figure can look acceptable while a step inside it fails badly.

A Pooled Number Can Hide a Failing Stage

Suppose your process runs a resume filter, a cognitive test, then a model-scored structured interview. The overall impact ratio comes back at 0.86 and everyone relaxes. Calculated separately, the filter sits at 0.95, the interview scoring at 0.93 and the cognitive test at 0.61. The favorable stages masked the unfavorable one, and a component with its own adverse effect needs its own justification. Compute selection rates at every stage where candidates are eliminated, including automated reference checks if a reference outcome can end a candidacy. The method is set out in How to Audit Recruitment AI Algorithms for Bias.

The 2026 Federal Shift, Stated Plainly

On June 9, 2026, the Department of Justice's Office of Legal Counsel issued an opinion concluding that the EEOC's disparate-impact guidelines, expressly including the Uniform Guidelines, are unconstitutional and inconsistent with Title VII. It implements Executive Order 14281, and the EEOC has aligned its enforcement posture. Under that framework, ordinary aptitude and knowledge tests would be presumptively job-related.

What did not happen matters more for procurement. The Uniform Guidelines remain codified at 29 CFR Part 1607. An OLC opinion is executive-branch legal advice: it does not bind courts, amend Title VII or overrule Griggs v. Duke Power Co., 401 U.S. 424 (1971). Disparate-impact liability is statutory, at 42 U.S.C. 2000e-2(k), and private plaintiffs keep their own right of action. Fourteen state attorneys general have issued joint guidance affirming disparate-impact enforcement, and California, New York, New Jersey and Illinois have codified protections. Federal exposure has receded; yours has relocated to private claimants and state regulators, not vanished.

The Vendor Question List

Send these before the demo, in writing, and keep the answers. Hedging is itself a data point in any AI hiring vendor evaluation.

• Technical manual - Ask for this: The full technical documentation: constructs measured, scoring model, reliability estimates, intended uses and stated limitations.; Treat as a warning sign: A marketing PDF, or documentation available only under an NDA you cannot share with counsel.
• Validation evidence - Ask for this: Criterion or content validity studies for job families like yours, with sample sizes and outcome measures named.; Treat as a warning sign: "Validated" with no study, or validity borrowed from an unrelated occupation.
• Reference population - Ask for this: Who the norms were built on: industry, level, geography, date range, sample size.; Treat as a warning sign: Norms described only as "global" or refreshed on an unstated schedule.
• Impact results - Ask for this: Selection rates and impact ratios by sex and race or ethnicity on the vendor's own data, per stage, with the cut scores used.; Treat as a warning sign: One aggregate ratio, or a badge with no underlying numbers.
• Support for your audit - Ask for this: Written commitment to export your candidate-level scores and stage outcomes so an independent auditor can run the analysis on your applicant flow.; Treat as a warning sign: Score exports refused, or offered only as a paid add-on at renewal.
• Accommodations - Ask for this: Documented alternative formats, extended time, screen-reader support, and a named path for requests mid-assessment.; Treat as a warning sign: "Contact support" with no documented alternative process.
• Medical and disability inquiries - Ask for this: Confirmation that no item screens for a disability or mental impairment, given ADA limits at 42 U.S.C. 12112(d).; Treat as a warning sign: Personality scales marketed as measuring stability, resilience or wellbeing.
• Biometric collection - Ask for this: A yes or no on face geometry, voiceprints and keystroke dynamics, plus consent and deletion handling.; Treat as a warning sign: Video analysis described only as "engagement signals".
• Retention and deletion - Ask for this: Retention periods per data type, deletion on request, and certified deletion at contract end.; Treat as a warning sign: Indefinite retention, or candidate data reused to train models without a documented basis.
• Subprocessors - Ask for this: A current list, including model providers and hosting regions, with notice before changes.; Treat as a warning sign: A generic reference to "trusted partners".
• Notice generation - Ask for this: Fields you need for jurisdictions with notice duties: what the tool assesses, data types and sources, and timing that supports a 10-business-day notice.; Treat as a warning sign: Instant scoring on submission with no way to hold candidates until notice has run.
• Contract terms - Ask for this: Audit rights, refreshed audits annually, notice on model retraining, and cooperation in litigation or an agency inquiry.; Treat as a warning sign: Broad indemnity running your way, with liability capped at fees paid.

Contract Terms, and the Indemnity That Will Not Save You

Two clauses decide whether your diligence survives the renewal cycle. The first is an audit right: score-level data export, on demand, machine-readable, for the life of the contract and a stated period after termination. Without it, third party assessment risk becomes unmeasurable the moment the relationship sours.

The second is indemnity, and it is weaker than buyers assume. Colorado Senate Bill 26-189 provides that a contract provision purporting to indemnify, defend or hold harmless a party against liability for its own acts or omissions in violation of Colorado anti-discrimination law is void as contrary to public policy, for consequential decisions made on or after January 1, 2027. Even outside Colorado, indemnity is a money transfer after the fact. It does not stop a charge or fix a broken cut score. Negotiate it, then treat it as irrelevant to whether you should buy.

What You Have To Do Yourself

Three things stay with you however good the vendor is. Compute selection rates and impact ratios on your own applicant flow before go-live, and repeat after any model change. Keep applicant-flow and score records; they are the practical defense on job-relatedness and also the evidence base for intentional-discrimination claims the EEOC says it is prioritizing. Document the decision: what you asked, what the vendor answered, why you proceeded.

The same scrutiny applies to tools you already run. If you use AI skill assessments or draw from a skill tests library, ask which tests are eliminating candidates, at what cut score, and whether anyone has checked the numbers since. Righteo supports reference checking and skills assessment; the validation judgment and the impact analysis stay with you and your counsel. Further reading sits in the HR and pre-employment resources library.

Related Resources

• AI Vendor Accountability Recruitment: Your Legal Duty
• How to Audit Recruitment AI Algorithms for Bias
• Righteo AI Assessment: AI Skill Assessments for Hiring Teams
• Righteo Skill Tests Library
• Righteo Automated Reference Checks
• Righteo HR and Pre-Employment Resources and Articles
• Uniform Guidelines on Employee Selection Procedures (29 CFR Part 1607)

Frequently Asked Questions

Can we rely on our vendor's bias audit?

Often yes as a practical matter, but the legal duty stays with you. Under the New York City rules for Local Law 144, an employer may rely on a bias audit conducted with other employers' historical data only if it contributed its own historical data, or has never used the tool. If you have been running the tool on your applicants and your data was not in the audit, you do not have an audit that covers you.

Does an indemnity clause transfer our Title VII exposure?

No. An indemnity allocates money between you and the vendor. It does not make the vendor the respondent to a charge against you, and Colorado voids clauses that shift a party's own liability for algorithmic discrimination in consequential decisions from January 1, 2027.

If federal enforcement of disparate impact has receded, can we stop measuring?

No. The Uniform Guidelines are still codified, disparate-impact liability is written into Title VII by statute, private plaintiffs can sue regardless of agency priorities, and several states have codified their own protections. Validation evidence and applicant-flow records are also what you would need to answer an intentional-discrimination claim.

How often should we recalculate the four-fifths comparison?

Before go-live, at least once a year, and again whenever the vendor retrains the model or you change a cut score. Build the notification duty into the contract so you learn about retraining before candidates are scored by a different model.

This article is general information for US employers and is not legal advice. Confirm your obligations with employment counsel in the states where you hire.

A Bias Badge Is Not a Liability Transfer

A bias badge tells you a vendor did some work on some data, and it is worth having. It is not a transfer of responsibility: under Title VII and the state and city rules written since, the employer that used the tool answers for the outcome. Good assessment vendor due diligence is unglamorous paperwork, named documents, written answers, an export right and a date in the calendar. Send the question list above to every screening vendor you currently pay, and give them two weeks to answer in writing.