The best AI recruiting tool is the one whose bias audit you can rerun on your own applicants, because under New York City's Local Law 144 the audit is your legal duty. Your vendor's pooled audit usually does not cover you. On that test, HireVue's structured interview assessments are our pick for professional and early-career hiring: the method has the strongest validity evidence of any selection procedure, and its audits, run by an industrial-organizational psychology firm, publish the bad numbers alongside the good ones.
If you already run Workday Recruiting, HiredScore is the practical choice, provided you audit it on your own data the way Pfizer did. Paradox earns its place for frontline scheduling and should stay away from rejection decisions until you have audited it yourself. Eightfold comes last for external screening right now: its newest audit rests on synthetic interviews, and a pending class action goes after the data its scores are built from.
The workload behind this comparison: about 40,000 applications a year for roles tied to a New York City office, a mix of professional and hourly roles, an existing applicant tracking system, and some hiring in the EU by 2028. Prices and public documents were checked on each vendor's site on October 28, 2026.
| Tool | What it scores | Latest public LL144 audit we found | Lowest published impact ratio | Litigation on record | Price (checked Oct 28, 2026) | Skip it if |
|---|---|---|---|---|---|---|
| HireVue assessments | Structured interview answers, tiered top, middle, bottom | DCI Consulting, July 2024, pooled across employers | 0.31 (Asian men, top tier, Relationship Building, 53 applicants) | 2025 ACLU agency complaint over Intuit's hiring; HireVue denies its AI was used | Not published; demo request | You hire hourly staff who will not sit a video interview |
| Workday HiredScore | A to D grade of match to the requisition | Pfizer's own-data audit, 2026 | 0.822 (Hispanic men, grade A, Pfizer) | Named in Mobley v. Workday class certification motion | Not published; sold through Workday | You are not on Workday or will not audit your own data yearly |
| Paradox (Workday) | Conversational screening and scheduling | None linked from its ethics page | None published | None found in the filings we reviewed | Not published; demo request | You need a published audit before you sign |
| Eightfold AI | 0 to 5 match score; LLM-scored AI interviews | BABL AI, March 2025 (matching) and June 2026 (interviewer) | 0.907 (Asian women, matching model) | Kistler v. Eightfold AI, FCRA class action | Not published; "Book a demo" | Your legal team cannot carry consumer-report risk this year |
Who Owns the Bias Audit Under Local Law 144?
You do. The city's DCWP FAQ states that "the vendor that created the AEDT is not responsible for a bias audit of the tool," and that employers "are ultimately responsible for ensuring a bias audit was done before using an AEDT."
An automated employment decision tool (AEDT) is software that uses machine learning, statistics or AI to substantially assist or replace a hiring or promotion decision, and screening counts. The audit must be repeated every year, and the summary you post must give the audit date, the source of the data, the count of people in unknown categories, and the selection or scoring rates and impact ratios for every sex, race and intersectional group. An impact ratio is one group's selection rate divided by the rate of the most-selected group; a ratio under 0.80 fails the traditional four-fifths rule of thumb, though the law itself requires no action on any result.
The clause most buyers miss is the one on pooled data. The same FAQ says you can rely on an audit built from several employers' data only if you gave your own historical data to the independent auditor, or if this is your first use of the tool. After your first year, a vendor's pooled summary stops covering you unless your data is in it.
City enforcement has been thin. A December 2025 state comptroller audit found DCWP had flagged one likely violation across 32 employer and vendor websites where the comptroller's auditors found 17, and only 3 of 12 test complaints placed through 311 reached DCWP. Weak enforcement does not lower your exposure much, because the larger risk sits in discrimination litigation, where an audit on your own pool is the evidence you will want.
HireVue: The Strongest Evidence, Including the Uncomfortable Rows
HireVue wins because it scores a structured interview, which is the selection method with the best evidence behind it, and because its published audits show where it falls short. A structured interview asks every candidate the same job-related questions and scores the answers against a fixed rubric.
The research case is Sackett, Zhang, Berry and Lievens, the 2022 re-analysis of personnel selection meta-analyses. It concluded that earlier corrections had overstated validity for most procedures, and structured interviews came out at the top of the revised ranking. That is the method HireVue sells. HireVue also stopped using candidates' facial expressions as a scoring factor, as Fortune reported in January 2021, so the score now comes from what the candidate says.
Read its audits the way a plaintiff's expert would. The DCI Consulting audit of HireVue's Adaptability assessment for intern and new-graduate jobs, produced July 5, 2024 and posted by a retail customer, pools nationwide applicants from 2022 to 2024 across employers. At the top-tier threshold, Asian applicants come in at 0.79, applicants of two or more races at 0.77, and Asian men at 0.74. The same document warns that its threshold-based rates "may or may not reflect employer-specific operational use" of the tool. That candour is the reason to prefer HireVue: you can see the risk before you buy it, and DCI's people are industrial-organizational psychologists who also do validation and expert-witness work.
HireVue has its own accommodation dispute on record. In March 2025 the ACLU filed complaints with the Colorado Civil Rights Division and the EEOC on behalf of a Deaf Indigenous employee denied a promotion at Intuit, alleging the video interview platform performs worse for non-white and deaf speakers and that her request for human captioning was refused. HireVue's CEO responded that Intuit "did not use a HireVue AI-based assessment." Whatever the outcome, the case shows where your exposure sits: in the accommodation path you run around the tool.
Who should not buy it: high-volume hourly employers whose applicants will not sit through a recorded interview, and buyers who want a price list. HireVue's pricing page lists no figures and routes you to a demo.
Workday HiredScore: Defensible Only With Your Own Audit
HiredScore is the right choice if Workday Recruiting is already your system of record, on one condition: you commission an audit on your own applicant data every year. Pfizer's posted audit is the template. It covers every applicant to requisitions with New York City hires from August 2022 to February 2025: 40,821 applicants by sex, with 2,953 of unknown sex or race.
What the tool scores is narrow, and Pfizer's summary says so plainly: the A to D grade "merely helps recruiters and hiring managers expediently identify which applicants have stated that they meet which requirements of the jobs." That is a match score. It measures overlap between an application and a requisition, which is a different quantity from predicted job performance, and neither Workday nor Pfizer publishes a criterion validity study for it. Workday's HiredScore datasheet adds that the model does not process social media or web data and that race and gender are used only to balance training sets and for testing. Those are vendor claims.
Pfizer's own-pool numbers show why your audit matters. Every group cleared 0.80, but Hispanic men reached grade A at a ratio of 0.822 and men of two or more races at 0.834. A different employer's requisitions will produce different numbers.
The litigation is real. In Mobley v. Workday, the court held in July 2024 that a software vendor acting as an employer's agent "may be independently liable as an employer." In May 2025 Judge Rita Lin conditionally certified a nationwide collective of applicants aged 40 and over. In September 2026 the plaintiffs moved for class certification with race, sex, age and disability subclasses covering HiredScore Spotlight, Fetch and Candidate Skills Match, alleging rejection disparities above 18 standard deviations for Black applicants. Workday disputes the claims and nothing has been decided on the merits. Note what Local Law 144 does not test: age. The Mobley collective is an age claim.
Who should not buy it: employers outside Workday, and anyone who will rely on the vendor's word in place of an annual audit on their own applicants.
Paradox: Buy It for Scheduling, Audit Before It Screens
Paradox, the conversational assistant known as Olivia, belongs in frontline hiring for scheduling and intake, and any automated knockout it applies should be treated as an AEDT and audited before go-live. Workday completed its acquisition of Paradox on October 1, 2025, so Workday now sells both a match scorer and a conversational screener.
The transparency gap is the problem. Paradox's ethical AI page, checked October 28, 2026, says the company "regularly conducts bias evaluations and reviews of its AI systems" and points to Workday resources, but links no Local Law 144 summary. Its pricing page publishes no figures.
Who should not buy it: anyone who needs a vendor-published audit in hand before contract signature, or who plans to let chat answers reject candidates without a human reading them.
Eightfold: The Loser for External Screening This Year
Eightfold finishes last because its two audits are weaker than they look and its core data model is under legal attack.
The March 2025 BABL AI audit of the Eightfold Matching Model looks clean: the model produces a 0 to 5 match score in half-point steps, and the lowest impact ratio is 0.907 for non-Hispanic Asian women. But it is pooled across Eightfold customers, counts applications rather than people, and excludes 50,852,137 applications with an unknown sex or race. The auditor also states it makes no determination whether the model is an AEDT at all.
The June 2026 audit of Eightfold's AI Interviewer is thinner. An LLM writes the questions, another LLM scores the transcript on a 0 to 5 rubric, and hiring managers see only the overall score. The sex analysis uses 1,250 men and 395 women from real use, with 1,235 more of unknown sex. The race and intersectional analysis uses synthetic interviews in which "an LLM was instructed to act as a candidate," with demographics signalled through generated names alone, 238 per race group. BABL flags that this may not match real interviews. To its credit, the audit tests age, which none of the other audits reviewed here does.
Then there is Kistler v. Eightfold AI, a proposed class action filed January 20, 2026 under the Fair Credit Reporting Act and California's ICRAA. The complaint alleges Eightfold builds applicant profiles from third-party data and produces "likelihood of success" scores that amount to consumer reports. Towards Justice announced the suit on behalf of two job applicants with STEM backgrounds. These are allegations, and no ruling on the claims had been reported in the coverage we checked on October 28, 2026. If the theory survives, employers using the scores could face FCRA disclosure and adverse-action duties.
Who should not buy it: anyone screening external applicants who cannot absorb consumer-report risk while Kistler is pending. If you already own Eightfold for talent intelligence, keep its match score out of external rejection decisions until the court rules. Eightfold publishes no price; its site offers "Book a demo."
What Does the EU AI Act Add for Hiring Tools?
The EU AI Act classifies recruitment AI as high-risk, and the obligations fall on you as the deployer from December 2, 2027. Annex III, point 4 covers systems used to place targeted job ads, analyse and filter applications, and evaluate candidates. The Digital Omnibus moved that date from August 2, 2026.
Article 26 requires you to assign human oversight to people with the competence and authority to overrule the tool, keep logs for at least six months, tell workers' representatives and affected workers before putting the system into use at work, and inform candidates that a high-risk system is helping decide about them. If you control the input data, it must be relevant and sufficiently representative for the purpose. A vendor's conformity paperwork does not discharge any of these.
Notice, Opt-Out and Accommodation: What Candidates Must Get
In New York City, candidates must get notice 10 business days before the tool is used, naming the job qualifications and characteristics it will assess and giving instructions for requesting a reasonable accommodation, per the DCWP FAQ. A notice on the employment section of your website satisfies it for applicants.
Video interviews carry an extra layer in Illinois. The Artificial Intelligence Video Interview Act requires you to notify applicants that AI may analyse the interview, explain how it works and what it evaluates, and obtain consent before the interview. Build the accommodation path into the workflow itself: a named recruiter who can route a candidate to a live interview, and a log that records each request.
The Audit Your Vendor Sends You Does Not Cover Your Applicant Pool
A vendor's pooled audit describes the tool across its customers and says nothing about your requisitions, your score thresholds or your applicant mix. HireVue's DCI report says as much about thresholds; Eightfold's matching audit runs on tens of millions of other employers' applications; and of the audits reviewed here, only Pfizer's HiredScore summary measures a single employer's own applicants.
Researchers who reviewed public Local Law 144 audits reached the same view. A FAccT 2025 paper found the audits hampered by missing demographic data, opaque aggregation, problematic test data and metrics that do not match how the tools are used. Our earlier piece on choosing a Local Law 144 auditor covers who can run your own-data audit.
How to Decide: The Criteria That Predict Regret
Four questions predict whether you will regret the purchase.
- Can you export per-candidate scores with your own demographic data attached? If the answer is no, you cannot audit, and you are relying on someone else's pool. Make the export a contract term.
- Does the vendor publish a criterion validity study for the score you will act on? HireVue's method has the research behind it. Match scores from HiredScore and Eightfold measure requisition fit; ask for the study, and if none exists, keep the score as a sort order and leave rejection to a person.
- Does any audit test age and disability? Local Law 144 requires only sex and race. Mobley includes age and disability claims.
- Does the vendor's data come only from the application? HiredScore says yes. The Kistler complaint alleges Eightfold also uses third-party data.
The answer changes in two cases. If you are entirely hourly and frontline, conversational scheduling matters more than interview validity, and Paradox moves up provided a human makes every rejection. If the court dismisses Kistler, Eightfold's matching model becomes a reasonable option for employers who also want internal mobility from one vendor.
This Week:
- List every tool in your hiring flow that scores, ranks or knocks out candidates, and mark which ones touch New York City roles.
- Pull the posted audit summary for each and check whether your own data went into it.
This Month:
- Ask each vendor for a per-candidate score export and its validity study, in writing.
- Engage an independent auditor to run your first own-data audit, including age.
Before Your Next Renewal:
- Put the score export, annual audit cooperation and notice of any lawsuit naming the product into the contract.
- Map Article 26 duties to named owners if you will hire in the EU after December 2, 2027.
The Bottom Line
AI hiring tools are where pre-employment testing was in the 1970s, when the four-fifths rule and validation studies became the way employers defended a test in court. The same two questions apply now: does the score predict the job, and what does it do to your own applicants? Buy the tool you can audit, start with HireVue if your roles suit a structured interview, and run the first own-data audit before your current one expires.
Continue Reading
- DCI Consulting Group Tests Employers' Own Data for NYC AI Audits
- Dayforce Agrees to Acquire Ribbon AI for Agentic Recruiting
- Claude Screening of Candidates Needs Review Under Anthropic Policy
- Human-in-the-Loop for AI Agents: Most Approval Gates Rubber-Stamp
- Workday Rising 2026 Sells Agents on Credits With No Published Price
- EU AI Act Governance Tools: Buy Inventory, Not Policy Packs