Build an AI Resume Screener That HR Will Trust

Almost every AI resume screener I have seen in the wild gets ignored, overridden, or quietly switched off within six months. The pattern is consistent. The vendor demos a ranked list, the head of talent signs the contract, and three months later the recruiters are back in the ATS keyword search because the AI scored a candidate they loved as a 3.2 out of 10 and they cannot tell why. The bot was a black box. Nobody trusted it. So nobody used it.

This post is the architecture we built for Xandidate - a rubric-grounded, explainable, override-friendly AI ATS - and the same one I now recommend on every HR-tech build. The thesis is simple: AI does not earn trust by being right more often. It earns trust by being legible, correctable, and accountable. Get the trust architecture right and recruiters will use the tool. Get it wrong and the most accurate model in the world will sit idle.

Why AI screeners get ignored or overridden

Three failure modes account for almost every dead AI screener I have seen audited. None of them are about model quality. They are about how the system communicates with the human who has to use it.

One: the black box rank. The screener outputs a single score from 0 to 100 with no per-criterion breakdown and no evidence. A recruiter looks at a candidate she knows from a referral, sees a 41, and has no idea whether the model penalised the candidate for missing a buzzword, for graduating from the wrong school, or for a layout quirk in the PDF. She cannot trust the score because she cannot trust the reasoning. So she ranks by hand and the AI becomes decoration.

Two: the bias fear. Every recruiter has read about Amazon's scrapped resume screener that learned to downrank women from its own historical hiring data. Most enterprise HR leaders now treat AI screeners with the suspicion that protein bars deserve at a gas station. Without an explicit story for how the system was designed to reduce bias - blind redaction, explicit rubric, bias audit reports - the legal team will block the rollout regardless of what the model can do.

Three: bad UX for the override case. Even the few screeners that show per-criterion scores often make overrides painful - a separate admin screen, no audit trail, scores that silently snap back on the next sync. Recruiters override constantly in the early weeks of any deployment; if the override path is clumsy, they switch back to the manual workflow inside a month and the AI investment is dead.

The fix for all three is the same architectural commitment: rubric-grounded scoring with per-criterion justifications, a first-class recruiter override flow, an audit log that survives both EEOC discovery and a GDPR data-subject request, and a deliberate blind-redaction stage to suppress the most obvious bias proxies. Build for trust first, accuracy second. The accuracy improvements are easy to land once recruiters are actually using the tool.

The trust architecture

The system has five visible layers and one invisible one. The visible layers are what the recruiter and the candidate interact with. The invisible layer - the audit log - is what the compliance team and the courts care about.

Rubric. Every job has an explicit rubric of criteria with weights. Recruiters draft it, the AI helps. The rubric is the contract between the screener and the recruiter - the screener only scores against criteria in the rubric and never invents new ones at inference time. The rubric is versioned. Every scored candidate carries a reference to the rubric version that scored them.

Structured extraction. Every PDF resume becomes a clean candidate record - work history, skills, education, projects, contact info - using a vision-capable model that handles both text-PDFs and image-PDFs. The extracted record is what gets stored and scored. The original PDF is kept as evidence.

Per-criterion scoring with justifications. For each criterion in the rubric, the model produces a 0 to 5 score plus a one-sentence justification that quotes evidence from the candidate record. A weighted sum produces the headline rank, but the recruiter always sees the breakdown - and can drill into the justification for any criterion to see the source quote.

Recruiter override. Every per-criterion score is editable from the candidate detail view. The override updates the rank in real time, logs the original AI value, the new value, the recruiter, the timestamp, and an optional reason. Overrides are first-class data, not exceptions.

Audit log. Every action - score generation, view, override, comment, advance, reject - is logged with an immutable timestamp and user ID. The log is queryable per-candidate (for GDPR data-subject requests) and per-decision (for EEOC adverse impact analysis). This is what makes the system shippable inside a regulated HR function.

Stack

The stack we use on Xandidate is boring on purpose. The novel parts belong in the rubric and the UI; the infrastructure should be the most predictable thing in the system. Next.js 15 app router and Server Actions for the recruiter dashboard. The Vercel AI SDK for model calls because the provider-swap story is the cleanest in the ecosystem. OpenAI structured outputs (and Anthropic JSON mode as fallback) for every model call that produces data. Postgres for the candidate records, audit log, and rubrics, with pgvector for skill embedding matching when a job has fuzzy skill requirements. Vercel Blob for the original PDFs.

The model choices are intentional. A vision-capable strong model (GPT-5 or Claude Sonnet 4.6) for the PDF-to-record extraction because layout matters and the cost is amortised over the candidate's whole pipeline life. A strong model again for the per-criterion scoring because the justifications are what the recruiter reads. A cheap fast model (GPT-5-mini or Claude Haiku 4.5) for the reflection check that runs after scoring to catch obvious mistakes. The cost math is in the FAQ at the bottom - the TL;DR is that even at the strong end the total per-candidate spend is rounding error against the recruiter time saved.

Step 1: Job spec to structured rubric

Most teams skip the rubric step and try to score candidates from a job description directly. This is the single biggest reason AI screeners feel arbitrary - a job description is a marketing document, not a scoring schema. The first job of the system is to help the recruiter turn the job description into an explicit rubric, then store it as structured data.

The recruiter pastes or attaches the job description. The AI proposes a draft rubric - three or four must-have criteria, three or four should-haves, two or three nice-to-haves - each with a suggested weight and a one-line description. The recruiter edits freely. The final rubric is the contract.

// src/rubric.ts
import { generateObject } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

const criterion = z.object({
  id: z.string(),
  name: z.string(),
  description: z.string(),
  tier: z.enum(["must", "should", "nice"]),
  weight: z.number().min(0).max(1),
});

const rubricSchema = z.object({
  role: z.string(),
  seniority: z.enum(["junior", "mid", "senior", "staff", "principal"]),
  criteria: z.array(criterion).min(4).max(12),
});

export async function draftRubric(jobDescription: string) {
  const { object } = await generateObject({
    model: openai("gpt-5"),
    schema: rubricSchema,
    system: `You design hiring rubrics. Convert the job description into
4-12 scorable criteria. Each criterion must be specific, evidence-able
from a resume, and free of protected-attribute proxies (no school
prestige, no age signals). Weights across all criteria must sum to 1.0.`,
    prompt: jobDescription,
  });
  return object;
}

The schema is the guardrail. The model cannot return more than 12 criteria, cannot return fewer than 4, must classify each criterion as must, should, or nice, and must produce weights between 0 and 1. The system prompt explicitly bans the model from inventing criteria that proxy for protected attributes. The recruiter still reviews and edits - the AI is a drafting assistant for the rubric, not the author.

Step 2: PDF to structured candidate record

Resume extraction is where most pipelines silently fail. PDFs are a hostile format - text-PDFs with clean layout, text-PDFs with two-column layout that confuses naive parsers, image-PDFs that need OCR, and the occasional photo of a printed resume. A vision-capable model handles all four uniformly. The cost is roughly $0.01 per resume and the resulting structured record is used downstream for scoring, search, and skill-matching.

The schema is the spec for what counts as a candidate. Anything not in the schema is dropped - which is also the first redaction step, because the schema omits photo, date of birth, and gender markers even if they appear on the resume.

// src/extract.ts
import { generateObject } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { z } from "zod";

const role = z.object({
  company: z.string(),
  title: z.string(),
  startDate: z.string(),
  endDate: z.string().nullable(),
  description: z.string(),
  tech: z.array(z.string()),
});

const candidateSchema = z.object({
  fullName: z.string(),
  email: z.string().nullable(),
  phone: z.string().nullable(),
  location: z.string().nullable(),
  links: z.array(z.string()),
  summary: z.string(),
  roles: z.array(role),
  education: z.array(
    z.object({
      institution: z.string(),
      degree: z.string(),
      graduationYear: z.number().nullable(),
    })
  ),
  skills: z.array(z.string()),
  projects: z.array(z.object({ name: z.string(), description: z.string() })),
});

export async function extractResume(pdfUrl: string) {
  const { object } = await generateObject({
    model: anthropic("claude-sonnet-4-6"),
    schema: candidateSchema,
    system: `Extract the candidate record from this resume PDF. Quote
verbatim from the source wherever possible. Do not infer skills,
job titles, or dates that are not explicitly present.`,
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Extract this resume." },
          { type: "file", data: pdfUrl, mimeType: "application/pdf" },
        ],
      },
    ],
  });
  return object;
}

The full PDF extraction story - including OCR fallbacks, the reflection step that catches hallucinated work history, and the validation layer that flags suspicious dates - is the same one I walk through in my AI document extraction post. The patterns transfer directly.

Step 3: Rubric scoring with explanations

This is the heart of the system. The model receives the rubric, the candidate record (post-redaction), and a structured-output schema that forces it to produce a 0 to 5 score plus a one-sentence justification for every criterion. The justification must quote or paraphrase evidence from the candidate record. If the record contains no evidence for a criterion, the model must return a score of 0 with a justification that says so explicitly.

// src/score.ts
import { generateObject } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

const criterionScore = z.object({
  criterionId: z.string(),
  score: z.number().int().min(0).max(5),
  justification: z.string(),
  evidence: z.string().nullable(),
});

const scoreSchema = z.object({
  scores: z.array(criterionScore),
  redFlags: z.array(z.string()),
});

const SYSTEM = `You score candidates against a hiring rubric. For each
criterion: assign 0-5, explain in one sentence, and quote the evidence
from the candidate record. If there is no evidence, score 0 and say so.
Never invent experience. Never reward or penalize based on name, school
prestige, photo, age, gender, or any protected attribute.`;

export async function scoreCandidate(
  rubric: Rubric,
  candidate: CandidateRecord
) {
  const { object } = await generateObject({
    model: openai("gpt-5"),
    schema: scoreSchema,
    system: SYSTEM,
    prompt: `Rubric:\n${JSON.stringify(rubric, null, 2)}\n\n
Candidate (redacted):\n${JSON.stringify(redact(candidate), null, 2)}`,
  });
  return object;
}

The schema is doing real work. Without it, the model would happily produce holistic prose that recruiters would have to interpret. With it, every score is structured data - directly renderable in the UI, directly queryable for bias audits, directly overridable in one click. The structured-output guarantee was the single biggest production unlock for the screener; the full mechanics are in my OpenAI structured outputs guide.

Step 4: Final rank with the breakdown visible

The headline rank is a weighted sum of the per-criterion scores, normalised against the maximum possible. But the rank is not the UI. The UI is the breakdown - every candidate detail view shows the criterion list, the AI score per criterion, the recruiter override slot (empty by default), and the justification with evidence. The headline rank is one number near the top, but the recruiter never has to take it on faith.

// src/rank.ts
export function rank(
  rubric: Rubric,
  scoreResult: ScoreResult,
  overrides: Record<string, number> = {}
) {
  const maxScore = 5;
  const total = rubric.criteria.reduce((sum, c) => {
    const aiScore = scoreResult.scores.find((s) => s.criterionId === c.id)?.score ?? 0;
    const effective = overrides[c.id] ?? aiScore;
    return sum + (effective / maxScore) * c.weight;
  }, 0);
  return Math.round(total * 100);
}

The override-aware rank is the contract with the recruiter. When she edits a score, the rank recomputes immediately. There is no magic. The system is doing arithmetic on her inputs; the AI just produced the defaults. This is what turns recruiters from skeptics into power users in the first week.

Step 5: Recruiter override and audit log

Every state change in the system writes to an append-only audit log. Score generation logs the rubric version, the candidate record version, the model, and the full score output. Each override logs the criterion, the original score, the new score, the recruiter, the timestamp, and an optional reason. Each candidate view logs the recruiter and timestamp. Each advance, reject, or comment logs the action with the recruiter and any notes.

// src/audit.ts
import { db } from "./db.js";

type AuditEvent =
  | { type: "score_generated"; rubricVersion: string; modelId: string; output: any }
  | { type: "score_overridden"; criterionId: string; from: number; to: number; reason?: string }
  | { type: "candidate_viewed" }
  | { type: "candidate_advanced"; stage: string }
  | { type: "candidate_rejected"; reason?: string }
  | { type: "comment_added"; body: string };

export async function audit(
  candidateId: string,
  userId: string,
  event: AuditEvent
) {
  await db.auditLog.create({
    data: {
      candidateId,
      userId,
      type: event.type,
      payload: event,
      timestamp: new Date(),
    },
  });
}

The log is queryable in two directions. Per-candidate, for GDPR data-subject requests - a candidate can request the full history of how their application was evaluated and the system can produce it in seconds. Per-decision, for EEOC adverse-impact monitoring - the compliance team can run quarterly reports that look at rejection patterns across protected groups and surface anomalies before they become lawsuits. The audit log is also what makes the system pattern-match to human-in-the-loop AI properly - the human is in the decision path, and the path is provable.

Bias mitigation: what works and what doesn't

Bias mitigation in AI screening is a domain where vendor marketing sells confidence that the underlying methods do not deserve. Here is the honest taxonomy from the deployments I have audited.

What works. Blind redaction of name, photo, age, date of birth, graduation year, and school name at the scoring stage - these are the highest-signal proxies for protected attributes and removing them measurably reduces disparate impact. Rubric-grounded scoring with per-criterion justifications, because explicit criteria make it possible to detect when the model is being inconsistent across demographic groups. Quarterly bias audits that compare advance-rates across protected groups using the four-fifths rule. A clear override path so that recruiters can correct AI mistakes without changing the underlying model.

What doesn't work. Asking the model to "be unbiased" in the prompt - this is theatre. Synthesising fake demographic data to train against - does not generalise. Using a single fairness metric to certify the system - different metrics conflict and gaming one usually breaks another. Treating bias mitigation as a one-time launch checklist instead of a quarterly audit cadence - every model upgrade, rubric change, and KB refresh can reintroduce drift.

The honest stance is that bias mitigation is harm reduction, not elimination. The goal is to ship a system that is measurably less biased than the human-only baseline, monitor continuously, and keep the human in the decision path. Promising more than that is what gets HR-tech vendors sued.

EEOC and GDPR compliance, in plain terms

Two regulatory regimes matter for AI screening in 2026. In the US, the EEOC's guidance on AI in employment makes clear that employers are liable for discriminatory outcomes regardless of whether the decision was made by a human or a model. NYC Local Law 144 adds an annual bias audit requirement and candidate-notice requirement for any automated employment decision tool used to substantially assist or replace discretionary decision-making.

In the EU, GDPR Article 22 gives candidates the right to not be subject to decisions based solely on automated processing that produce legal or similarly significant effects. Employment screening qualifies. The practical implication is that a human must materially review every adverse decision, candidates must be informed that AI is used, and they have a right to obtain a meaningful explanation of how a decision was reached.

The architecture in this post is designed to satisfy both. The recruiter override flow and the audit log together prove the human is in the decision path. The per-criterion justifications provide the right-to-explanation answer. The rubric versioning and the audit log let the compliance team produce the annual bias audit. None of this is bolted on - it is the structure of the system from day one.

UX patterns that build trust

The interface is where the trust architecture either lands or falls apart. A handful of UX patterns have done most of the work on the deployments I have shipped.

Confidence badges. Every score carries a confidence indicator - high when the evidence quote is verbatim and the model produced a low-variance answer across two samples, medium when one of those is missing, low when both are. Low- confidence scores get a different visual treatment that nudges the recruiter to review. The recruiter learns quickly that the AI is honest about when it is uncertain.

Side-by-side comparisons. When a recruiter wants to compare two candidates on a specific criterion, the system shows the two justifications and evidence quotes side by side. This is the highest-frequency interaction in the tool and it outperforms a ranked list for actual hiring decisions.

Recruiter notes that re-score. Recruiter comments are first-class evidence. When a recruiter adds a note like "great culture fit signal in this side project," the system can be configured to re-run scoring with the note included as additional context. This makes the recruiter feel like a co-author of the score rather than a reviewer of it.

Score history. Every score has a visible history - the original AI score, every override, every reason. Recruiters can see how a colleague evaluated the same candidate. Hiring managers can see why a recruiter advanced or rejected. The history turns hiring from a series of opaque decisions into a transparent, reviewable record.

Real numbers from Xandidate

Numbers from the Xandidate production deployment, aggregated across the first cohort of customers, six months in production, roughly 280,000 candidate screens processed.

The override rate is the most interesting number to share with recruiter teams during sales conversations - it is high enough to prove the system is not trying to replace them, and low enough to prove the AI is doing real work. The disparate impact improvement is the number compliance officers want to see; crossing the four-fifths threshold is what unlocks adoption inside enterprise HR. The false-reject rate measurement requires a recruiter to re-review a sampled subset of rejected candidates monthly - expensive to maintain, but it is the single most credible quality signal we produce.

What I would change today

Three things I would build differently if I were starting Xandidate over. First, treat the rubric as a versioned, branching document from day one - we ended up retrofitting versioning after a customer changed their rubric mid-pipeline and lost the ability to compare candidates fairly. Cheap to build in upfront, painful to retrofit.

Second, build the bias audit dashboard before the scoring is production-ready, not after. The dashboard forces you to define your fairness metrics, your reference groups, and your reporting cadence in concrete terms - which surfaces architectural decisions that should be made before launch, not patched after a customer escalates.

Third, push harder on the recruiter-as-coauthor model. The comment-re-scores-the-candidate feature was the highest-impact UX choice we shipped, and we shipped it in month four. It should have been in the first private beta. Anything that makes the recruiter feel ownership of the score makes the override rate more honest and the system more trustworthy. Most AI screeners treat the recruiter as a reviewer; the ones that win treat the recruiter as a co-author. The same principle drives every successful human-in-the-loop deployment I have audited.

Where this fits in your stack

If you are scoping an AI screener - as a standalone tool or as a layer inside an existing ATS - the architecture above is the starting point. The rubric design and the per-criterion scoring are non-negotiable. The audit log and override flow are the compliance backbone. The blind redaction is the bias floor. Everything else - model choice, vector index for skill matching, UI polish, integrations into Greenhouse or Lever or Workable - is negotiable and can be staged.

If you need a senior engineer who has shipped this exact system end to end, my AI integration and AI agent development practices cover exactly this scope, and the underlying retrieval and reasoning patterns are the same ones in my RAG architecture tutorial. Cost mechanics for the model spend are in my OpenAI API cost breakdown. You can also hire an AI developer in Kosovo directly. Same person who built Xandidate - the AI ATS this entire architecture came from.

Frequently asked questions