What AI Job Interviews Reveal About AI Evaluation

An industry AI interview grader was recently put to a simple test: could it tell when a job candidate was stretching the truth? It couldn't. A new University of Georgia study found the AI scored exaggerators just as highly as honest applicants, while human reviewers spotted the difference. That's a hiring story on its surface, but for anyone building AI systems that evaluate, rank, or judge, it's a sharper lesson about how automated evaluators fail. This piece unpacks what the study found and what it means if you rely on AI to score anything.

Key Takeaways

• A 2026 University of Georgia study, published in Information Systems Research, found job seekers exaggerated more when they knew an AI, not a human, would grade their interview.
• The industry AI grader didn't penalize the truth-stretchers; it scored them as highly as honest candidates with the same qualifications.
• Human evaluators generally caught the embellishment and rated authentic candidates higher, a gap between machine and human judgment.
• Explaining what the AI assessed made candidates behave as authentically as those who thought a human was reviewing them.
• For builders, it's a case study in automated evaluators being gamed, and why transparency, testing, and human review matter for any AI that scores.

What Did the AI Interview Study Find?

A new University of Georgia study on AI job interviews carries a wider lesson about AI evaluation. When candidates knew an AI, not a human, would grade them, they exaggerated more, and the AI evaluator scored the truth-stretchers just as high as honest candidates. Humans caught the deception the AI missed. But when the study explained what the AI was assessing, candidates behaved authentically again. For anyone building AI evaluation systems, that's three warnings and one fix.

The research, led by Akshat Lakhiwal of UGA's Terry College of Business with co-authors at Arizona State, Indiana University, and National Taiwan Normal University, studied hundreds of online job seekers in one-way video interviews. It appeared in Information Systems Research in 2026 under the title "From Opacity to Transparency." In the one-way format, a candidate records answers to standard prompts like "Why should we hire you?" Increasingly an AI system, not a person, grades and sorts those responses.

The behavioral finding was clear. Told their video would be judged by an AI, applicants reported, and then displayed, a marked increase in "exaggerations" and "deceptive embellishments." As Lakhiwal described it to reporters, people "seemed to be throwing the kitchen sink at the situation" to give the evaluator what they imagined it wanted. Not knowing how the system worked, they gamed toward a guess. Interestingly, when asked whether embellishing was ethical, many said it felt necessary to perform in an unpredictable setting, not that they set out to deceive.

The Study's Findings at a Glance

The contrast between how the AI and humans judged the same videos is the part worth staring at. Here it is compressed.

Condition	What happened
Candidates told AI would grade them	More exaggeration and deceptive embellishment
AI grading the exaggerators	Scored them as highly as honest candidates
Humans grading the same videos	Caught embellishment; rated authentic candidates higher
Candidates told what the AI assessed	Behaved as authentically as with a human reviewer

Read the middle two rows together. The same behavior, exaggeration, earned a high score from the machine and a low one from a person. The evaluator you choose doesn't just measure the candidate; it changes what "a good answer" even means, and it can reward the wrong thing without anyone noticing. That last part is the quiet danger: the score looks fine while the thing underneath it rots.

Why AI Evaluation Has Blind Spots

Because an automated evaluator scores the signals it can see, and confident exaggeration produces exactly those signals. The AI grader in the study weighed things like facial expressions, verbal sentiment, and keywords, all of which a polished embellishment supplies in abundance.

That's the core failure mode of automated evaluation: it optimizes for measurable proxies, not the underlying truth. A candidate who "throws the kitchen sink" hits the keywords, projects confidence, and sounds enthusiastic, so the model rates the answer well, whether or not it's accurate. Humans, reading context and subtler cues, discounted the same performance. The machine graded the surface; the people graded the substance.

It's worth being precise about what the humans did better. They weren't smarter about the job requirements. They were better at reading authenticity. A rehearsed, inflated answer has a tell that a person picks up and a keyword-and-sentiment model does not. That gap is small in a low-stakes quiz and large in a hiring decision. The more the outcome depends on judging a human, the wider it gets.

The honest read: this generalizes far beyond hiring, and it should worry anyone using AI to judge. The same dynamic shows up in content scoring, automated moderation, and especially LLM-as-judge setups where one model grades another's output. If your evaluator rewards visible proxies, then whatever it's scoring, a person or another model, will drift toward those proxies, and your scores will look great while measuring less and less. We treat this as a first-class risk in AI agent evaluation, because a gamed metric is worse than no metric: it's a false sense of safety.

How Transparency Improves AI Evaluation

By turning a guessing game into a known task, which calms the behavior that distorts the result. This is the study's most counterintuitive and useful finding.

Companies have traditionally hidden how their hiring evaluations work, on the theory that transparency lets applicants game the system. The UGA team found the reverse. When one group was told exactly what the AI would assess, its criteria and the dimensions it rated, teamwork, job-related abilities, work style, and personality, those candidates behaved as authentically as people who believed a human was watching. As Lakhiwal noted, "telling applicants more about the process allows them to be more authentic."

The honest read: the anxiety of being judged by an opaque system is itself a source of bad data. People embellish because they don't know the rules, not always because they intend to cheat. Explaining the process, not the model internals, removes the incentive to guess and game. For any AI system that evaluates people, that's a design principle: opacity invites gaming, clarity invites honesty. It's the same reason we build understandable checks into conversational AI platform evaluation rather than hiding the rubric.

There's a useful distinction hidden in the finding, and it's worth stating plainly. Transparency about the "what" is not the same as transparency about the "how." The study didn't hand candidates the model weights or the exact scoring formula. It told them what the system was looking for. That's enough to make sense of the task, which is all a person needs to stop guessing. You can keep your implementation private and still tell people, and other systems, what you actually measure.

Lakhiwal made the point directly: a candidate doesn't need to know which model analyzes the video, only to understand the process the way they'd understand a human interview. That framing scales to any evaluator. Publish the rubric, not the source code. The gaming companies fear comes from secrecy, not from clarity, and the study is evidence that clarity is the safer default.

Where Else Does This Failure Show Up?

Almost anywhere an AI scores something a person or model can adapt to. The interview is just a vivid case; the pattern is everywhere in modern AI systems. Once you see it, you spot it in your own stack.

Here are common places the same blind spot appears:

• LLM-as-judge. One model grades another's output. If the judge rewards fluent, confident, keyword-rich answers, the graded model learns to produce those, whether or not they're correct.
• Content and SEO scoring. A tool that rates "quality" by surface signals gets gamed by writing to the signals, not the reader.
• Automated moderation. Systems that flag on visible markers miss cleverly phrased violations and over-flag innocent ones.
• Performance and productivity metrics. When people know an automated system is watching a metric, they optimize the metric, not the work behind it.
• Chatbot and agent quality checks. An eval that scores tone and completeness can miss a wrong answer delivered confidently.

The common thread is simple. The moment a target learns the score, it starts optimizing for the score. That's true of a nervous job candidate, and it's true of a model being fine-tuned against a reward. The interview study just measured it cleanly in humans.

What Builders Should Learn About AI Evaluation

That an automated grader can be confidently wrong, quietly gamed, and behavior-changing all at once, so you have to design against all three. The study is a compact checklist for anyone whose system scores something that matters.

A few principles follow directly:

• Test your evaluator adversarially. Feed it exaggerated, padded, and keyword-stuffed inputs and see if the score holds. If gaming raises the score, your metric is measuring the wrong thing, a discipline we detail in AI agent evaluation security.
• Keep humans in the loop for high-stakes calls. Humans caught what the AI missed, so reserve consequential decisions for human review rather than an opaque score.
• Be transparent about what you measure. Clarity about the rubric reduced gaming here; hidden criteria invite it. Explain the "what," even if you keep the "how."
• Don't confuse a confident score with a correct one. The model rated embellishment highly and cheerfully. Fluent output and a high number are not evidence of truth.
• Watch for proxy drift. Anything optimized against a visible metric will chase that metric, so re-check periodically whether your score still tracks the outcome you care about.

One more habit is worth building in: measure your evaluator against reality, not just against itself. An AI score is only useful if it predicts the outcome you care about. So check it against results.

Do the candidates the model ranked highly actually perform on the job? Do the outputs it scored well hold up in production? If the score and the outcome drift apart, the evaluator has quietly stopped working, no matter how confident its numbers look. That check is cheap, and skipping it is how teams end up trusting a broken metric for months.

The honest read: none of this says stop using AI evaluators. They save real time, which is why AI interviews are spreading in the first place. It says build them knowing their two weaknesses: they grade proxies rather than truth, and the thing they grade adapts to them. Design for both, and an AI evaluator becomes a useful filter instead of a gameable one.

How Van Data Team Builds Evaluators You Can Trust

We help teams design AI evaluation systems that hold up when people, or other models, try to game them. That means validating a grader against adversarial inputs, pairing it with human review where the stakes justify it, and being clear about what the system measures so the behavior it triggers stays honest.

A practical audit starts with one question: what would a lazy or dishonest input do to your score? We run exaggerated and adversarial cases through the evaluator, look for where a high score doesn't match a good outcome, and decide which decisions are too important to leave to the model alone. Often the fix isn't a better model. It's a clearer rubric and a human check at the right step.

If your team relies on an AI to score candidates, content, model outputs, or anything else that matters, this study is a reason to pressure-test it. Our work on AI agent evaluation and AI agent evaluation security turns a finding like this into a concrete audit of your own scoring pipeline. The goal is simple: an evaluator that rewards the real thing, not a confident imitation of it.