We Recorded the Interview. Nobody Checked the Outcome.
An engineering director in San Jose offered the most useful assessment of interview intelligence I heard while working on our latest research, one that probably wouldn’t fit all that well on product marketing collateral or vendor websites.
“We had recordings. Nobody was watching the recordings, because watching recordings is not anybody’s job.”
There it is: an entire technology category’s unfinished business, explained in two sentences. The organization had bought the software, captured the conversations and accumulated a searchable record of its hiring process. Somewhere between implementation and improvement, someone needed to do something with the information. And apparently, that person hadn’t been hired yet.
That gap is a recurring theme that connects the research findings from The State of Interview Intelligence, our new Kyle & Co study, sponsored by Textio. We recently surveyed hundreds of job seekers and employers, alongside interviews with talent and HR leaders. Textio funded the research without directing its design, analysis or conclusions (they did, however, do some very helpful copy editing).
What emerged should frankly unsettle anyone who assumes that better documentation automatically produces better hiring outcomes. Recruiting has embraced the ability to document and record candidate conversations while leaving the data, and outcomes, those conversations generate largely unexamined.
Among employers in our sample, 85 percent already use interview intelligence tools across some or most interviews; 97 percent are at least piloting them, while only 23 percent report using the technology to improve interview quality or reduce bias.
There’s value in better notes, particularly if you’ve ever tried to reconstruct a hiring decision from three overdue scorecards and a manager who remembers liking somebody. But adoption tells us the software arrived. It tells us considerably less about whether anyone’s judgment improved.
Employers nevertheless sound pretty pleased with themselves. Asked how well their interviews predict job performance, they gave themselves an average of 8.3 out of 10. Ninety-four percent scored themselves seven or higher. Among the talent leaders we interviewed, only a handful had actually compared interview evaluations with subsequent performance.
That’s an impressive level of confidence (or lack of self-awareness) for a process so rarely required to show its work. I’ve spent enough years in this industry to recognize the appeal: checking whether your instincts are right, of course, creates the unfortunate possibility of discovering that going with your gut isn’t going the way you want it to. Ignorance, as they say, is bliss. It’s also a major miss for employers across markets, industries and company sizes.
One director of talent acquisition I spoke to at a financial services firm in Des Moines had actually done the work of correlating historical interview answers and their respective structured scorecard methodology to hiring outcomes.
Their team compared interview scores with twelve-month performance ratings for about 400 hires. The overall process held up better than they’d feared. Three of the company’s most requested interviewers, however, showed almost no relationship between their scores and the performance of the candidates who were ultimately hired. Their popularity, as it turns out, had nothing to do with their predictive abilities.
This describes a problem that’s always been invasive, and ubiquitous, in talent acquisition. This example also demonstrates what’s possible when an organization connects its hiring decision making with post-hire performance and productivity.
Turns out, conversational intelligence and talent intelligence are inexorably intertwined - and, some employers, finally, are starting to realize that the job interviews create more meaningful data and actionable analytics than any other part of the hiring process.
The candidate findings themselves reveal another (unintended, but tangible) consequence of not testing assumptions of efficacy against actual hiring outcomes.
Employers rated their interview experience at +48 using Net Promoter scoring. Candidates rated theirs at +3. These were separate samples, rather than matched accounts of the same interviews, but the contrast is difficult to dismiss. Only 11 of 257 employers were detractors of their own process; 59 of 174 candidates were detractors of theirs.
A process can look excellent from the side of the table that controls the interview scheduling and outcomes, but lacking from the candidates sitting on the other side, whose expectations and experience rarely seem to align with what many employers deliver during job interviews.
Consider the issue of feedback; eighty-one percent of employers said rejected finalists always or usually receive substantive feedback. Only 43 percent of candidates rejected at the final stage said they received anything useful. Somewhere between the policy and the person waiting for an explanation, a considerable amount of substance disappears.
A transcription could help close that gap, provided it contains evidence somebody has evaluated. If a candidate says they managed a large budget; the software can capture that claim and categorize it as having a specific skill, in this case, financial acumen or P&L oversight - aligning candidate responses directly with job requirements.
Of course, unless someone asked how they allocated resources, measured returns or changed course when spending underperformed, the resulting scorecard has preserved an assertion without examining it.
The recruiter then inherits the task of turning “we’re passing” into a useful explanation. Better formatting won’t supply the missing reasoning, although it will make the absence look considerably more professional and less conspicuous.
Honestly, candidates aren’t demanding perfection. Asked what mattered most in judging a hiring process, they put fairness and communication first; together, those factors accounted for roughly two-thirds of responses. Getting the job ranked last, at 6 percent.
That finding makes the familiar dismissal of candidate criticism as ‘sour grapes’ considerably harder to maintain. People can accept rejection while still expecting the employer to explain the process and finish the conversation.
Interviews themselves remain worth improving. A meta-analysis in the Journal of Applied Psychology ranked structured interviews highest among the selection procedures examined. The opportunity is substantial when employers establish relevant criteria, gather comparable evidence and check their conclusions.
There’s also information hiding in plain sight on the employer’s side of the conversation. A manager describes responsibilities that barely resemble the job posting. Two panelists disagree because they’re evaluating different versions of success. Candidates repeatedly question a workload or compensation package leadership considers perfectly reasonable.
Those are potential signals about the business, worth investigating beyond the immediate hiring decision. They aren’t validated predictions simply because a transcript contains them. Someone has to ask the question, examine the pattern and decide what needs to change.
That’s why this research matters beyond the next software renewal. It asks whether employers can explain their decisions, learn from their mistakes and use the conversations they’re already paying to have.
The full study examines where those practices break down and what organizations can do about them, including clearer evaluation criteria, useful feedback and comparisons between interview judgments and later outcomes. Much of the work begins with assigning responsibility and asking better questions.
Read The State of Interview Intelligence. Your next hire may generate another recording, another summary and another recommendation based on instinct instead of evidence. Interviews are more than just another part of the hiring process - they’re the key to unlocking the causation data that connects talent acquisition with employee performance and business outcomes.
And that’s really what quality of hire is really all about.