Walk any HR tech expo floor right now and you'll notice something. Every booth has discovered AI. The chatbot from 2019 is generative now. The keyword-matching tool got a glow-up and a gradient logo. The pitch deck promises a future that stays parked about six months out, which is a convenient distance, because six months from now there will be a new deck.
Here's the part nobody on the sales side wants in the brochure: most of what gets sold as AI does not run in production.
Independent research from the Fosway Group found that only 27 percent of the AI features vendors call "live and usable" hold up under real conditions. The other three-quarters live on a roadmap. And this isn't a recruiting quirk.
MIT's NANDA initiative looked across industries and found that 95 percent of enterprise generative AI pilots produced no measurable impact on the bottom line. The hype is loud. Production tells a different story.
So how do you spend a budget without funding someone's R&D fantasy? Simple. You start by understanding where - and why - the gap exists.
The demo is easy, product is hard
Anybody can build a demo in an afternoon. A demo runs on clean data, one happy path, and a presenter who knows which button to avoid. The thing about theatre is, it's cheap, controllable and inherently fictional.
The expensive work shows up after you sign. Making a tool function across sixteen job boards, two ATSs that hate each other, a CRM nobody finished migrating, and brands that operate in three languages is a different animal. That's where most vendors fail.
MIT data backs this up: companies that buy from specialized vendors and build real partnerships succeed about 67 percent of the time, while teams trying to wire it all together in-house succeed at about a third of that rate. The model was never the hard part; the infrastructure was.
When you watch a demo, you're seeing the best day that software will ever have. Guaranteed.
What works today
Some AI in talent acquisition earns the label. Programmatic job advertising is the clearest example. Campaign optimization, automated media planning, and bid adjustments that used to eat a media buyer's afternoon now happen without constant babysitting.
Recommendation engines run at scale too, powering job suggestions and audience targeting inside live enterprise environments.
A lot of the rest is workflow automation trying to get a higher valuation from private markets. Auto-responses to FAQs and interview scheduling are useful, and they are not intelligent. Knowing the difference keeps you from paying intelligence prices for a glorified calendar.
Where the wheels are still coming off: explainability, audit trails, and bias controls. Those are the parts that keep you out of a lawsuit, and they're the parts vendors tend to build last. It's worth noticing before you sign anything.
Vet the tool before the vendor gets to perform
The strongest buyers I know don't open with the sales demo; they open with competitive intel.
Find people who already bought the thing. Recruiting communities, peer Slack groups, the recruiting corners of Reddit, the hallway conversations that never make the conference recap. Ask a current user to show you the product in real time, with their data, doing their job. Then sit through the vendor's scripted version and compare the two. The distance between those experiences tells you most of what you need to know.
When you do get in the room, a few questions cut through the fog faster than any feature list:
- *What is live in production today, not on the roadmap?*
- *What happens when the AI gets it wrong, and how do I see it and fix it?*
- *Can you show me three current customers where this moved a number, with the before and after?*
That middle question carries extra weight in Europe, where the EU AI Act puts real obligations on high-risk hiring tools.
Accountability isn't a regional concern, though. If a vendor can walk you through a baseline, the change, and the measured result, keep talking. If the answer is a glossy case study with no numbers in it, you have your answer.
"Usable without heroics" is the whole game
Here's a phrase worth stealing for your next evaluation: no heroics required. If a tool works only when your most technical recruiter spends two hours a day coaxing it, you didn't buy software. You bought a second job.
A decent benchmark for whether a company is built around AI or just dressed up in it: bring them a new problem and see how long the fix takes. A real AI shop turns it around in under three months. If the honest answer is three quarters, you're talking to a roadmap with a sales team attached.
The next wave is already getting overhyped
AI agents are the shiny new thing, which means the rebranding has started. Gartner calls it "agent washing," the practice of slapping the agent label on the same old chatbots and automation, and the firm estimates that only about 130 of the thousands of agentic AI vendors are the genuine article. That same forecast expects more than 40 percent of agentic AI projects to get canceled by the end of 2027.
The more interesting movement is happening earlier in the funnel. A handful of platforms now run intake meetings with hiring managers, draft interview questions, and shave a week or more off time to hire before a req gets posted. The hiring manager persona has been ignored for years, which is strange when you consider how much of the process they actually steer.
It's not magic
AI in recruiting isn't magic; it's math, plus a brutal amount of integration work, plus the guardrails that keep your hiring decisions out of a deposition. The vendors worth your money are the ones solving the boring problems: clean data, repeatable integration, outcomes you can put a number on.
So next time someone shows you a glowing demo and a future that stays almost here, ask the one question that ends the meeting fast: "Show me three live customers with real results."
A vendor who can do that is worth your time; a vendor who dodges it just told you everything.