Hiring in an AI-First Company: What to Assess When Agents Do the Work

Hiring in an AI-First company forces two changes: what you look for, and how you verify it. When part of the work is executed by agents, the ability to produce stops being the differentiator and the ability to direct, verify and decide takes its place. And because candidates can use AI in your assessment — with your permission or without it — tests designed to measure production have stopped measuring anything at all.

This isn't theoretical. It's what happens the moment a team starts operating with agents: the writer producing ten pieces a day competes with a system producing a hundred, while the one who knows which of those hundred is wrong, and why, becomes more valuable than before. We hire this way in our own teams, and this article describes the process we use, including its limits. If you're not yet clear on how the roles land, start with our AI-First org chart and roles; here we assume you already know which seats you're filling.

What actually changes in a role when agents are involved?

The point in the process where the person adds value changes. You used to hire execution capacity: someone to do the work. Now you hire definition and verification capacity: someone who can specify what needs doing and spot when the output is wrong.

In practice, three competencies rise in value and two fall. Rising: domain judgment — knowing what a good result looks like in your field — the ability to decompose a problem into verifiable steps, and the discipline to check rather than assume. Falling: raw production speed and memorized procedural knowledge, because those are precisely what an agent does better and cheaper.

There's an uncomfortable consequence worth saying out loud: the traditional junior role compresses. Entry-level positions used to consist of repetitive supervised work, which is the first thing to be automated. That doesn't mean stopping junior hiring — do that and in five years you'll have no seniors — it means changing what their first year consists of: less production, more exposure to decisions, with a mentor reviewing judgment rather than spelling.

How do you assess someone who can use AI during the test?

By assuming they will, and designing the test accordingly. Banning it is futile — you can't verify it — and counterproductive, because you'd be measuring a situation that will never occur in the actual job.

The move that works is this: you supply the output and ask for the judgment. Instead of "write a commercial proposal for this client", hand over three already-written proposals — one good, one mediocre, one with a substantive error — and ask them to rank them, identify the problems, and explain which they'd send and why. AI doesn't solve that task for them, because it requires the judgment you're buying.

Three formats we use, and what each reveals:

• Error-detection test. A realistic deliverable with two or three planted faults: an inconsistent figure, a conclusion the data doesn't support, a breach of internal policy. It measures exactly what they'll do in the role.
• Open-tools task with an oral defence. They can use anything they like for 48 hours; then they spend 20 minutes explaining why they made each decision. The conversation reveals within five minutes whether there's judgment behind the work or just a copied result.
• A case with insufficient information. The brief deliberately omits data. What you're assessing is whether they ask, whether they spot the gap, or whether they fill it with unflagged assumptions. That last behaviour is the one that causes most trouble when working with agents.

All three share one property worth naming: the candidate can't win by producing more. That's the design goal. A test where AI meaningfully improves the submission is a test measuring the tool rather than the person — and since every candidate has the same tool, it discriminates on nothing. Build the test so a strong AI user and a weak one submit work of similar surface quality, and the difference shows up entirely in the reasoning.

What to assess now versus what you assessed before

The last row discriminates most. Ask about a time they changed how they worked, and why. Anyone without that story — or with one where it was imposed and they resisted — will struggle in an organization that redesigns processes every quarter. The cost of that adaptation isn't theoretical: we break it down in our article on AI training for employees and its ROI.

Five interview questions that genuinely separate

1. "Tell me about a time a system or tool gave you a result that looked right and wasn't. How did you catch it?" Look for the checking mechanism, not the anecdote. If it has never happened to them, either they don't check or they don't remember; neither is a good sign.
2. "If you had to explain this task to someone with no context, what would you tell them?" A direct test of the ability to specify, which is 80% of directing an agent.
3. "Which part of your current job would you automate tomorrow, and which wouldn't you touch?" Reveals whether they understand where their own value sits. The best candidates point at the boundary precisely and argue for it.
4. "How would you know you'd done it well?" If they can't name a verifiable criterion, they won't be able to verify an agent's work either.
5. "Tell me about a decision you made with incomplete information." Look for whether they took the risk consciously and flagged it, or simply never saw the gap.

What should not change in your hiring process

Automating the recruitment process itself is a separate topic — we cover it in AI recruiting automation — but some limits belong here because they directly affect the quality of who you hire.

• Automated candidate rejection. An agent can sort, summarize and flag; the decision to reject a person is taken by a person. Beyond being right, it's what the European framework requires for automated decisions with significant effect.
• The final interview. Cultural fit, motivation and trust are assessed by talking to someone. No system replaces that, and whoever tries hires badly.
• Transparency with candidates. If you use AI at any stage, say so. Opacity is a reputational and legal risk that buys you nothing.
• Bias review. A system trained on your historical hires reproduces your historical biases more efficiently. If you automate screening, audit outcomes by profile every quarter.

Redesigning the process in four steps

1. Rewrite the job description starting from decisions, not tasks. "Decides which campaigns get stopped" says more than "manages campaigns", and it forces you to be clear about what you expect.
2. Replace the production test with a judgment test. Highest-impact change and the cheapest: an afternoon's work reusing real deliverables from your team.
3. State the AI-use policy for the process. In writing, in the posting: what's allowed, what's assessed, and what the oral defence will probe. Good candidates appreciate it.
4. Measure at six months. Compare recent hires' actual performance against what the test predicted. If they don't correlate, the test is wrong and needs rebuilding; it's the only improvement loop that works.

Onboarding needs the same treatment, and it's the step most companies skip. A new hire in an AI-First team has to learn two things at once: your business, and where the boundary sits between what they decide and what the agents execute. Make that boundary explicit in writing during week one — which actions run automatically, which need their approval, what they're accountable for when an agent gets it wrong — instead of leaving them to infer it over six months. The people who leave in the first year almost never leave over the tooling; they leave because nobody ever told them what they were actually responsible for.

This redesign isn't a standalone HR project: it only works if the operating model has already defined what people do and what agents do in each process. That order — model first, hiring second — is set out in the AI-First operating model.

Frequently asked questions

Should I require prior AI experience in the posting?

It's a filter with poor cost-benefit. It screens out people with excellent judgment who simply haven't had the chance, and lets through anyone who's used an assistant for six months without changing anything fundamental. It's more useful to ask for evidence of adaptation — having changed method at least once — and train on the specific tool, which takes weeks.

Do I hire fewer people, then?

Experience at companies that have made the transition points the other way in revenue-generating functions: the same headcount does more, and the bottleneck moves to selling or delivering, not producing. Where headcount does fall is pure process administration, and that is worth planning with internal redeployment before it becomes a morale problem.

What do I do with the team I already have?

First, don't import from outside what you can build inside: knowledge of your business takes years to acquire and the tool takes weeks to learn. A reskilling plan with protected hours and one real use case per person returns more than three new hires, and costs less.

How do I stop a candidate from "passing" the test with AI but without judgment?

The oral defence. Twenty minutes of "why this option and not the other?" and "what would you do if the input data were false?" separates unambiguously between someone who understands what they submitted and someone who copied it. You don't need to ban anything: assess where the tool doesn't reach.

Does this apply to non-technical roles?

It applies to non-technical roles first. Administrative, sales, marketing and customer-service profiles transform the most, because their work is primarily text production, information handling and coordination. Technical roles change too, but tend to adapt on their own because they're used to switching tools every few years.