Your newest hire is fast with AI. That says nothing about what they can do without it.

In February 2027 a cabinetmaker in my town will hand a job sheet to a 23-year-old and go back to the machine. The job sheet was built by software. The cut will be measured by a steel square, not by the software, and it will either fit or it will not. That moment — the square against the joint, nobody watching, no assistant to ask — is the one this article is about.

On 3 September 2026, with a revision in October, David Autor and six co-authors published a working paper through the National Bureau of Economic Research on what happens to professional judgement when a tool does the drafting. It runs against the way most small businesses hand out AI licences, and it does so with numbers rather than a hunch.

What the study actually measured

Patent drafting is a good stand-in for any business where a junior produces a document a senior signs. It has a fixed format, a body of rules, and a client who pays for judgement. It is also slow to learn, which is the point.

The team recruited 133 practising patent lawyers at eleven US intellectual property firms and gave a random half of them access to a custom AI drafting assistant for three months. The design was pre-registered, and every draft was scored blind by expert patent attorneys who did not know which lawyer had the tool. That last detail is what makes the result usable: nobody was grading their own colleague.

Two measurements were taken. While the tool was available, lawyers were scored on benchmark drafting tasks. Then, after the three months, everyone did a different exercise with the assistant switched off — redlining an existing patent application, which is the core of the job and cannot be faked by a template.

Where the gains went

With the assistant, output quality rose for the group that had it. The improvement was measured at 0.34 standard deviations after ten days and 0.38 after ninety, and it was largest among the junior lawyers. Read that as a business owner and it looks like a straightforward win: give the newest people the tool and their work gets better immediately.

Then the tool was taken away. On the unassisted task the treated lawyers still beat the control group, by 0.32 standard deviations overall. Split that by seniority and the shape changes. The whole advantage sat with the senior lawyers, at 0.45 standard deviations. The juniors, as a group, showed no average gain at all.

Not because they learned nothing. Because their scores came apart in both directions: far fewer mediocre results, and more results at both ends. Some juniors used the tool as a springboard and came out better than they would have. Others used it as a cushion and came out worse. The averages cancelled, which is the most dangerous outcome for a manager, because an average hides exactly the person you needed to notice.

The paper's own reading is careful and worth keeping: foundational expertise may be a prerequisite for turning AI-assisted practice into durable skill.

Why the same tool helps one junior and hollows out another

A senior lawyer reads an AI paragraph and something in the back of their head flags the clause that is technically valid and commercially wrong. That flag is the product of years of being wrong in public. A first-year lawyer has no flag yet, so a confident, well-formatted, plausible answer looks exactly like a correct one.

This is not an argument against the tool. It is an argument about sequence. When the assistant arrives before the judgement, the person cannot tell which of its suggestions to accept, so they accept the fluent ones. And fluency is what the tool is best at.

The same pattern shows up in work that has nothing to do with law. An estimator who cannot yet read a set of drawings by eye will accept a quantity take-off that is off by fifteen percent, because there is nothing in their head to contradict it. A bookkeeper who has never reconciled a messy month by hand will accept a categorised ledger with a duplicated supplier in it. The output is better than what they could produce alone. Their ability is not.

What this looks like in a business of ten people

You do not need a patent department to recognise the problem. Take the three roles where a small firm usually puts AI first: the person answering enquiries, the person building quotes, and whoever keeps the accounts.

In each case the tool produces something that looks finished. The reply is polite, the quote has all its line items, the ledger balances. A senior notices the reply promises a delivery window the workshop cannot hit, that the quote forgot the crane hire, that a refund was booked against the wrong month. A junior with two months of experience notices none of it, and the customer finds out instead.

The uncomfortable version of this sits in your hiring. If you screen candidates by asking them to complete a task with AI allowed, you are measuring the tool. Give the same task with the tool off and you measure the person. Most small firms now do the first and call it the second.

How to hand AI to someone who is still learning

Keep one unassisted rep in the week. Pick the task that matters most in that role and let the junior do it alone, on purpose, with a deadline. Then check it the way you would check any piece of work. The point is not to punish them for being slow. The point is to keep the skill exercised while it is still forming, and to give yourself a reading that the tool cannot fake.

Review the reasoning, not just the output. When a junior brings you an AI-assisted document, ask why this clause, why this cut, why this price. If the answer is "the assistant suggested it", you have found the gap before the customer does. If the answer is a reason, you have found a person who is learning, whether or not the document is perfect.

Decide who gets the tool first, and it is rarely the newest hire. In the trial the seniors extracted durable skill from assisted practice and the juniors did not, on average. That does not mean withholding AI from young staff forever. It means the person with the least judgement should not be the one with the most automated output and the least supervision. Pair the tool with a reviewer, or delay it a quarter until the basics are there.

What to stop doing

Stop counting speed as the sign that the tool is working. The lawyers in the trial delivered better documents and the junior group learned nothing measurable from it. A team that ships twice as fast and cannot function when the subscription lapses is not twice as good; it is one outage away from the output it had two years ago.

Stop judging new hires on their first AI-assisted draft. It is the easiest sample to collect and the least informative one. And stop assuming that a confident document means a confident author — the two are correlated in people who have done the work for a decade, and unrelated in people who have not.

Frequently asked questions

Does this mean I should not give AI tools to young employees?

No. It means the tool and the judgement should not arrive at the same time with no supervision in between. In the trial the juniors with an assistant produced better work while they had it. The risk appears later, when the tool is gone or wrong, and nobody in the chain can tell. Pair the tool with a reviewer and keep some unassisted practice in the week.

My business is not law. Does a study about patent lawyers apply?

The mechanism does. Any role where a junior produces something a senior signs has the same structure: quotes, estimates, invoices, replies, drawings, treatment plans. What changes is the consequence of a plausible mistake, not the pattern that produces it.

How would I even measure this in a small team?

Pick one recurring task, wait for a normal week, and ask the person to do it once without the tool. Compare it with the AI-assisted version. Do that every quarter for the two or three roles where mistakes cost money. It takes an hour and it tells you more than any adoption dashboard.

Is this a reason to wait before adopting AI at all?

For an owner who already has the judgement, no: the senior lawyers in the 2026 trial kept a real advantage even without the tool. For the newest member of the team, the answer is to sequence it, not to skip it. The tool is not the problem. Putting it where the judgement is thinnest, unwatched, is.