I’ve been thinking a lot about what happens when we hand junior professionals a powerful tool too early in their careers. A new study from David Autor and Tanya Rodchenko at MIT, published by the National Bureau of Economic Research, tackles this question head-on through a three-month field experiment with patent attorneys. The findings are sobering and worth sitting with.
The researchers randomized 133 lawyers across eleven IP firms into treatment and control groups. The treatment group got early access to a Google Labs AI patent writing assistant (now part of Gemini Notebook). After 10 and 90 days, everyone drafted patents from scratch. At 90 days, they also did something crucial: a redlining task (reviewing and correcting flawed patents) without AI assistance.
The headline number sounds good. AI access improved drafting performance by 0.38 standard deviations by day 90, equivalent to an 11-percentile point jump. Lawyers worked faster and with better results. But here’s where it gets interesting.
The Seniority Gap
When researchers removed AI for the redlining task, the magic disappeared for junior lawyers. Senior lawyers with 7+ years of experience still outperformed controls by 0.45 SD on unassisted work. But juniors? They showed zero average improvement, and their scores bifurcated into more very low scores and more good scores, but no more excellent ones.
This tells me something uncomfortable: AI didn’t accelerate learning for juniors; it created a performance illusion. They could produce acceptable work with AI, but they weren’t building the judgment that distinguishes experts from novices.
The qualitative analysis reveals why. Junior lawyers worked mechanically, sequentially fixing easy stuff first (copy-editing introductions) before running out of time for substantive patent claim work. They diagnosed problems but didn’t execute fixes. They swapped synonyms instead of reconsidering commercial scope. Three months of AI-assisted work didn’t change these patterns at all.
Senior lawyers did something different. They treated AI output as a “logic auditor,” not a finished product. They spent time rebuilding claims from scratch, articulating the why behind edits, engaging with legal doctrine. The AI forced them to strengthen foundational expertise they’d already developed.
What This Means for Developer Learning
I think this maps directly onto software development. Consider two developers given copilot-style tools: one with three years of experience, one with ten. The experienced developer uses it to iterate faster on architectural decisions, catching edge cases the AI missed. The junior developer uses it to generate boilerplate faster but never learns why that boilerplate matters.
Judgment isn’t about moving faster. It’s about knowing when to break your own rules, when templates fail, when the obvious solution is wrong. That comes from thousands of hours wrestling with constraints under mentorship. AI removes the tedium, but it can also remove the friction where learning happens. If a junior never has to debug a complex issue because they copy-pasted a solution, they don’t develop the pattern recognition that makes debugging instinctive later.
The research highlights something I’ve been thinking about on the AI and productivity question: we’re conflating short-term output gains with long-term skill building. They’re not the same thing. You can have one without the other. You might have both. But the Autor/Rodchenko study suggests that for less-experienced workers, you often don’t.
The Missing Mentorship Layer
Here’s what I found most revealing: juniors weren’t getting smarter with AI, but not because AI is bad. They were missing the feedback loop. A senior lawyer reviews a junior’s draft, points out what’s wrong, and explains the reasoning. The junior internalizes it. That’s painful but essential.
AI can generate a better draft. It can’t explain why your draft was worse in a way that builds durable judgment. And when juniors outsource the hard thinking to AI, they don’t get that feedback from colleagues either. The senior partner doesn’t review a problematic draft because the AI fixed it first.
This is why I’m increasingly concerned about AI in educational and early-career contexts without strong scaffolding. The productivity gains are real, but they might come at the cost of the kind of deliberate struggle that builds expertise.
The real question isn’t whether AI improves work today. It does. The question is whether we’re accidentally sabotaging tomorrow’s experts by making their learning years too frictionless.