I’ve been thinking a lot lately about the gap between being productive and being skilled. A recent study from David Autor and Tanya Rodchenko at the National Bureau of Economic Research crystallizes something I’ve suspected: AI tools can make us look better at our jobs without actually making us better professionals.
The research followed 133 patent lawyers at 11 IP law firms over three months. Two-thirds got early access to a Google Labs AI patent drafting tool; one-third didn’t. The results were striking: lawyers with AI access produced notably better work on drafting tasks. Their scores improved by roughly 10-11 percentile points compared to controls. Productivity gains were real too, with junior lawyers finishing tasks 18 minutes faster.
But here’s where it gets interesting. After 90 days, researchers asked all lawyers to complete a redlining task (reviewing and correcting an existing patent) without AI assistance. This is where the illusion shattered.
The Bifurcation Problem
When the training wheels came off, senior lawyers maintained their advantage. They outperformed controls by roughly 13 percentile points on unassisted work. These experienced professionals had used the AI tool as what they called a “logic auditor.” They weren’t accepting AI outputs wholesale. Instead, they were interrogating the tool’s reasoning, forcing themselves to articulate why certain changes mattered and how legal doctrine applied.
But junior lawyers? Their scores bifurcated. Some got better, but others got worse. More excellent work from some, more poor work from others. The middle collapsed. Three months of AI-assisted drafting didn’t translate into lasting skill development.
The qualitative patterns were revealing. Juniors using AI followed rigid, sequential workflows. They edited from top to bottom, polishing low-stakes introductory sections while running out of time for substantive patent claims. When they spotted serious flaws, they documented the problem rather than executing the fix. Crucially, control group juniors did this too. The AI didn’t remedy their fundamental deficit; it just hid it behind better final output.
What This Means for Developer Growth
I think the implications here extend well beyond patent law. As developers, we’re increasingly working with AI pair programmers, code generation tools, and automated debugging assistants. The question isn’t whether these tools improve our immediate output. They do. The question is whether they’re building our judgment or replacing it.
Look at posts on developer productivity: plenty celebrate the time saved and velocity gained. Fewer examine what’s lost when junior developers spend six months with Copilot handling architectural decisions without developing the instinct for when a particular pattern matters. Fewer still ask whether we’re creating a generation of developers who can write functional code but can’t design systems.
The study hints at what separates senior professionals from juniors: judgment earned through thousands of hours of deliberate struggle. Seniors reviewing complex problems aren’t just pattern-matching. They’re integrating domain knowledge, tradeoffs, and contextual reasoning in ways that take years to internalize. When senior lawyers used AI, they already had that foundation. They could pressure-test the tool’s output. They knew what to question.
Juniors don’t have that yet. And if AI lets them skip the hard work of building it, we’re not accelerating their growth. We’re creating a capability gap that widens over time.
The Expertise Trap
There’s a genuine tension here. I don’t want to romanticize struggle or suggest that AI assistants should be restricted from junior developers. But the evidence suggests we need to be intentional about how we deploy these tools in learning contexts.
The most concerning pattern: AI can help juniors produce adequate work in the moment without helping them develop the judgment needed to produce excellent work independently later. It’s a local optimization that undermines long-term growth. We see this already in codebases where AI-generated code works but reflects no coherent architectural philosophy. It’s functional. It’s not excellent.
This connects to broader thinking about AI and learning. The real leverage of AI tools for junior professionals likely comes from embedding them thoughtfully into supervised learning workflows, not replacing the struggle entirely. A junior developer working with a senior who explains why a particular refactoring matters, who uses AI to accelerate execution while maintaining intentional decision-making, is building different skills than one who uses AI in isolation.
The study suggests that three months isn’t enough time for pattern changes to persist. But in software development, where tools evolve quarterly and codebases turn over yearly, we might be looking at an even shorter window before AI-assisted habits calcify into permanent skill gaps.
The question for our industry is whether we’ll design AI workflows that preserve the struggle that builds judgment, or whether we’ll optimized for pure productivity until we realize we’ve created a workforce that’s fast but shallow.