testing.
12 writings found
Latest Archives
How a 66MB Test Model Caught a Two-Month Bug in Transformers
Model Integration Contracts let small fixture models catch regressions on CPU that full integration tests miss. Here's how Transformers uses them to ship faster.
AI-Generated Tests Are Catching Real Bugs at Production Scale
Meta's Catching JiTTests demonstrate how LLMs can generate executable tests that catch bugs before code lands, achieving 60% true positive rates and preventing 20% of privacy incidents.
The Two Specs Every Production System Has
Why your tests might miss what your system actually does, and how characterization testing catches regressions that intent-based tests never will.
Professional Skepticism in an AI-Driven Development World
Exploring how developers can apply critical thinking, test-driven development, and rigorous testing practices to build reliable AI systems and agents.
Skepticism as a Superpower in the Age of AI Testing
Professional skepticism matters more than ever. David Burns on test-driven development for AI agents, flaky tests, and building reliable systems.
Professional Skepticism in an AI-Driven Testing World
Why developers need healthy skepticism when adopting AI agents, test-driven development for agentic systems, and what flaky tests really tell us about our code.
Making Teams AI Native: Where Testing Becomes the Bottleneck
How agentic engineering is shifting computational bottlenecks from coding to validation, and why robust testing is now the competitive advantage for AI-native teams.
Sensors That Keep AI-Generated Codebase Maintainable
How I used linters, coupling metrics, and mutation testing to help AI agents self-correct and maintain code quality over time
Tests Are the Real Safety Net: Why Your AI Specs Need Executable Validation
Writing specs for LLMs is trendy. But without automated tests, you're flying blind. Here's why the spec document isn't your safety net.
Just-in-Time Tests: How Meta Is Rethinking Software Testing for the Agentic Era
Meta's JiTTests use LLMs to auto-generate bespoke tests per code change, killing false positives and test maintenance overhead in agentic dev workflows.
Just-in-Time Tests: When AI Writes Your Tests Right Before Deployment
Meta's Catching JiTTests use LLMs to auto-generate tests on-demand, targeting regressions without maintenance overhead. A radical shift in testing philosophy.
Building Tools So AI Agents Can Actually Show You What They Made
Simon Willison built Showboat and Rodney to solve a critical problem: getting coding agents to demonstrate their work without cheating.