We’re at an inflection point that most developers haven’t fully internalized yet: intelligence is about to become essentially free. GPT-4-class capabilities that cost $30 per million tokens in early 2023 now cost under a dollar, with some providers pushing below $0.10. The median decline sits around 50x per year. This isn’t just a pricing story—it’s a fundamental shift in what’s economically viable to build.
But here’s where it gets interesting for those of us building systems: cheap intelligence creates a new problem. It’s not about affording AI anymore. It’s about architecting systems that can handle what AI actually does at scale.
Agents Think Differently Than Users
When an AI agent queries your database, it doesn’t behave like a person using a BI tool. It performs what researchers call “agentic speculation”—a high-volume, heterogeneous stream of exploratory work. A single user request might generate thousands of individual SQL queries as the agent explores schema, introspects columns, formulates partial hypotheses, then full ones.
Consider a simple task: “Why did coffee sales in Berkeley drop this year?” A human analyst might run a few targeted queries. An agent runs combinatorial explorations across joins, aggregations, filters, and cohorts. On text-to-SQL benchmarks, researchers found that across multiple agents attempting the same task, only 10-20% of sub-plans are distinct. That means 80-90% of sub-queries are duplicate work.
This is both problem and opportunity. The redundancy is actually helpful—task success rates increase with more agentic attempts. But from a data system perspective, it’s wasted compute. Traditional databases weren’t designed for this access pattern.
Data systems need to be redesigned for agents. We could reuse results across overlapping sub-plans using multi-query optimization and shared scans (techniques from decades-old literature that suddenly matter again). We could return approximate answers good enough for agents to make progress, leveraging approximate query processing. We could rethink the interface entirely—instead of issuing single SQL queries, agents could submit batches with their own approximation requirements.
Most radically, data systems could stop being passive executors. They could be proactive, steering agents toward efficient paths, providing latency estimates before executing expensive queries, preparing materialized and virtual views in advance. The system knows more about its data and performance characteristics than any agent will discover through exploration.
Infrastructure for Swarms
Beyond individual agent-database interactions lies a harder problem: how do we run thousands of agents reliably? Where do they live? How do they remember? How do they coordinate without stepping on each other?
Current solutions rely on files. Agents write their learnings to markdown files, searched via grep or embeddings. At small scale, this works. At scale, it breaks. Context windows have limits. Retrieval latency matters. When agents are doing the bulk of knowledge work, unstructured file-based memory doesn’t cut it.
What’s needed is structured memory organized across multiple attributes. An agent debugging a flaky test shouldn’t retrieve all memories via keyword matching. It should pull only memories tagged with relevant module, language, framework, and failure mode. For data agents, dimensions might include tables, columns, operation types, and corrective instructions.
This mirrors database schema design. In fact, agents might help us define and refine these schemas over time. But the hard part isn’t storage—it’s consistency. When thousands of agents are making concurrent edits to shared state, where most transactions will be rolled back except one “correct” one, you need exactly-once semantics, CRDTs, operational transformation. You need to prevent livelock, where compensating actions never converge.
Failure handling matters too. What happens when one agent in a coordinating swarm fails? How do agents negotiate and reach consensus on shared decisions? How do we handle stragglers? Some frameworks like Temporal exist, but whether they scale to thousands of concurrent agents remains unclear.
Building Systems We Can Trust
Here’s the most unsettling part: agents can now synthesize entire data systems from scratch. Given a workload, an agent can write the schema, indexes, queries, maybe even the serving layer. This is powerful—custom systems for each problem, built instantly.
But how do we verify they actually work? How do we trust systems built by agents? This isn’t just testing; it’s about verification at a level most development teams haven’t tackled. When the artifact is code that will run in production, generated entirely by an LLM, our current approaches to validation need to evolve.
For developers, the implications are profound. The bottleneck isn’t compute or even intelligence anymore—it’s architecture. It’s designing systems that treat agents as first-class citizens, that handle agentic coordination, that enable efficient speculation, that support verification and trust.
We’re not building tools for AI. We’re building infrastructure that AI will use to build the next generation of infrastructure. The question is whether we’ll design it intentionally or stumble into it.