When you’re serving machine learning features to millions of requests per day, the theoretical elegance of your system design collides hard with operational reality. Meta’s F3 Lightning represents a fascinating case study in how to preserve efficiency while unlocking developer velocity.
The core tension F3 Lightning solves is this: shared computation is efficient, but shared releases are slow. And at scale, slow kills iteration.
The Merged Graph Problem
F3, Meta’s feature computation platform, represented each feature as a directed acyclic graph (DAG) where nodes are operations (reading data, filtering, aggregating) and edges represent dependencies. The clever part was merging these individual feature graphs into one unified graph, collapsing duplicate work. If ten features all need a seven-day click count, compute it once.
At a few hundred features, this worked beautifully. But as F3 scaled to 20,000 features, the architecture became a bottleneck. One malformed feature definition would block the entire merged graph from shipping. A single model’s expensive validation tests would hold up thousands of unrelated features waiting for deployment.
I find this pattern everywhere in systems design: the structure that makes sense at small scale becomes a release bottleneck at large scale. The problem wasn’t the merged graph itself. The problem was using it as both the execution unit AND the deployment unit.
Separating Concerns
F3 Lightning’s insight is deceptively simple: deployment and execution don’t have to move together. Instead of one artifact that determines both when features ship and how they run, F3 Lightning splits these concerns:
Each feature compiles independently into its own configuration, carrying everything it needs to compute itself. This isolation means feature A ships regardless of whether feature D has bugs. The release path becomes an ordered queue instead of a batch that either ships completely or not at all. Features that no model uses yet move fast. Changes to live models get extra testing without blocking others.
But here’s where it gets interesting: while features now release independently, the system still needed to share execution at runtime. The early prototypes separated deployment only, and CPU usage nearly doubled because every read, filter, and aggregation executed separately for each feature.
Execution Without Merging
Bringing back shared execution without merging requires solving a thorny problem: how does the runtime recognize identical computations across independently compiled configurations?
F3 Lightning’s solution is elegant. Instead of using node numbers (which are local to each feature graph), it derives each identifier from what the work actually does: the operation, its parameters, and the identifiers of its inputs. This captures the entire upstream computation chain. A seven-day filter compiled by feature A last week and feature B this week will have the same identifier.
At startup, rather than rebuilding a merged graph, F3 Lightning examines all active feature configurations, finds shared work by comparing identifiers, and builds a single execution plan using topological sort. Multiple copies of the same computation collapse into one. Requests traverse this plan as a flattened ordered list, not a pointer-by-pointer graph traversal.
The Memory Locality Surprise
Here’s what surprised me most about this project: even after eliminating all duplicate computation, CPU usage remained above traditional F3. The culprit was memory layout.
Each feature configuration maintained its own request-time data container, so intermediate results scattered across separate memory allocations. Walking the execution plan meant jumping between them, destroying cache locality. Modern CPUs care as much about where results live as how many computations run.
The fix consolidated everything into one buffer per request. At startup, every feature’s references get rebound to slots in that single buffer. This restored the sequential access pattern the merged graph had, and removed per-feature allocations on every request. As a bonus, the system stores results once and references them from multiple consumers instead of copying list and map values around.
The final optimization precomputed every identifier-to-slot mapping at startup, replacing expensive runtime hash lookups with direct array indexing. Identifiers became purely a compile and startup concern.
What This Teaches Us
F3 Lightning ships features 3.6x faster than traditional F3 with no infrastructure cost increase. But beyond the performance metrics, there’s something more important: a demonstration that deployment and execution should have separate boundaries, that request-path work belongs at startup when possible, and that architectural elegance often hides behind smart data layout decisions.
The lesson that generalizes most: you can eliminate every redundant computation and still leave performance on the table if you ignore memory patterns on modern hardware.