I’ve been watching the speculative decoding space evolve, and I think we just hit an inflection point. Liquid AI’s new DSpark draft model for their LFM2.5-VL vision-language model represents something I don’t see discussed enough: the practical limits of optimization, and how they force us to think differently about model architecture.
Let me start with the headline numbers, because they’re genuinely impressive. On an M5 Max running MLX, decoding gets 2.3-3.1x faster. On H100 GPUs, you’re looking at 20x faster decoding. But here’s where I think the real story lives: end-to-end latency only improves 1.56-2.62x on the Mac and 1.64-2.27x on the H100. That gap is Amdahl’s law doing its thing, and it’s the most honest technical communication I’ve seen from a model release in months.
Why Speculative Decoding Works, and When It Doesn’t
For anyone not deep in the weeds, speculative decoding uses a smaller draft model to propose multiple tokens at once, which the larger target model verifies in a single forward pass. If the drafts are right, you’ve just turned multiple slow sequential steps into one faster parallel check. The math works beautifully for decoding, which is memory-bound on modern hardware.
But here’s what kills the speedup: prefill and vision encoding don’t benefit. The vision encoder processes images into token representations, then the language backbone ingests hundreds of visual tokens alongside your text prompt. On datacenter GPUs, prefill is mostly compute-bound and its cost grows quadratically with prompt length. On edge devices, it’s even worse because you have less compute capacity, so prefill dominates the wall clock time.
This is why the M5 Max shows better relative gains than you’d naively expect: those per-core neural accelerators narrow the gap between prefill and decode work. But even there, the end-to-end win is capped by the parts you can’t accelerate.
The Architecture That Makes This Work
Liquid AI’s implementation is elegant in its restraint. The draft model taps into the target’s hidden states at fixed layers, conditioning on identical dimensionality representations whether you’re processing images or text. That means the inference algorithm is literally unchanged from their text models. They settled on a 4-layer attention-only drafter with a block size of 8-9 after ablations and 10 epochs of training on their vision-language SFT mixture.
The result: 280M parameters, adding just 8.9% to the deployed model’s footprint. That’s the kind of efficiency that changes the economics of serving these models. You’re not doubling your parameter count; you’re basically adding a thin adapter layer.
What strikes me most is the honesty about training and evaluation. They ran 10 epochs and explicitly measured acceptance rates at each step, stopping when they hit diminishing returns. Real work, real measurement, real constraints. They evaluated across six diverse tasks (VQA, text-OCR, captioning, chart understanding, reasoning, multi-turn) following the MMSpec benchmark.
What This Means For Your Deployment
If you’re running vision models on edge devices or even in datacenter settings, this changes your calculus. The day-one support for llama.cpp, MLX-VLM, and SGLang means you’re not waiting months for framework support to catch up.
For edge deployment specifically, those 1.56-2.62x end-to-end gains are real wins, especially if you’re time-to-first-token constrained. Users notice latency, and cutting 40% off your response time registers in the product experience. On H100s, the 1.64-2.27x end-to-end improvements make this worthwhile for high-throughput inference scenarios where you’re batching requests.
I’d be watching how this plays out in the market for https://mgks.dev/tags/edge-ai/ deployment. The constraint isn’t usually the decoding speed anymore; it’s prefill and encoding. That means future optimization work needs to focus there, not just on decode paths. Speculative decoding is approaching diminishing returns on its own.
The Broader Implication
This release tells me something important about where we are in the ML infrastructure cycle. We’ve optimized decode speed to near-physics limits through techniques like speculative decoding. The next frontier is attacking prefill, and that probably requires architectural changes, not just inference tricks. Check out our recent coverage of inference optimization techniques to see how this landscape is shifting.
The fact that Liquid AI is shipping with exact verification, readable configs, and clear performance reporting across diverse hardware suggests we might finally be moving past the era of hand-wavy benchmark claims. That’s worth something. When vision models can run meaningfully faster on edge devices without requiring entirely new model architectures, we’re entering a phase where deployment efficiency becomes a first-class citizen rather than an afterthought.
The question now isn’t whether you can run vision models on edge devices, but whether you can afford not to.