LFM2.5のDSparkで推論が最大3.18倍に高速化、量子化版も公開

Today, we release DSpark [1] draft model checkpoints for three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, the recently released LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality. The draft models reach up to 3.18 throughput improvement on a GPU and up to 2.87x on-device. ／ This is the first public release of speculative decoding models for Liquid Foundation Models (LFMs). ／ The DSpark draft models for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B are available today on Hugging Face. Additionally, the LFM-compatible DSpark integration is open-sourced upstream in llama.cpp and SGLang. ／ The resulting draft models are relatively small, with each around ~300M parameters, as shown in Table 1. ／ Table 1: Draft model size. Embedding and LM head are tied to the target, not carried by the draft. — 冒頭の発表文と学習の節の記述より

Today, we release updated 4-bit checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B trained with Quantization-Aware Distillation (QAD). QAD is a technique to distill a high-precision teacher model into a quantized student model. These checkpoints keep the low memory footprint and high throughput of Q4_0 GGUFs while recovering most of the accuracy lost to quantization: all four land at roughly 97% of their BF16 averages. ／ QAD Q4_0 uses the native Q4_0 throughput because the two checkpoints have the same GGUF format, tensor layout, and runtime path. — 冒頭の発表文と Deployment tradeoffs の節の記述より

The decode phase in LLM inference is traditionally memory-bound. Most of the latency comes from streaming weights from DRAM into SRAM, not from intense computation. This is true for both powerful GPUs, such as the NVIDIA H100, and for edge devices like MacBooks or iPhones. ／ Speculative decoding addresses this issue by using a lightweight draft model to produce candidate tokens. The target model then verifies them all in a single forward pass, sharing the cost of loading the weights across all tokens we are verifying. ／ A lightweight sequential head, modeled as a Markov chain between neighboring tokens, that biases each position's logits toward continuations consistent with the token sampled just before it. This adds dependency between drafted tokens, which is absent in the DFlash approach, raising the acceptance rate at later positions. ／ A confidence-scheduled verifier: a separate head predicts each draft token's acceptance probability conditioned on all previous ones being accepted, and a hardware-aware scheduler prunes low-confidence suffixes whenever verifying them would cost more batch capacity than they're worth. ／ Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model's own token takes its place. The emitted sequence is therefore identical to baseline greedy by construction, so benchmark accuracy (pass@1 or exact match) is unchanged. ／ Throughout the ablations, we settled on 5 layers with a block size of 9. For each draft model, we ran 15 epochs on the entire dataset. ／ For the final published checkpoints, we therefore select the epoch with the highest acceptance rate rather than based on loss. ／ We trained each model and ran all ablation studies exclusively on AMD hardware using Liquid AI’s training framework. — How DSpark works・Quality parity・Training の各節の記述より

We measure on-device throughput with llama.cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights at batch size 1, temperature 0, and up to 256 output tokens. We measure GPU throughput with SGLang on a single H100 80 GB in BF16 at batch size 1 and temperature 0. Both configurations use DSpark block size 9 and are evaluated on five benchmark datasets (MATH500, GSM8K, HumanEval, MBPP, MT-Bench). ／ Speedup on MacBook is especially noticeable as it pushes the interactivity level a user can enjoy to a level far exceeding the throughput offered by most proprietary cloud models (around ~140 tok/s, depending on the dataset). ／ Our main goal for LFM2.5-2.6B was to make it the first viable on-device agentic model. In agentic workloads, the model reasons before every tool call, and the user waits through it all. That is where speculation pays most. ／ Across various multi-tool scenarios, DSpark reduces the latency by 57% on average. ／ Note that, in contrast to the 2.6B model, 1.2B is a non-reasoning model. For 1.2B, we observe much more variance in dataset acceptance rates, so speedup varies by as much as 52% depending on the distribution of the underlying text. — 測定条件と2.6B・1.2B-Instructの結果に関する記述より

For LFM2.5-8B-A1B, the inference gets more challenging. The acceptance rate goes up compared to two dense models (8B-A1B is also a thinking model), but the observed real-world speedup does not reflect this. While we can achieve a reasonable 2.54x throughput improvement on average on a GPU, we get only an 18% improvement on edge devices, as shown in Table 4. This is due to the current MoE implementation in llama.cpp's Metal backend, and to the fact that verifying k tokens activates more experts and thus more weight traffic than a single decode step.This will be a subject of subsequent work, and we publish the numbers for transparency. — 8B-A1B の結果に関する記述より

For all four models, we compare their released GGUFs produced with post-training quantization (PTQ) against the trained QAD Q4_0 checkpoints on a benchmark suite spanning reasoning, instruction-following, tool use, and agentic capabilities: GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. The BF16 GGUF serves as the in-format ceiling. We also add one scale-appropriate math evaluation: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B. We report the mean across five repeats. ／ Across model sizes, QAD substantially improves the Q4_0 checkpoint, with the strongest recovery generally occurring in the smaller models where PTQ Q4_0 incurs a larger relative quality loss. We quantify recovery as the fraction of the BF16-to-Q4_0 quality gap that QAD closes. QAD closes 70.6% of the gap for LFM2.5-230M, 73.4% for LFM2.5-350M, 65.5% for LFM2.5-1.2B, and 48.4% for LFM2.5-2.6B. The resulting checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance. — Benchmarks の節の記述より

Natively, Q4_0 is the weakest 4-bit format in terms of quality and the fastest across our backends. This is precisely the tradeoff QAD targets. QAD Q4_0 matches Q5_K_M quality within run-to-run variance at 230M and 350M and comes within half a point at 1.2B, while staying smaller and faster. ／ It also beats Q4_K_M, the standard 4-bit option, and matches Unsloth’s UD-Q4_K_XL, a strong external post-training quantization checkpoint. ／ MacBook Pro: GPU inference on an Apple M5 Max; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown. ／ NucBox EVO-X2: GPU inference on an AMD Ryzen AI Max+ 395; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown. ／ Samsung Galaxy S26 Ultra: Arm CPU inference on a Qualcomm Snapdragon 8 Elite Gen 5; 100 generated tokens after a 256-token prompt prefill. BF16 and F16 reference points are shown, with F16 serving as the practical full-precision reference. ／ Raspberry Pi 5: Arm CPU inference on a Broadcom BCM2712. For the 230M, 350M, and 1.2B models, throughput is measured using a shorter decode-only run of 32 generated tokens with no prefill, with F16 as the full-precision reference. — Deployment tradeoffs の節の記述より

DSpark draft models for LFM2.5 ship with day-one support across the inference ecosystem: ／ llama.cpp — GGUF checkpoints for efficient edge inference ／ SGLang — GPU-accelerated serving for production throughput ／ SGLang implementation builds on the official SGLang implementation of DSpark, and llama.cpp builds on the official codebase, which we run with experimental metal kernels. — Faster Inference の節の記述より