Google 找到了一種讓本地 AI 速度提升最高 3 倍的方法——無需任何新硬體

📄完整原文· 由 trafilatura 自動擷取5749 字

In brief - Google released Multi-Token Prediction (MTP) drafters for Gemma 4, delivering up to a 3x speedup at inference without any degradation in output quality. - The technique—called speculative decoding—uses a lightweight "drafter" model to predict several tokens at once, which the main model then verifies in parallel, bypassing the one-token-at-a-time bottleneck. - MTP drafters are available on Hugging Face, Kaggle, and Ollama under the same Apache 2.0 license as Gemma 4, and work with tools like vLLM, MLX, and SGLang. Running an AI model on your own computer is great—until it isn't. The promise is privacy, no subscription fees, and no data leaving your machine. The reality, for most people, is watching a cursor blink for five seconds between sentences. That bottleneck has a name: inference speed. And it has nothing to do with how smart the model is. It's a hardware problem. Standard AI models generate text one word fragment—called a token—at a time. The hardware has to shuttle billions of parameters from memory to its compute units just to produce each single token. It's slow by design. On consumer hardware, it's painful. The workaround most people reach for is running smaller, weaker models—or heavily compressed versions, called quantized models, that sacrifice some quality for speed. Neither solution is great. You get something that runs, but it's not the model you actually wanted. Now Google has a different idea. The company just released Multi-Token Prediction (MTP) drafters for its Gemma 4 family of open models—a technique that can deliver up to a 3x speedup without touching the model's quality or reasoning ability at all. The approach is called speculative decoding, and it's been around as a concept for years. Google researchers published the foundational paper back in 2022. The idea didn't go mainstream until now because it required the right architecture to make it work at scale. Here's the short version of how it works. Instead of making the big, powerful model do all the work alone, you pair it with a tiny "drafter" model. The drafter is fast and cheap—it predicts several tokens at once in less time than the main model would take to produce just one. Then the big model checks all of those guesses in a single pass. If the guesses are right, then you get the whole sequence for the price of one forward pass. According to Google, "if the target model agrees with the draft, it accepts the entire sequence in a single forward pass—and even generates an additional token of its own in the process." Nothing is sacrificed: The large model—Gemma 4's 31B dense version, for example—still verifies every token, and the output quality is identical. You're just exploiting idle compute power that was sitting unused during the slow parts. Google says the drafter models share the target model's KV cache—a memory structure that stores already-processed context—so they don't waste time recalculating things the larger model already knows. For the smaller edge models designed for phones and Raspberry Pi devices, the team even built an efficient clustering technique to further cut generation time. This isn't the only attempt the AI world has made at parallelizing text generation. Diffusion-based language models—like Mercury from Inception Labs—tried a completely different approach: Instead of predicting one token at a time, they start with noise and iteratively refine the entire output. That’s fast on paper, but diffusion LLMs have struggled to match the quality of traditional transformer models, leaving them more of a research curiosity than a practical tool. Speculative decoding is different because it doesn't change the underlying model at all. It's a serving optimization, not an architecture replacement. The same Gemma 4 you'd already run gets faster. The practical upside is real. A Gemma 4 26B model running on an Nvidia RTX Pro 6000 desktop GPU gets roughly twice the tokens per second with the MTP drafter enabled, according to Google's own benchmarks. On Apple Silicon, batch sizes of 4 to 8 requests unlock around 2.2x speedups. Not quite the 3x ceiling in every scenario, but still a meaningful difference between "barely usable" and "actually fast enough to work with." The context matters here. When Chinese model DeepSeek shocked the market in January 2025—wiping $600 billion from Nvidia's market cap in a single day—the core lesson was that efficiency gains can hit harder than raw compute. Running smarter beats throwing more hardware at the problem. Google's MTP drafter is another move in that direction, except aimed squarely at the consumer end of the market. The whole AI industry is right now a triangle that considers inference, training, and memory. Each breakthrough in either area tends to boost or shock the entire ecosystem. DeepSeek’s training approach (achieving powerful models with lower end hardware) was one example, while Google’s TurboQuant (shrinking AI memory without losing quality) paper was another. Both crashed the markets as companies tried to figure out what to do. Google says the drafter unlocks "improved responsiveness: drastically reduce latency for near real-time chat, immersive voice applications and agentic workflows"—the kind of tasks that demand low latency to feel useful at all. Use cases snap into focus quickly: A local coding assistant that doesn't lag; a voice interface that responds before you've forgotten what you asked; an agentic workflow that doesn't make you wait three seconds between steps. All of this, on hardware you already own. The MTP drafters are available now on Hugging Face, Kaggle, and Ollama, under the Apache 2.0 license. They work with vLLM, MLX, SGLang, and Hugging Face Transformers out of the box.

資料狀態✓ 已擷取全文閱讀原文（Decrypt）

🔍歷史類似事件· 關鍵字 + 標的比對5 則

2026-05-02

新的 Bitcoin 量子提案為 Satoshi Nakamoto 提供了一種無需移動 BTC 即可證明控制權的方法

相似度 130%關鍵字 new/way

2026-04-29

Hyperliquid 正準備透過一種交易現實世界事件的新方式來挑戰 Polymarket

相似度 100%關鍵字 new/way

2026-04-28

新錢包提供無需分叉即可應對 Bitcoin 量子風險的方法

相似度 100%關鍵字 new/way

2026-04-23

Google 推出新款 Tensor 晶片以推動 AI 熱潮，劍指 Nvidia

相似度 100%關鍵字 new/google

2026-04-15

Google 最新的 AI 更新讓工業機器人變得更聰明——原理如下

相似度 100%關鍵字 google/way

💡 目前用關鍵字 + 標的比對（MVP）· 之後會升級為 embedding 語意搜尋

原始資訊

ID：cece3c9452

來源：Decrypt

發佈：2026-05-07 13:13:49

分類：一般 · 導出分類 neutral

標的：未指定

社群投票：+0 / −0 · ⭐ 0 重要 · 💬 0 留言