标签：#mtp

I saw "MTP round-trip" on a checklist for a Megatron conversion pipeline and had no idea what it meant. Two acronyms, one hyphen, apparently important enough that someone had listed it as a thing to verify. Working out what it meant took me somewhere I didn't expect. The interesting part turned out not to be MTP at all — it was the reason speculative decoding works in the first place, which rests on a fact about hardware that I had backwards. TL;DR Generating text is slow because it's sequential: one full forward pass per token. But a forward pass over five tokens costs about the same as over one. Generation is bottlenecked by moving weights, not by arithmetic. Speculative decoding exploits that: something cheap drafts k tokens, the big model verifies all of them in one pass. It is exact , not an approximation. Same output distribution as normal decoding. MTP (Multi-Token Prediction) is one way to produce those drafts — a small module trained into the model itself. MTP has two separate lives: a training-time auxiliary loss you can throw away, and an inference-time draft head you can't. Why generating text is slow To produce token N+1, the model needs token N. There's no way around that ordering — it's what "language model" means. So generating 100 tokens means 100 full passes through the network. For a model like GLM-5.2, that's 78 layers, 100 times over. The obvious conclusion is that generation is 100 times as expensive as reading the prompt. The obvious conclusion is wrong, and the way it's wrong is the whole point. The part that got me A forward pass processing one token and a forward pass processing five tokens take roughly the same wall-clock time. I had assumed compute scaled with tokens. It doesn't, because compute isn't the bottleneck. Every forward pass has to read the model's weights out of memory and into the compute units. That's hundreds of gigabytes moving across a memory bus, and it happens whether you're processing one token or fifty . The actual ar

All tests run on an 8-year-old MacBook Air. You're transferring a batch of large files over MTP. The first one flies at 45 MB/s. Then the second file starts — and you're at 30 MB/s. The third is slower still. Nothing changed. Same cable, same device, same app. So what's happening? The Cause Is in the Protocol Itself Between every file, MTP requires a full negotiation cycle — SendObjectInfo followed by SendObject . This isn't an implementation detail you can optimize away. It's how MTP works. During that gap, a few things happen in sequence: The Android device's flash controller is still committing the previous file to storage The USB pipe is flushed and re-established for the next object The device's MTP stack is processing metadata before it's ready to receive data again The result is a speed dip at every file boundary. The longer the previous file, the longer the device needs to catch up. What I Tried Building HiyokoMTP, I went through the obvious candidates: Tokio thread pool exhaustion — sync Read/Write calls blocking async threads were a real issue. Fixing it improved overall stability, but didn't eliminate the inter-file dip. Chunk size tuning — adjusting the USB bulk transfer buffer (up to 4 MB per chunk) helped peak throughput, but not the boundary behavior. Intentional cooldown between files — adding a short pause actually helped in some cases, giving the device's flash controller time to breathe before the next transfer starts. Why It Can't Be Fully Fixed The inter-file overhead is structural. MTP was designed as a stateful, command-response protocol — not a streaming pipeline. Every file is a discrete transaction with its own negotiation. There's no mechanism to pre-stage the next file while the current one is still writing. Non-async bulk transfer pipelining (similar to io_uring or Zero Copy USB) could theoretically reduce this, but it would require deep nusb-level changes and device-side support that most Android MTP stacks don't expose. MTP vs ADB: A F