Science explained from zero

Why home AI reads fast and writes slow

Same machine. Same model. Two jobs that pull in opposite directions.

vizmaya desk · 3 November 2026

What happens when one computer has to do both?

Source: arXiv:2610.39216v1 (preprint, 29 Oct 2026), p.1

An NVIDIA A100 data-centre graphics card, a long black board with a heat sink — Photo: Nvidia / Wikimedia Commons, CC BY-SA 4.0. The A100 is one of the cards the authors tested.

The setup

A big AI meets a small graphics card

A graphics card (GPU) does the fast math. Its memory is small. In this study it was capped at 24 GB, and only 5.8 to 8.7 GB was left for the model's parts.

The rest waits in the computer's normal memory (RAM), like the stick below.

The model has a lot of parts. Do all of them work at once?

Source: arXiv:2610.39216v1, p.1 (memory problem) and p.7 (24 GB cap, 5.8 to 8.7 GB left)

The model

Only a few parts work on each word

This design is called Mixture of Experts. Each layer holds many small specialists, called experts. For every word, it calls only a few.

Mixtral: 2 of 8. Qwen3: 8 of 128. DeepSeek: 6 of 64.

Which few? The model decides layer by layer, as the word passes through.

Source: arXiv:2610.39216v1, Table 4 (p.8) and §2.1 (p.2)

The catch

Parts in memory must cross a narrow cable

Experts reach the GPU over a link called PCIe. At top speed, one DeepSeek expert needs 0.53 ms. One Mixtral expert needs 10.67 ms.

Does the cable hurt every job the AI does?

Source: Table 1 (p.4) for the limits. Expert size: Mixtral 336 MB vs DeepSeek 16.5 MB, Table 4 (p.8)

Several long PCI Express slots on a computer motherboard, next to an older, shorter PCI slot — Photo: snickerdo / Wikimedia Commons, CC BY-SA 3.0. PCI Express slots on a 2005 board. Same kind of link, older speed.

Two jobs

Reading and writing are different jobs

Reading your prompt is called prefill. The AI handles many words at once. Writing the answer is called decode. It makes one word at a time.

Illustration

Does the traffic jam stay in the same place?

Source: arXiv:2610.39216v1, pp.1 to 3

The first crack

The traffic jam moves

Reading, the CPU is busiest: 2557 ms against 292 ms for the GPU. Writing, the cable is: 1449 ms against 109 ms.

Contested: span of totals not stated

Can one plan fix both?

Source: Fig. 3 (p.3), Mixtral-8x7B on an A100 with the HybriMoE scheduler. Read the shape, not the exact times.

The seesaw

Every old fix helps one job and hurts the other

Keep experts on the GPU: writing is smooth, reading wastes the GPU. Load experts on demand: reading is fast, writing stalls. The authors call this the seesaw.

Is there a way off the seesaw?

Source: arXiv:2610.39216v1, p.2 and §3.2 (p.3), Figs. 1 and 2

The second crack

The cable is slower than its own speed limit

3.68×

slower than the limit, in a clean test (computed: 1.95 ms / 0.53 ms)

One DeepSeek expert took 1.95 ms. The limit is 0.53 ms. Busy reading a prompt, it took 2.78 ms.

And what about the math itself?

Source: Table 1 (p.4), DeepSeek-V2-Lite, PCIe 4.0 x16

The third crack

Running one expert takes longer than its math

0.21ms of math

0.43ms in total

This is one DeepSeek expert on an Intel server chip. Set-up work eats the rest. Across nine tests, total time is 1.14 to 2.13 times the math (computed).

What if a system planned around these hidden costs?

Source: Table 2 (p.4). Photo shows a different Intel Xeon model from the paper's.

The turn

Guess ahead. Plan each job on its own.

A small guesser for each layer predicts which experts come next, so the cable starts moving them early. Reading gets heavy loading. Writing gets very light loading.

Diagram, not measured

How good can the guesses be?

Source: arXiv:2610.39216v1, p.2, pp.4 to 6, and p.10 (one small guesser per layer)

The test

The guesser scores 90 to 100 percent

That is the paper's Top-6 hit score on four models and three kinds of text. A guess within 2 places counts as a hit. On new kinds of text, it drops 4.8 to 5.3 percent on average.

Contested: lenient scoring rule

So how much faster does it all run?

Source: Table 5 and §5.5 (p.10). No random-guess score is given for this rule.

The result

Up to 10.3 times faster. Against the closest rival, 2.5.

Best cases, measured by the authors against five rivals. Reading: 10.3 times against Fate, 2.5 against MoE-APEX. Writing: 6.1 against Fate, 2.1 against HybriMoE and MoE-APEX.

Where did that speed come from?

Source: arXiv:2610.39216v1, p.8 (reading) and p.9 (writing). "Up to" means best case.

The twist

The biggest gain was the plumbing

The authors added parts one at a time. Faster copying gave 1.34 to 2.24 times in writing. The clever guesser and planner together gave 1.17 to 1.38 times.

In reading, the two ranges overlap: 1.67 to 1.88 against 1.39 to 2.06. Each step is measured on top of the ones before it.

So what should a reader take from this?

Source: §5.4 and Fig. 12 (p.9). Baseline: Fiddler on an A100.

The payoff

How to read a speed claim

Find the "up to"

10.3× was one best case. The closest rival's best case was 2.5×.

p.8

Ask which chip

Moving from the 3090 box to the A100 box sped up KTransformers 3.23× in reading, PDScope 1.88×.

p.8

Ask where time goes

In Fig. 3, writing spent 1449 ms on the cable and 109 ms on the GPU. A faster GPU alone may not touch that.

p.3. Our reading, from one test.

One thing left: where does this break?

Source: arXiv:2610.39216v1, p.3 (Fig. 3) and p.8

The limits

Where it breaks, and who else was there

Best cases, run by the authors on two lab machines.
Built for small batches. Reading and writing never overlap.
The guess score allows a 2 place slack.

Contested: novelty

Who stands behind these numbers?

Source: pp.7 to 11 for limits. Related papers: arXiv 2509.07379, 2509.23638, 2610.01950. Not in the paper's reference list (pp.11 to 12).

Credit and sources

Who did the work

The paper

Enda Yu, Dezun Dong, Zhaoning Zhang, Xiaojian Yang, Weiling Yang, Shaonan Ma, Dongsheng Li, Yongwei Wu, Xiangke Liao. "PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE Inference." arXiv:2610.39216v1, 29 Oct 2026. Preprint, no venue named. National University of Defense Technology and Qiyuan Laboratory, China.

Method

Every number comes from the paper's pages, tables and Fig. 3 or 12. "Computed" figures: 3.68× (1.95 / 0.53) and 1.14 to 2.13× (Table 2). No outside citation record or replication found.

Related papers

DuoServe-MoE, PreScope, MoE-CORE.

Photos

A100: Nvidia, CC BY-SA 4.0. RAM: PantheraLeo1359531, CC BY 4.0. PCIe slots: snickerdo, CC BY-SA 3.0. Seesaw: Killarnee, CC BY-SA 4.0. Bridge: Rick and Lea Miller, CC BY-SA 2.0. CPU: Dsimic, CC BY-SA 4.0.

Byline

vizmaya desk, 3 November 2026. More stories on vizmaya.fyi.

Source: research file 2026-11-03_landmark_enda-yu.md