Benchmarks
Setup
- Laptop: ASUS ProArt PX13, AMD Radeon 8060S, Performance mode at about 70 W, Windows 11. 23 October to 2 November 2026.
- Models: Huihui Qwen3.8 Flash Next, UD-Q4_K_XL, is the file that every engine ran. Bubo also ran three ISTA-DASLab files: IQ3_S (55 GB), IQ3_XXS and IQ2_XS (39 GB). The other engines were pointed at the published IQ3_S and IQ2_XS files as they are.
- Bubo 0.1.2 on UD-Q4_K_XL was measured on 1 November with the default prefill mode. Bubo 0.1.0 (3ff74dce) ran on the same day with the same driver and prompts. The ISTA rows, the 100K row and the fast-mode gains are from the final Bubo 0.1.2 build on 2 November. The IQ2_XS rows and the fast-mode gains on UD-Q4_K_XL and IQ2_XS are the mean of two starts of each mode. The IQ3_XXS rows and their fast-mode gain, the Rulith rows and the other 2 November rows are single starts.
- The other engines, as measured on 2 November, each started once per cell, in the same slot as a Bubo start:
- gufo Windows 0.24.0, build b72e01b. Prebuilt release, with MTP on.
- gufo-win 0.5.0, release v0.5.0. Prebuilt release, started with its launcher’s arguments and recommended English options.
- Rulith Inference 0.6.10, its own model file. Its app’s default profile, on its own model file: EXL3, 4.05 bits per weight.
- Rows marked “sampled” use temperature 1.0, top-p 0.95, top-k 20, thinking on, medium effort. Edit rows are greedy.
- All speeds are tokens per second. Higher is better. Writing speed is measured at the client for every engine: the number of tokens minus one, divided by the time from the first to the last token.
Results
Bubo’s speeds and the “vs Bubo 0.1.0” column are from the runs named above. Each engine column divides Bubo’s speed in the same 2 November slot by that engine’s speed in that slot. The table shows no speed for the other engines, only how much faster Bubo is. “Level” means the ratio is between 0.97 and 1.03, inside the spread between two starts of the same engine. The column “with fast mode” shows how much faster Bubo is with Fast prefill mode than in normal mode, in the same run, with two starts of each. “Same” means that the row has fewer than 4,096 new tokens, or that it measures writing speed, and fast mode changes nothing there.
Huihui UD-Q4_K_XL
The model file that every engine ran.
| Row | Setting | Bubo 0.1.2 | gain with fast mode | vs Bubo 0.1.0 | Bubo is faster than | ||
|---|---|---|---|---|---|---|---|
| gufo Windows 0.24.0 | gufo-win 0.5.0 | Rulith Inference 0.6.10* | |||||
| Prose | sampled | 57.1 | same | +9 % | +24 % | +5 % | +7 % |
| Python code | sampled | 66.3 | same | +4 % | +17 % | +10 % | +13 % |
| C++ code | sampled | 77.5 | same | +11 % | +52 % | +35 % | +42 % |
| Six-prompt mix (agent, code, prose) | sampled | 65.6 | same | +12 % | +26 % | +19 % | n/a |
| Long agent session, writing (10 turns, 31K to 37K) | sampled | 66.2 | same | +7 % | +33 % | +17 % | n/a |
| Long agent session, reading new text | sampled | 971 | same | level | see below | see below | n/a |
| Code edit, rename | greedy | 109.0 | same | +12 % | +147 % | level | +55 % |
| Code edit, comment change | greedy | 97.6 | same | +7 % | +127 % | −4 % | +46 % |
| Cold 32K prompt | natural text | 1,415 | +16.8 % | +6 % | level | level | n/a |
| Cold 100K prompt | natural text | 1,384 | +16.5 % | not measured | not measured | not measured | |
| Cold 2K prompt | natural text | 1,320 | same | +6 % | +4 % | +6 % | +22 % |
ISTA-DASLab IQ3_S, 55 GB
The ISTA file that scores the same as full precision on ISTA’s benchmarks. The other engines were pointed at the published file as it is.
| Row | Setting | Bubo 0.1.2 | Bubo is faster than | |
|---|---|---|---|---|
| gufo Windows 0.24.0 | gufo-win 0.5.0 | |||
| Prose | sampled | 58.7 | does not load | does not load |
| Python code | sampled | 66.9 | does not load | does not load |
| C++ code | sampled | 70.2 | does not load | does not load |
| Six-prompt mix (agent, code, prose) | sampled | 63.7 | does not load | does not load |
| Long agent session, writing (10 turns, 31K to 37K) | sampled | 64.9 | does not load | does not load |
| Long agent session, reading new text | sampled | 813 | does not load | does not load |
| Code edit, rename | greedy | 106.9 | does not load | does not load |
| Code edit, comment change | greedy | 97.2 | does not load | does not load |
| Cold 32K prompt | natural text | 1,229 | does not load | does not load |
| Cold 100K prompt | natural text | 1,198 | does not load | does not load |
| Cold 2K prompt | natural text | 1,078 | does not load | does not load |
gufo Windows 0.24.0 does not load this file. Its message: “Failed to open GGUF: Invalid GGUF tensor shape or unsupported storage type: blk.0.ffn_gate_exps.weight”.
gufo-win 0.5.0 does not load this file. Its message: “Failed to open GGUF: Invalid GGUF tensor shape or unsupported storage type: blk.0.ffn_gate_exps.weight”.
ISTA-DASLab IQ3_XXS
Bubo only. The other engines were not run on this file.
| Row | Setting | Bubo 0.1.2 | gain with fast mode |
|---|---|---|---|
| Prose | sampled | 64.6 | same |
| Python code | sampled | 70.0 | same |
| C++ code | sampled | 71.9 | same |
| Six-prompt mix (agent, code, prose) | sampled | 70.0 | same |
| Long agent session, writing (10 turns, 31K to 37K) | sampled | 68.6 | same |
| Long agent session, reading new text | sampled | 814 | same |
| Code edit, rename | greedy | 115.4 | – |
| Code edit, comment change | greedy | 105.5 | – |
| Cold 32K prompt | natural text | 1,225 | +16.8 % |
| Cold 2K prompt | natural text | 1,094 | same |
ISTA-DASLab IQ2_XS, 39 GB
The smaller ISTA file. The other engines were pointed at the published file as it is.
| Row | Setting | Bubo 0.1.2 | gain with fast mode | Bubo is faster than | |
|---|---|---|---|---|---|
| gufo Windows 0.24.0 | gufo-win 0.5.0 | ||||
| Prose | sampled | 65.3 | same | does not load | does not load |
| Python code | sampled | 78.0 | same | does not load | does not load |
| C++ code | sampled | 77.8 | same | does not load | does not load |
| Six-prompt mix (agent, code, prose) | sampled | 73.4 | same | does not load | does not load |
| Long agent session, writing (10 turns, 31K to 37K) | sampled | 68.3 | same | does not load | does not load |
| Code edit, rename | greedy | 131.4 | same | does not load | does not load |
| Code edit, comment change | greedy | 118.7 | same | does not load | does not load |
| Cold 32K prompt | natural text | 1,170 | +17.1 % | does not load | does not load |
| Cold 100K prompt | natural text | 1,147 | +16.5 % | does not load | does not load |
| Cold 2K prompt | natural text | 1,136 | same | does not load | does not load |
gufo Windows 0.24.0 does not load this file. Its message: “Failed to open GGUF: Invalid GGUF tensor shape or unsupported storage type: blk.0.ffn_down_exps.weight”.
gufo-win 0.5.0 does not load this file. Its message: “Failed to open GGUF: Invalid GGUF tensor shape or unsupported storage type: blk.0.ffn_down_exps.weight”.
* Rulith ran its own model file, not the file the other engines ran. “n/a”: Rulith produced no result on the agent-session prompts, so the agent-session rows, the six-prompt mix and the first 32K request have no result for it. On the frozen 32K prompt body, Bubo reads 13 % faster than Rulith in the same slot (1,405 against 1,241 tok/s).
“See below”: the engines count the cached part of a conversation differently. Over turns 1 to 9 of the agent session, Bubo read 6,779 new tokens, gufo-win 0.5.0 read 6,761 and gufo 0.24.0 read 10,489, so a speed per second would not compare. The wait that matters is below.
“Not measured”: at a prompt of about 100K tokens, the free host memory fell below the 2.5 GiB limit of the test harness during the run of these engines, so the row has no result for them.
gufo-win 0.5.0 reports its own speed from its server, and its server rate and the client rate differ (agent session: 67.8 and 54.7 tok/s). The table uses the client rate for every engine.
Reasoning check for fast mode: on 24 GPQA-Diamond questions with verbose answers, fast mode and normal mode both answered 21 correctly. It was measured on 28 October, with an earlier build that applied fast mode to every prompt.
Waiting for the first token
Total wait for the first token, summed over turns 1 to 9 of the ten-turn agent session, in seconds:
| Engine | Seconds |
|---|---|
| Bubo 0.1.2 | 8.8 |
| gufo Windows 0.24.0 | 13.7 |
| gufo-win 0.5.0 | 15.7 |
Other readings, Bubo 0.1.1
| Reading | tok/s | Setting |
|---|---|---|
| 2K prompt, benchmark tool, first run | 1,530 | bubo bench, empty context |
| 2K prompt, through the server | 1,384 to 1,403 | sustained, plain and with drafting |
| Eight users at 40K each, total | 99.1 | server, drafting off |
Power
From 70 W to 95 W, 2K prompt speed rose 12 % and generation 2.4 % on this laptop. At 140 W, a desktop-class limit, the estimate is 1.2x to 1.3x for short prompts and 1.04x to 1.07x for generation. These are estimates, not measurements.
Older results
The results of 0.1.0 (12 October 2026) stay on the 0.1.0 release page.