RoamingPigs Research Notes

Record

Energy and speed per token on one mini PC: Vulkan against ROCm

Four local language models at four context lengths, three repeats each, on two software backends, measured at the wall plug and at the chip. I wrote the plan and my expectations down first and pushed them before the run.

MeasuredSelf-published, not peer reviewedOne machineRun paired4, 12 to 13 October 2026

The machine is an AMD Ryzen AI MAX+ 395 mini PC with 128 GB of memory. It replaces the energy figures I published earlier, which were derived from reported power and speed, not measured per token. Idle, the whole box draws 18 W at the wall. Across the 48 Vulkan cells, energy per generated token (wall plug, net of idle) is 1.7 to 4.8 joules (96 cells were measured in all, on the two backends). 7 of my 8 written expectations held.

What I found #

  • Speed. ROCm decoded more slowly than Vulkan in 16 of 16 cells, by 1 to 12%, and the repeat ranges of the two backends do not overlap in those cells. It was never faster.
  • Energy, 9B dense. ROCm used more energy per token than Vulkan: +8 to +20% across the four context lengths (wall plug, net of idle).
  • Energy, 12B dense. ROCm used about the same (the repeat ranges overlap in some cells) energy per token than Vulkan: -6 to +3% across the four context lengths (wall plug, net of idle).
  • Energy, 26B mixture. ROCm used about the same (the repeat ranges overlap in some cells) energy per token than Vulkan: -6 to -1% across the four context lengths (wall plug, net of idle).
  • Energy, 35B mixture. ROCm used less energy per token than Vulkan: -8 to -6% across the four context lengths (wall plug, net of idle).
  • Repeatability. Vulkan on the newer kernel, on another day, matched study 1 within 1.52% in every one of the 16 cells. Idle power was 17 W against 16.5 W.

Speed against context length #

Decode speed against context lengthTokens per second by model and context length. 9B dense, Vulkan: 64 37.8, 4k 37.2, 16k 34.7, 32k 31.6; 9B dense, ROCm: 64 37.0, 4k 36.1, 16k 33.9, 32k 31.3; 12B dense, Vulkan: 64 29.3, 4k 27.3, 16k 26.5, 32k 25.7; 12B dense, ROCm: 64 28.8, 4k 26.4, 16k 25.7, 32k 24.8; 26B mixture, Vulkan: 64 72.5, 4k 64.7, 16k 59.0, 32k 53.1; 26B mixture, ROCm: 64 66.6, 4k 58.4, 16k 55.1, 32k 51.0; 35B mixture, Vulkan: 64 54.3, 4k 52.7, 16k 49.2, 32k 44.7; 35B mixture, ROCm: 64 47.8, 4k 46.9, 16k 44.6, 32k 41.9Tokens per second, by context length (tokens already in the cache)VulkanROCm9B dense, 4-bit020406080644k16k32k3832373112B dense, 4-bit020406080644k16k32k2926292526B mixture, 4B active, 4-bit020406080644k16k32k7353675135B mixture, 3B active, 8-bit020406080644k16k32k54454842
Decode speed. Tokens per second as the prompt already in the cache grows from 64 to 32,768 tokens. The band is the lowest to highest of three repeats. All four panels share one scale.

Energy per token #

Energy per generated tokenJoules per token, wall net of idle. 9B dense, Vulkan: wall net 64 3.30, 4k 3.31, 16k 3.76, 32k 4.07; 9B dense, ROCm: wall net 64 3.86, 4k 3.96, 16k 4.18, 32k 4.41; 12B dense, Vulkan: wall net 64 4.55, 4k 4.73, 16k 4.76, 32k 4.80; 12B dense, ROCm: wall net 64 4.28, 4k 4.57, 16k 4.51, 32k 4.96; 26B mixture, Vulkan: wall net 64 1.67, 4k 1.87, 16k 1.99, 32k 2.23; 26B mixture, ROCm: wall net 64 1.57, 4k 1.82, 16k 1.96, 32k 2.16; 35B mixture, Vulkan: wall net 64 2.33, 4k 2.34, 16k 2.54, 32k 2.69; 35B mixture, ROCm: wall net 64 2.15, 4k 2.17, 16k 2.34, 32k 2.52Joules per generated token, net of idleVulkanROCmwall plugchip package01.22.43.64.869B dense9B dense, Vulkan, 64: wall 3.30 J/token9B dense, Vulkan, 4k: wall 3.31 J/token9B dense, Vulkan, 16k: wall 3.76 J/token9B dense, Vulkan, 32k: wall 4.07 J/token9B dense, Vulkan, 64: package 2.47 J/token9B dense, Vulkan, 4k: package 2.47 J/token9B dense, Vulkan, 16k: package 2.82 J/token9B dense, Vulkan, 32k: package 3.07 J/token4.079B dense, ROCm, 64: wall 3.86 J/token9B dense, ROCm, 4k: wall 3.96 J/token9B dense, ROCm, 16k: wall 4.18 J/token9B dense, ROCm, 32k: wall 4.41 J/token9B dense, ROCm, 64: package 2.86 J/token9B dense, ROCm, 4k: package 2.94 J/token9B dense, ROCm, 16k: package 3.10 J/token9B dense, ROCm, 32k: package 3.27 J/token4.4112B dense12B dense, Vulkan, 64: wall 4.55 J/token12B dense, Vulkan, 4k: wall 4.73 J/token12B dense, Vulkan, 16k: wall 4.76 J/token12B dense, Vulkan, 32k: wall 4.80 J/token12B dense, Vulkan, 64: package 3.42 J/token12B dense, Vulkan, 4k: package 3.58 J/token12B dense, Vulkan, 16k: package 3.58 J/token12B dense, Vulkan, 32k: package 3.58 J/token4.8012B dense, ROCm, 64: wall 4.28 J/token12B dense, ROCm, 4k: wall 4.57 J/token12B dense, ROCm, 16k: wall 4.51 J/token12B dense, ROCm, 32k: wall 4.96 J/token12B dense, ROCm, 64: package 3.14 J/token12B dense, ROCm, 4k: package 3.37 J/token12B dense, ROCm, 16k: package 3.31 J/token12B dense, ROCm, 32k: package 3.65 J/token4.9626B mixture26B mixture, Vulkan, 64: wall 1.67 J/token26B mixture, Vulkan, 4k: wall 1.87 J/token26B mixture, Vulkan, 16k: wall 1.99 J/token26B mixture, Vulkan, 32k: wall 2.23 J/token26B mixture, Vulkan, 64: package 1.22 J/token26B mixture, Vulkan, 4k: package 1.36 J/token26B mixture, Vulkan, 16k: package 1.45 J/token26B mixture, Vulkan, 32k: package 1.63 J/token2.2326B mixture, ROCm, 64: wall 1.57 J/token26B mixture, ROCm, 4k: wall 1.82 J/token26B mixture, ROCm, 16k: wall 1.96 J/token26B mixture, ROCm, 32k: wall 2.16 J/token26B mixture, ROCm, 64: package 1.14 J/token26B mixture, ROCm, 4k: package 1.34 J/token26B mixture, ROCm, 16k: package 1.44 J/token26B mixture, ROCm, 32k: package 1.60 J/token2.1635B mixture35B mixture, Vulkan, 64: wall 2.33 J/token35B mixture, Vulkan, 4k: wall 2.34 J/token35B mixture, Vulkan, 16k: wall 2.54 J/token35B mixture, Vulkan, 32k: wall 2.69 J/token35B mixture, Vulkan, 64: package 1.72 J/token35B mixture, Vulkan, 4k: package 1.75 J/token35B mixture, Vulkan, 16k: package 1.89 J/token35B mixture, Vulkan, 32k: package 2.01 J/token2.6935B mixture, ROCm, 64: wall 2.15 J/token35B mixture, ROCm, 4k: wall 2.17 J/token35B mixture, ROCm, 16k: wall 2.34 J/token35B mixture, ROCm, 32k: wall 2.52 J/token35B mixture, ROCm, 64: package 1.56 J/token35B mixture, ROCm, 4k: package 1.60 J/token35B mixture, ROCm, 16k: package 1.72 J/token35B mixture, ROCm, 32k: package 1.84 J/token2.52
Joules per generated token, net of idle. Filled dots are the wall plug, hollow dots the chip's own sensor, each from 64 tokens of context (small dot) to 32k (large). The chip sensor always reads lower than the plug, because it sees only the chip.

Where the watts go #

Where the wall watts go9B dense: 140 W at the wall, of which idle 18, extra at the package 91, elsewhere 31; 12B dense: 151 W at the wall, of which idle 18, extra at the package 100, elsewhere 33; 26B mixture: 130 W at the wall, of which idle 18, extra at the package 82, elsewhere 31; 35B mixture: 140 W at the wall, of which idle 18, extra at the package 90, elsewhere 32Idle (18 W)Added at the chipAdded elsewhere in the box0 W40 W80 W120 W160 W200 W9B dense140 W12B dense151 W26B mixture130 W35B mixture140 W
While decoding (Vulkan, 64 tokens of context). The whole box draws about 18 W idle. The chip package accounts for most of what is added. The rest is everything else in the box that the chip sensor does not see.

How it was measured #

Power during one measured cell: 26B mixture, 4k contextWall power and package power over 174 seconds; idle baseline 18 W;03060901201500 s60 s120 sidle baseline 18 Wmeasured windowwall plug (whole box)chip package sensorWatts against time. Net energy = area above the dashed line, inside the window, divided by tokens generated.
One measured cell, raw. The plug is read about once a second and the chip sensor ten times a second. Each dip is the gap between one 512-token request and the next. Energy per token is the area above the idle line inside the window, divided by the tokens generated in it.

ROCm against Vulkan #

ROCm against VulkanDecode speed ratio. 9B dense 64: 98% (real); 9B dense 4k: 97% (real); 9B dense 16k: 98% (real); 9B dense 32k: 99% (real); 12B dense 64: 98% (real); 12B dense 4k: 97% (real); 12B dense 16k: 97% (real); 12B dense 32k: 96% (real); 26B mixture 64: 92% (real); 26B mixture 4k: 90% (real); 26B mixture 16k: 93% (real); 26B mixture 32k: 96% (real); 35B mixture 64: 88% (real); 35B mixture 4k: 89% (real); 35B mixture 16k: 91% (real); 35B mixture 32k: 94% (real)644k16k32k9B dense98%97%98%99%12B dense98%97%97%96%26B mixture92%90%93%96%35B mixture88%89%91%94%
ROCm decode speed as a share of Vulkan. A plain number means the repeat ranges of the two backends do not overlap. A dotted outline means they overlap, so I make no claim. ROCm here is the 7.1 runtime packaged in Ubuntu 26.04.

Plan and expectations #

Each line below was written before the run and is reported whether it held or not. The first study's six expectations also held on both backends. Several of today's lines are not independent predictions, because I had seen the first study's results and one smoke test before writing them; the plan says so.

Expectations written in advanceP1 Vulkan on kernel 38 within 3% of study 1 in every cell: held; P2 ROCm within 25% of Vulkan; slower for the 26B mixture at 64: held; P3 ROCm gap shrinks or reverses with context for some model: held; P4 Idle under 25 W and within 2 W of study 1, both backends: held; P5 Wall power within 15% between backends, same cell: wrong; P6 Models rank the same by energy per token in both backends: held; P7 Repeat spread under 5% in every cell: held; P8 All 96 cells, 3 repeats; parse error only with the 26B mixture: heldP1Vulkan on kernel 38 within 3% of study 1 in every celllargest difference 1.52%heldP2ROCm within 25% of Vulkan; slower for the 26B mixture at 64ROCm at 88 to 99% of VulkanheldP3ROCm gap shrinks or reverses with context for some modelheldP4Idle under 25 W and within 2 W of study 1, both backendsidle 17 W Vulkan, 17 W ROCm, 16.5 W in study 1heldP5Wall power within 15% between backends, same cellROCm wall power 87 to 117% of VulkanwrongP6Models rank the same by energy per token in both backendsheldP7Repeat spread under 5% in every celllargest spread 1.26%heldP8All 96 cells, 3 repeats; parse error only with the 26B mixture12 requests recovered after the parse error, 0 failed cells, 0 contaminatedheld
Wrong: P5. The server answered HTTP 500 on 12 requests with one model, after a reply containing bytes that are not valid text. The harness took those timings from the server's own log and the report counts them.

All the numbers #

Vulkan: mean over three repeats. Energy is net of idle.
ModelContexttok/sWall WChip WWall J/tokenChip J/tokenRepeats
9B dense, 4-bit6437.8140953.32.473
9B dense, 4-bit4k37.2139943.312.473
9B dense, 4-bit16k34.71461003.762.823
9B dense, 4-bit32k31.6145994.073.073
12B dense, 4-bit6429.31511044.553.423
12B dense, 4-bit4k27.31471014.733.583
12B dense, 4-bit16k26.5145994.763.583
12B dense, 4-bit32k25.7141964.83.583
26B mixture, 4B active, 4-bit6472.5130861.671.223
26B mixture, 4B active, 4-bit4k64.7126831.871.363
26B mixture, 4B active, 4-bit16k59.0124811.991.453
26B mixture, 4B active, 4-bit32k53.1125832.231.633
35B mixture, 3B active, 8-bit6454.3140942.331.723
35B mixture, 3B active, 8-bit4k52.7138932.341.753
35B mixture, 3B active, 8-bit16k49.2140942.541.893
35B mixture, 3B active, 8-bit32k44.7135912.692.013
ROCm: mean over three repeats. Energy is net of idle.
ModelContexttok/sWall WChip WWall J/tokenChip J/tokenRepeats
9B dense, 4-bit6437.01611093.862.863
9B dense, 4-bit4k36.11631103.962.943
9B dense, 4-bit16k33.91611094.183.13
9B dense, 4-bit32k31.31571064.413.273
12B dense, 4-bit6428.8142944.283.143
12B dense, 4-bit4k26.4140924.573.373
12B dense, 4-bit16k25.7134894.513.313
12B dense, 4-bit32k24.8142944.963.653
26B mixture, 4B active, 4-bit6466.6123801.571.143
26B mixture, 4B active, 4-bit4k58.4125821.821.343
26B mixture, 4B active, 4-bit16k55.1127831.961.443
26B mixture, 4B active, 4-bit32k51.0129852.161.63
35B mixture, 3B active, 8-bit6447.8121782.151.563
35B mixture, 3B active, 8-bit4k46.9122792.171.63
35B mixture, 3B active, 8-bit16k44.6124802.341.723
35B mixture, 3B active, 8-bit32k41.9125812.521.843

What this does not show #

  • Anything about other hardware, other chips, discrete GPUs, other ROCm versions or packaging, or other builds of llama.cpp.
  • The accuracy of the wall watts. The plug is uncalibrated and reads whole watts; its own energy counter and the sum of its readings differ by 1.7%, but both come from the same plug.
  • Batched or concurrent serving. Everything is one stream at a time.
  • Quality of the output. The prompts are licence texts and the replies are not scored.
  • That anyone outside this project has repeated it.

Three earlier starts of this run were stopped after a few cells each because outside load reached the machine (requests from an unknown caller on the network, backup pushes from two other computers, a backup job on the machine itself). The last start had no failed cell and no contaminated cell. The whole account is in the repository's DEVIATIONS.md.

Check it #

The plan, the harness, the raw wall-power and chip-power logs, every deviation from the plan and a standalone version of this report with a PDF are in github.com/RoamingPigs/strix-halo-energy. The files this page draws on are copies with a SHA-256 for each, taken from commit d82510a. Corrections to earlier energy figures are in the corrections log. Self-published by Cisco Caceres. No DOI exists.