2x R9700, 64 GB DDR5 is an absolute beast machine with vLLM Radiance / R9V and Qwen 3.8 27b and Flash next

I've been tinkering with local LLMs since the beginning of the year when I had an Intel Arc B580 and 32 GB of DDR5. Curiosity got the best of me and I bought the first R9700 about half a year ago, also because I wanted to upgrade my gaming graphics for 4k. As the 5090 was about 3 times as expensive, I had a "sweet spot", kind of. On the last prime days, I found a X870E mainboard for ~150 € below the standard price, and it got to my head that I can use an upgraded machine for gaming and local inference tinkering. Anyways. Fast forward to this week, I now have the following setup Ryzen 7500F 64 GB DDR5 CL40 6400 MT/s Asus ProArt Creator X870E 2x R9700 32 GB, each running at PCIe 5.0 x8 (Gigagbyte) Currently running ubuntu on an old Samsung EVO 860 1 TB drive; this will become intersting for the ngram / PLE offload; I have Windows and the gaming related stuff on a gen4 NVMe, but will soon add another Gen 5 NVMe with decent random reads The only issue that I can report so far is that one of the cards runs quite hot, so I will definitely implement power limiting to 210 W and some light undervolting. The other card runs 10-15 °C cooler.. Case is a purebase 501 with 4 fans, 2 intake in front, one back and top for output. Now long story short I wanted to give some results of Qwen 3.8 27b FP8 and MXFP4, as well as Qwen 3.8 flash next after the first day tinkering with it. What I found super interesting is that the SATA SSD does not seem to be super terrible when using Qwen 3.8 flash next. Considering the whole build costs ~4k €, or more than 1k less than a single RTX 5090 with 32 GB, I kinda like this setup price/performance wise. Next step is checking context degradation / KV quants. I am using local inference mostly for deep research, summarization, image creation, light coding and non-trivial data analysis Cheers Qwen3.8 benchmarks on 2× Radeon AI PRO R9700 Hardware: 2× AMD Radeon AI PRO R9700 32 GB, 61 GiB system RAM Benchmark: BetterBench 0.2.2, corpus v1.0, single-stream, greedy decoding, 2 warm-ups + 10 measured runs per category, 8k benchmark context. Model | Weight format | Runtime | Server context | Max sequences | Speculative decoding | Weighted decode median | ITL 1% low | TTFT p50 | Prefill ~2k | Prefill ~4k | Prefill ~7k Qwen3.8-27B |Quark AWQ MXFP4 |vLLM Radiance, TP2 |131,072 |1 |MTP, up to 8 tokens |111.4 tok/s |77.9 tok/s |81 ms |4,224 tok/s |4,322 tok/s |4,410 tok/s Qwen3.8-27B |Native block FP8 |vLLM Radiance, TP2 |16,384 |8 |MTP, up to 8 tokens |87.6 tok/s |61.9 tok/s |73 ms |4,134 tok/s |4,329 tok/s |4,305 tok/s Qwen3.8-Flash-Next |UD-IQ4_XS GGUF |R9V/vLLM, TP2, tiered expert offload |131,072 |1 |MTP, 2 tokens, FP8 draft |35.4 tok/s |27.3 tok/s |290 ms |1,727 tok/s |1,986 tok/s |1,925 tok/s Notes Qwen 3.8 27b in FP8 and AWQ MXFP4 served with vLLM Radiance Qwen 3.8 Flash next served with vLLM / R9V fork Decode metrics come from the 10-pass standard run. Prefill measurements use cold, nonce-prefixed prompts. Prompt-token medians for the prefill columns were 1,556, 3,024 and 5,226 tokens. No concurrency sweep was included in these results. I expect decode of Flash next to increase a bit when an NVMe is used, and, as I am writing this and checked, I found EXPO was not enabled........oh my god I swear I turned it on when I updated the bios yesterday