NVIDIA Nemotron Bolt 74B vs Qwen3.8-27B: What I Would Run on a 32 GB GPU

If I want fast responses or need a lot of room for a 256K context, I’d start with NVIDIA’s Nemotron Bolt 74B. If complete citations or applied work with thinking off matter more, I’d start with Unsloth’s Qwen3.8-27B UD-Q6_K. On the paired coding problems, the quality result was tied, so my choice there would come down to speed and the rest of the workflow.

I tested both on my RTX 5090 with 32 GB of VRAM. Bolt decoded about 2.5 times faster on the native SPEED-Bench coding workload. That didn’t make it the better model for every job. The answer changes with the task, the thinking budget, and how much context I need.

Two models that use their memory differently

I used NVIDIA’s Nemotron 3.5 Bolt 74B-A9B 24GB GGUF and the Unsloth Qwen3.8-27B UD-Q6_K GGUF. Their main model files are within 0.16% of each other in size, but the designs are quite different.

Bolt has 74 billion parameters in total and about 9 billion active per token. It’s a mixture-of-experts model with a hybrid Mamba-Transformer design. Its GGUF is mixed precision, with 2-bit experts; other parts use NVFP4 or Q8_0. Qwen3.8-27B is a dense model, so all 27 billion parameters participate in each token. Its UD-Q6_K file uses a dynamically selected mix of quantization types.

Those file sizes don’t include the full memory cost of context, cache, or an optional draft model. The amount of VRAM left after loading matters just as much as whether the model starts.

My test setup and how to read the comparisons

Both models ran on the same RTX 5090 32 GB workstation with an Intel i9-13900K, 128 GB of DDR5 memory, and Windows 11 Pro. I used llama.cpp b11669, the Windows CUDA 13.4 build, and NVIDIA driver 617.14. The shared native quality profile used a 64K context, Q8 K/V cache, flash attention, one request at a time, and each model’s own chat template. The long-context, HH-2.0, and speed tests used their stated profiles.

I tested thinking off and each vendor’s maximum-quality thinking configuration, which I call xHigh. Those maximum-quality modes aren’t equal compute. Qwen also uses different samplers with thinking on and off, so changes between its modes are descriptive rather than an isolated measurement of thinking alone. HH-2.0 used its own frozen profile: temperature zero and thinking off.

For quality comparisons I report each test separately. A result is tied when its paired 95% interval includes or touches zero; that doesn’t prove the models are identical. My speed tests measure serving performance, not whether an answer was correct.

Qwen led on thinking-off applied work and citations

On Evidence-to-Action, which tests grounded decisions and citations, Qwen was ahead with thinking off: 73.1% decision accuracy versus Bolt’s 42.0%. Complete citations were part of the job, and Bolt often missed the policy citation supporting a decision.

With thinking on, the official output cap was 16,384 tokens. Bolt exhausted that budget on three of 18 requests. A separate 32K-cap diagnostic allowed those requests more room to finish: both models reached 98.5% decision accuracy, a tie, while Qwen remained ahead on citation accuracy, 96.0% versus Bolt’s 86.1%. This follow-up explains what happened under a larger budget; it doesn’t replace the official result.

QuantBench covers applied tasks with partial credit. With thinking off, I ran the same 50 questions across five seeds. Qwen averaged 57.4, ahead of Bolt’s 44.4. With thinking on, the separate single pass was tied at 80.6 for Qwen and 71.6 for Bolt. The remaining thinking-on passes weren’t run, so I wouldn’t treat that single pass like the five-pass result.

Compass knowledge tests were tied. ShellGame state tracking and SimonSays instruction following were tied in their paired comparisons too. The tests still have different scopes and modes, so I wouldn’t collapse them into one score.

Mirage needs its formatting caveat

Bolt’s official thinking-off Mirage neutral score was 8.7 versus Qwen’s 82.8. Bolt frequently copied the available verdict options instead of selecting one. That formatting failure affected the score, so 8.7 isn’t a measured hallucination rate. Both official thinking-off Mirage comparisons favor Qwen, but are format-confounded.

In a post hoc lower-bound diagnostic, the echoed verdict line was normalized in copies of the saved responses and the same grader was used. On the neutral condition, Bolt scored 71.2 versus Qwen’s 82.8. Qwen remained ahead overall, while resistance to fabricated questions was tied. The normalized result is a diagnostic, not the official score.

With thinking on and no normalization, the neutral scores were 77.7 for Bolt and 82.5 for Qwen, with Qwen ahead. Under pressure, the 69.5 versus 74.2 comparison was tied. If you use Bolt with thinking off in a workflow that depends on structured output, I’d validate the format against your own application.

Coding quality tied on the shared LiveCodeBench problems

The small Ops repair screen was tied at six out of eight for each model. The extended LiveCodeBench run compared the first 70 problems both models completed in frozen order: Bolt solved 47 and Qwen solved 46. The paired result was tied overall and at each difficulty.

The cutoff was set by time before grading, and the first 70 aren’t a random sample. Most misses came from exhausting the 32K output budget while still thinking: 17 for Bolt and 19 for Qwen. The v6 pool spans January through April 2025, including 44 January problems, so age and possible contamination matter when interpreting a coding benchmark.

For this tested workload, I’d try Bolt with its qualified LiLiCorr drafter when I value a faster coding loop. I’d still run my own tests on the generated code. The quality comparison doesn’t establish a coding winner, and speculative decoding is a speed setting, not a proven accuracy improvement.

At 256K, Bolt had more headroom and led HH-2.0

HH-2.0 tests finding and using information in long inputs. In the primary six-haystack run, Qwen was ahead at 32K, the models were tied at 64K and 128K, and Bolt was ahead at 256K: 79.4 versus 65.2.

A later post hoc eight-haystack check kept the same verdicts. That extension was chosen after the six-haystack result was seen, so I treat it as a robustness check rather than an independent replication. At 128K, Qwen’s drop on the two added haystacks involved a format failure; that tier remains tied.

Both models fit the 256K text-only test on my GPU, but with very different margins. Bolt had 8.1 GiB of free VRAM. Qwen passed my 1 GiB headroom target at a micro-batch size of 256, with 1.04 GiB free; the 512 micro-batch configuration didn’t meet that target. That’s a narrow margin for Qwen on this workstation. Your desktop, runtime, cache, and other GPU allocations can change what fits.

Bolt was faster in both speed tracks

SPEED-Bench coding	Bolt	Qwen3.8-27B
Native decoding	154.4 tokens/s	62.7 tokens/s
Both accelerated	347.9 tokens/s	127.1 tokens/s

The native comparison is about 2.5 times faster for Bolt. With speculative decoding enabled for both models, Bolt used LiLiCorr at draft length seven and Qwen used DFlash2 at five. That comparison is about 2.7 times faster for Bolt. Speculative decoding uses a smaller draft model to propose tokens for the target model to verify. It adds memory and configuration requirements, and native and accelerated rates should be compared within their own tracks.

Native conversation time was 14.6 seconds for Bolt versus 73.9 for Qwen. The larger time difference also reflects output length: Bolt averaged 2,221 tokens per conversation and Qwen 4,608. These are serving measurements, capped turns are included, and the content wasn’t graded for correctness.

NVIDIA reported 308.35 tokens/s for its experimental RTX 5090 Bolt recipe. My recipe replication reached 361.7 tokens/s with draft length six, the projector loaded, and the benchmark client sending temperature zero. The driver differed, and I couldn’t confirm NVIDIA’s historical weight digest, so I wouldn’t call it an identical-weights reproduction.

Settings I’d check before running either model

Use a runtime that supports the exact model and keep its native chat template. Explicitly select the thinking mode, reserve VRAM for context and any drafter, and check request-level settings as well as server settings. An application can override its server. Thinking and the final answer share the output budget, so leave space for both.

For Bolt, NVIDIA’s recipe specifies temperature 1.0 and top-p 0.95. My runs used top-k 40 and min-p 0.05 because those were the llama.cpp defaults inherited by the recipe; they aren’t NVIDIA recommendations. My higher-effort runs used enable_thinking=true and medium_effort=false. Based on the larger-budget diagnostic, I’d plan for a 24K output budget or more for Bolt’s maximum-quality mode, while remembering that even 32K stopped many hard coding requests for both models.

For Qwen with thinking on, I used temperature 1.0, top-p 0.95, top-k 20, min-p 0, presence penalty 0, and repetition penalty 1.0, with reasoning_effort=xhigh, enable_thinking=true, and preserve_thinking=true. With thinking off, I used temperature 0.7, top-p 0.8, top-k 20, min-p 0, presence penalty 1.5, and repetition penalty 1.0. Check Qwen’s best practices and Unsloth’s setup guide for your runtime.

For the optional accelerated profiles, the files I used are Bolt LiLiCorr and Qwen DFlash2. NVIDIA also provides its Windows/GeForce Bolt recipe.

What this comparison doesn’t settle

This is one workstation, two specific GGUF builds, and the listed profiles. I didn’t score multi-step agentic tool use or vision. AgentPerf replay measures serving latency for recorded agent-style traffic; it doesn’t establish whether an agent completed a task. The extended run also didn’t complete thinking-on Ops, thinking-on SimonSays, the remaining thinking-on QuantBench passes, the BF16 comparison, or NVIDIA’s Qwen comparator on this GPU.

Bolt’s files and llama.cpp support were released on October 15, 2026. Later builds may change speed or behavior. Neither the vendor-recommended modes nor the different quantizations isolate architecture as the cause of a result.

For my 32 GB card, I’d start with Bolt for speed and very long inputs, and Qwen for citation-heavy work and the tested thinking-off applied tasks. In my Bolt versus Qwen video, I walk through the results and the caveats. Tell me your GPU, model build, settings, and workload in the YouTube comments, and subscribe to DeepWakeLabs for more local AI testing.

Explore the recorded benchmark profiles. Each profile keeps its workload, mode and limits attached.