Local LLMs / VIDEO COMPANION

KV Cache Quantization: What I Gained in Memory and What Changed in the Answers

8,000 graded answers on Qwen3.8-27B and Mistral Small 3.2: cache savings, answer churn, practical context and a build-specific mixed-cache slowdown.

By Casey12 min read

BENCHMARK EDITIONSPaired synthetic retrieval; 7,800 core answers plus 200 MTP-off; separate guarded capacity checks

Video November 4, 2026Source review November 5, 2026

Test capture & runtime

Tests captured: 14 Oct 202614 Oct 2026 – 15 Oct 2026Not recordedSee date precision and per-result details below
Application / version: llama-server b11580
Runtime / extension: llama.cpp b11580 (4413a1924)

Dates identify the original test captures. Video, review, and regrade dates are separate. Unavailable values are marked “Not recorded.”

Cache format, allocated context and boot session can differ. Short-answer timing is descriptive; capacity verification is not accuracy above 128K. Missing configurations remain unavailable.

View capture dates, versions & settings by test profile

KV cache · 4K · Asset registry

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 4K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 04:15:18.148602 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:28:18.910959 +00:00
Capture finished: 2026-10-14 · 04:29:26.081863 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:56:02.781532 +00:00
Capture finished: 2026-10-14 · 04:57:09.280527 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:23:33.767875 +00:00
Capture finished: 2026-10-14 · 05:24:40.536810 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:11:32.020388 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:20:40.863843 +00:00
Capture finished: 2026-10-14 · 03:21:27.535756 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:30:50.070014 +00:00
Capture finished: 2026-10-14 · 03:31:35.490003 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:51:37.521183 +00:00
Capture finished: 2026-10-14 · 03:52:28.851377 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 4K · Config tracking

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 4K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:15:19.986390 +00:00
Capture finished: 2026-10-14 · 04:16:28.165618 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:29:27.951668 +00:00
Capture finished: 2026-10-14 · 04:30:34.828206 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:57:11.193129 +00:00
Capture finished: 2026-10-14 · 04:58:15.088770 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:24:42.540244 +00:00
Capture finished: 2026-10-14 · 05:25:48.518622 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:11:32.093674 +00:00
Capture finished: 2026-10-14 · 03:12:13.779306 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:21:27.671453 +00:00
Capture finished: 2026-10-14 · 03:22:08.485092 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:31:35.624515 +00:00
Capture finished: 2026-10-14 · 03:32:15.449423 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:52:29.126801 +00:00
Capture finished: 2026-10-14 · 03:53:10.763141 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 16K · Asset registry

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 16K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:16:33.962592 +00:00
Capture finished: 2026-10-14 · 04:17:34.967613 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:30:40.565087 +00:00
Capture finished: 2026-10-14 · 04:31:42.113139 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:58:20.832361 +00:00
Capture finished: 2026-10-14 · 04:59:22.634912 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:25:54.353507 +00:00
Capture finished: 2026-10-14 · 05:26:56.287468 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:12:13.874837 +00:00
Capture finished: 2026-10-14 · 03:13:01.660278 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:22:08.628396 +00:00
Capture finished: 2026-10-14 · 03:22:57.141156 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:32:15.647602 +00:00
Capture finished: 2026-10-14 · 03:33:03.305385 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:53:10.956905 +00:00
Capture finished: 2026-10-14 · 03:54:02.961589 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 16K · Config tracking

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 16K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:17:41.048674 +00:00
Capture finished: 2026-10-14 · 04:18:38.462625 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:31:48.169223 +00:00
Capture finished: 2026-10-14 · 04:32:45.162042 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:59:28.583973 +00:00
Capture finished: 2026-10-14 · 05:00:25.532267 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:27:02.383134 +00:00
Capture finished: 2026-10-14 · 05:27:59.421549 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:01.952061 +00:00
Capture finished: 2026-10-14 · 03:13:44.694167 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:22:57.449898 +00:00
Capture finished: 2026-10-14 · 03:23:40.035558 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:33:03.557230 +00:00
Capture finished: 2026-10-14 · 03:33:44.248917 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:54:03.258152 +00:00
Capture finished: 2026-10-14 · 03:54:47.218618 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 32K · Asset registry

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 32K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:18:50.195643 +00:00
Capture finished: 2026-10-14 · 04:20:18.385489 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:32:56.837503 +00:00
Capture finished: 2026-10-14 · 04:34:25.638186 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:00:37.198922 +00:00
Capture finished: 2026-10-14 · 05:02:04.553779 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:28:11.099819 +00:00
Capture finished: 2026-10-14 · 09:41:25.536999 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:44.852782 +00:00
Capture finished: 2026-10-14 · 03:14:59.599627 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:23:40.379697 +00:00
Capture finished: 2026-10-14 · 03:24:52.500525 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:33:44.542537 +00:00
Capture finished: 2026-10-14 · 03:34:55.629743 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:54:47.502904 +00:00
Capture finished: 2026-10-14 · 03:56:05.059150 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 32K · Config tracking

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 32K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:20:30.665415 +00:00
Capture finished: 2026-10-14 · 04:21:54.297332 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:34:37.640765 +00:00
Capture finished: 2026-10-14 · 04:36:01.753229 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:02:16.514931 +00:00
Capture finished: 2026-10-14 · 05:03:40.362378 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:41:36.577981 +00:00
Capture finished: 2026-10-14 · 09:42:48.552514 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:14:59.860199 +00:00
Capture finished: 2026-10-14 · 03:16:08.402680 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:24:52.812723 +00:00
Capture finished: 2026-10-14 · 03:25:59.150729 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:34:55.913641 +00:00
Capture finished: 2026-10-14 · 03:36:00.400747 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:56:05.403486 +00:00
Capture finished: 2026-10-14 · 03:57:14.160193 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 64K · Asset registry

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 64K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:22:20.884164 +00:00
Capture finished: 2026-10-14 · 04:24:50.451253 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:36:28.791509 +00:00
Capture finished: 2026-10-14 · 04:39:00.246064 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:04:07.081163 +00:00
Capture finished: 2026-10-14 · 05:06:37.120369 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:43:13.797562 +00:00
Capture finished: 2026-10-14 · 09:45:26.016699 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:16:08.660125 +00:00
Capture finished: 2026-10-14 · 03:18:13.665437 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:25:59.492714 +00:00
Capture finished: 2026-10-14 · 03:28:12.482763 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:36:00.697298 +00:00
Capture finished: 2026-10-14 · 03:38:10.212083 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:57:14.567782 +00:00
Capture finished: 2026-10-14 · 03:59:37.195192 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 64K · Config tracking

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 64K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:25:18.420680 +00:00
Capture finished: 2026-10-14 · 04:27:45.412007 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:39:28.416692 +00:00
Capture finished: 2026-10-14 · 04:41:56.377435 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:07:04.833424 +00:00
Capture finished: 2026-10-14 · 05:09:32.314410 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:45:52.250245 +00:00
Capture finished: 2026-10-14 · 09:48:01.112140 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:18:13.975975 +00:00
Capture finished: 2026-10-14 · 03:20:10.561169 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:28:12.882966 +00:00
Capture finished: 2026-10-14 · 03:30:19.388489 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:38:10.588250 +00:00
Capture finished: 2026-10-14 · 03:40:13.160646 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:59:37.643440 +00:00
Capture finished: 2026-10-14 · 04:01:50.574155 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 128K · Asset registry

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 128K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 14:06:25.896241 +00:00
Capture finished: 2026-10-14 · 14:10:57.290717 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:43:10.409610 +00:00
Capture finished: 2026-10-14 · 04:48:38.587162 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:10:45.630486 +00:00
Capture finished: 2026-10-14 · 05:16:13.627250 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:49:11.225016 +00:00
Capture finished: 2026-10-14 · 09:54:03.424044 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · Not recorded

Test captured: Not recorded
Date precision: No accepted captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted allocation
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 14:17:04.453095 +00:00
Capture finished: 2026-10-14 · 14:22:12.392690 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:40:13.747740 +00:00
Capture finished: 2026-10-14 · 03:45:40.123737 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:01:51.178028 +00:00
Capture finished: 2026-10-14 · 04:07:47.581844 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 128K · Config tracking

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 128K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 14:12:05.431576 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:49:55.331797 +00:00
Capture finished: 2026-10-14 · 04:55:26.339894 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:17:29.582002 +00:00
Capture finished: 2026-10-14 · 05:22:57.429055 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:55:16.173029 +00:00
Capture finished: 2026-10-14 · 10:00:08.794758 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · Not recorded

Test captured: Not recorded
Date precision: No accepted captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted allocation
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 14:22:12.695270 +00:00
Capture finished: 2026-10-14 · 14:27:19.828653 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:45:40.698279 +00:00
Capture finished: 2026-10-14 · 03:51:06.054837 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:07:48.228195 +00:00
Capture finished: 2026-10-14 · 04:13:35.848193 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 4K · short-answer speed

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 4K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 04:16:28.165618 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:28:18.910959 +00:00
Capture finished: 2026-10-14 · 04:30:34.828206 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:56:02.781532 +00:00
Capture finished: 2026-10-14 · 04:58:15.088770 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:23:33.767875 +00:00
Capture finished: 2026-10-14 · 05:25:48.518622 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:12:13.779306 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:20:40.863843 +00:00
Capture finished: 2026-10-14 · 03:22:08.485092 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:30:50.070014 +00:00
Capture finished: 2026-10-14 · 03:32:15.449423 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:51:37.521183 +00:00
Capture finished: 2026-10-14 · 03:53:10.763141 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 16K · short-answer speed

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 16K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:16:33.962592 +00:00
Capture finished: 2026-10-14 · 04:18:38.462625 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:30:40.565087 +00:00
Capture finished: 2026-10-14 · 04:32:45.162042 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:58:20.832361 +00:00
Capture finished: 2026-10-14 · 05:00:25.532267 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:25:54.353507 +00:00
Capture finished: 2026-10-14 · 05:27:59.421549 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:12:13.874837 +00:00
Capture finished: 2026-10-14 · 03:13:44.694167 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:22:08.628396 +00:00
Capture finished: 2026-10-14 · 03:23:40.035558 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:32:15.647602 +00:00
Capture finished: 2026-10-14 · 03:33:44.248917 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:53:10.956905 +00:00
Capture finished: 2026-10-14 · 03:54:47.218618 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 32K · short-answer speed

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 32K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:18:50.195643 +00:00
Capture finished: 2026-10-14 · 04:21:54.297332 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:32:56.837503 +00:00
Capture finished: 2026-10-14 · 04:36:01.753229 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:00:37.198922 +00:00
Capture finished: 2026-10-14 · 05:03:40.362378 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:28:11.099819 +00:00
Capture finished: 2026-10-14 · 09:42:48.552514 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:44.852782 +00:00
Capture finished: 2026-10-14 · 03:16:08.402680 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:23:40.379697 +00:00
Capture finished: 2026-10-14 · 03:25:59.150729 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:33:44.542537 +00:00
Capture finished: 2026-10-14 · 03:36:00.400747 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:54:47.502904 +00:00
Capture finished: 2026-10-14 · 03:57:14.160193 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 64K · short-answer speed

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 64K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:22:20.884164 +00:00
Capture finished: 2026-10-14 · 04:27:45.412007 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:36:28.791509 +00:00
Capture finished: 2026-10-14 · 04:41:56.377435 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:04:07.081163 +00:00
Capture finished: 2026-10-14 · 05:09:32.314410 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:43:13.797562 +00:00
Capture finished: 2026-10-14 · 09:48:01.112140 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:16:08.660125 +00:00
Capture finished: 2026-10-14 · 03:20:10.561169 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:25:59.492714 +00:00
Capture finished: 2026-10-14 · 03:30:19.388489 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:36:00.697298 +00:00
Capture finished: 2026-10-14 · 03:40:13.160646 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:57:14.567782 +00:00
Capture finished: 2026-10-14 · 04:01:50.574155 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · 128K · short-answer speed

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 128K test bucket; allocated capacity per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 14:06:25.896241 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:43:10.409610 +00:00
Capture finished: 2026-10-14 · 04:55:26.339894 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:10:45.630486 +00:00
Capture finished: 2026-10-14 · 05:22:57.429055 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:49:11.225016 +00:00
Capture finished: 2026-10-14 · 10:00:08.794758 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · Not recorded

Test captured: Not recorded
Date precision: No accepted captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted allocation
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 14:17:04.453095 +00:00
Capture finished: 2026-10-14 · 14:27:19.828653 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:40:13.747740 +00:00
Capture finished: 2026-10-14 · 03:51:06.054837 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:01:51.178028 +00:00
Capture finished: 2026-10-14 · 04:13:35.848193 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · practical context capacity

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: Per-result allocated context; 4K input verification
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-14 · 15:12:48.944906 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · 15 Oct 2026

Test captured: 15 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-15 · 00:08:42.017583 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 208,896 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-14 · 15:14:44.753906 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 262,144 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · 15 Oct 2026

Test captured: 15 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-15 · 00:11:25.096896 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 253,952 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-14 · 15:16:39.736911 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 77,824 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-14 · 15:18:06.895175 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-14 · 15:19:34.167047 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Recorded capacity verification window start; not a full-length inference timestamp
Short-input capacity verification window: 2026-10-14 · 15:21:00.941955 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · main-cache allocation at 128K

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 131,072 allocated tokens
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 8 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 8/8 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 8/8 records
GPU driver version: 617.14 · 8/8 records
CUDA runtime version: Windows CUDA 13.4 build · 8/8 records

Per-result dates, versions & settings (8)

Qwen3.8-27B UD-Q6_K · F16 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · F16 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · Not recorded

Test captured: Not recorded
Date precision: Derived allocation requirement, not an independent inference capture
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens (requirement calculation)
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · paired accuracy and answer churn

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 4K–128K; bucket and task per result
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 68 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 68/68 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 68/68 records
GPU driver version: 617.14 · 68/68 records
CUDA runtime version: Windows CUDA 13.4 build · 68/68 records

Per-result dates, versions & settings (68)

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-4k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 04:29:26.081863 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-4k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 04:57:09.280527 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-4k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 05:24:40.536810 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-4k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:15:19.986390 +00:00
Capture finished: 2026-10-14 · 04:30:34.828206 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-4k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:15:19.986390 +00:00
Capture finished: 2026-10-14 · 04:58:15.088770 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-4k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:15:19.986390 +00:00
Capture finished: 2026-10-14 · 05:25:48.518622 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-16k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:16:33.962592 +00:00
Capture finished: 2026-10-14 · 04:31:42.113139 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-16k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:16:33.962592 +00:00
Capture finished: 2026-10-14 · 04:59:22.634912 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-16k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:16:33.962592 +00:00
Capture finished: 2026-10-14 · 05:26:56.287468 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-16k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:17:41.048674 +00:00
Capture finished: 2026-10-14 · 04:32:45.162042 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-16k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:17:41.048674 +00:00
Capture finished: 2026-10-14 · 05:00:25.532267 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-16k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:17:41.048674 +00:00
Capture finished: 2026-10-14 · 05:27:59.421549 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-32k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:18:50.195643 +00:00
Capture finished: 2026-10-14 · 04:34:25.638186 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-32k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:18:50.195643 +00:00
Capture finished: 2026-10-14 · 05:02:04.553779 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-32k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:18:50.195643 +00:00
Capture finished: 2026-10-14 · 09:41:25.536999 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-32k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:20:30.665415 +00:00
Capture finished: 2026-10-14 · 04:36:01.753229 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-32k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:20:30.665415 +00:00
Capture finished: 2026-10-14 · 05:03:40.362378 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-32k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:20:30.665415 +00:00
Capture finished: 2026-10-14 · 09:42:48.552514 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-64k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:22:20.884164 +00:00
Capture finished: 2026-10-14 · 04:39:00.246064 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-64k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:22:20.884164 +00:00
Capture finished: 2026-10-14 · 05:06:37.120369 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-64k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:22:20.884164 +00:00
Capture finished: 2026-10-14 · 09:45:26.016699 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-64k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:25:18.420680 +00:00
Capture finished: 2026-10-14 · 04:41:56.377435 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-64k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:25:18.420680 +00:00
Capture finished: 2026-10-14 · 05:09:32.314410 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-64k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:25:18.420680 +00:00
Capture finished: 2026-10-14 · 09:48:01.112140 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:43:10.409610 +00:00
Capture finished: 2026-10-14 · 14:10:57.290717 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:10:45.630486 +00:00
Capture finished: 2026-10-14 · 14:10:57.290717 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:49:11.225016 +00:00
Capture finished: 2026-10-14 · 14:10:57.290717 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-q8-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:43:10.409610 +00:00
Capture finished: 2026-10-14 · 05:16:13.627250 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-q8-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:43:10.409610 +00:00
Capture finished: 2026-10-14 · 09:54:03.424044 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:49:55.331797 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 05:17:29.582002 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 09:55:16.173029 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-q8-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:49:55.331797 +00:00
Capture finished: 2026-10-14 · 05:22:57.429055 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-q8-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:49:55.331797 +00:00
Capture finished: 2026-10-14 · 10:00:08.794758 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-4k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:21:27.535756 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-4k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:31:35.490003 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-4k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:52:28.851377 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-4k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:11:32.093674 +00:00
Capture finished: 2026-10-14 · 03:22:08.485092 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-4k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:11:32.093674 +00:00
Capture finished: 2026-10-14 · 03:32:15.449423 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-4k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:11:32.093674 +00:00
Capture finished: 2026-10-14 · 03:53:10.763141 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-16k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:12:13.874837 +00:00
Capture finished: 2026-10-14 · 03:22:57.141156 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-16k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:12:13.874837 +00:00
Capture finished: 2026-10-14 · 03:33:03.305385 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-16k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:12:13.874837 +00:00
Capture finished: 2026-10-14 · 03:54:02.961589 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-16k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:01.952061 +00:00
Capture finished: 2026-10-14 · 03:23:40.035558 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-16k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:01.952061 +00:00
Capture finished: 2026-10-14 · 03:33:44.248917 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-16k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:01.952061 +00:00
Capture finished: 2026-10-14 · 03:54:47.218618 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-32k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:44.852782 +00:00
Capture finished: 2026-10-14 · 03:24:52.500525 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-32k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:44.852782 +00:00
Capture finished: 2026-10-14 · 03:34:55.629743 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-32k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:13:44.852782 +00:00
Capture finished: 2026-10-14 · 03:56:05.059150 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-32k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:14:59.860199 +00:00
Capture finished: 2026-10-14 · 03:25:59.150729 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-32k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:14:59.860199 +00:00
Capture finished: 2026-10-14 · 03:36:00.400747 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-32k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:14:59.860199 +00:00
Capture finished: 2026-10-14 · 03:57:14.160193 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-64k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:16:08.660125 +00:00
Capture finished: 2026-10-14 · 03:28:12.482763 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-64k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:16:08.660125 +00:00
Capture finished: 2026-10-14 · 03:38:10.212083 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-64k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:16:08.660125 +00:00
Capture finished: 2026-10-14 · 03:59:37.195192 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-64k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:18:13.975975 +00:00
Capture finished: 2026-10-14 · 03:30:19.388489 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-64k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:18:13.975975 +00:00
Capture finished: 2026-10-14 · 03:40:13.160646 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-64k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:18:13.975975 +00:00
Capture finished: 2026-10-14 · 04:01:50.574155 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-128k-asset · Not recorded

Test captured: Not recorded
Date precision: No accepted paired captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted paired allocation; component allocations are in result details
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-128k-asset · Not recorded

Test captured: Not recorded
Date precision: No accepted paired captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted paired allocation; component allocations are in result details
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-128k-asset · Not recorded

Test captured: Not recorded
Date precision: No accepted paired captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted paired allocation; component allocations are in result details
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-q8-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:40:13.747740 +00:00
Capture finished: 2026-10-14 · 14:22:12.392690 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-q8-128k-asset · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:01:51.178028 +00:00
Capture finished: 2026-10-14 · 14:22:12.392690 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16-128k-config · Not recorded

Test captured: Not recorded
Date precision: No accepted paired captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted paired allocation; component allocations are in result details
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16-128k-config · Not recorded

Test captured: Not recorded
Date precision: No accepted paired captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted paired allocation; component allocations are in result details
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16-128k-config · Not recorded

Test captured: Not recorded
Date precision: No accepted paired captures for this configuration.
Capture timezone: Not recorded
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: No accepted paired allocation; component allocations are in result details
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-q8-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:45:40.698279 +00:00
Capture finished: 2026-10-14 · 14:27:19.828653 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-q8-128k-config · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:07:48.228195 +00:00
Capture finished: 2026-10-14 · 14:27:19.828653 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · pooled differences versus F16

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: Qwen 4K–128K; Mistral 4K–64K
Thinking: Off
MTP: Qwen: on (3 draft tokens); Mistral: off
K / V cache: F16 / Q8 / Q4 / K8V4; per result

Versions below cover 6 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 6/6 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 6/6 records
GPU driver version: 617.14 · 6/6 records
CUDA runtime version: Windows CUDA 13.4 build · 6/6 records

Per-result dates, versions & settings (6)

Qwen3.8-27B UD-Q6_K · Q8 · vs-f16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q8
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · Q4 · vs-f16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · K8V4 · vs-f16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:14:11.328160 +00:00
Capture finished: 2026-10-14 · 14:16:38.060753 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q8 · vs-f16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:30:19.388489 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: Q8
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · Q4 · vs-f16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 03:40:13.160646 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: Q4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

Mistral Small 3.2 24B Instruct 2506 Q6_K · K8V4 · vs-f16 · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 03:10:44.924010 +00:00
Capture finished: 2026-10-14 · 04:01:50.574155 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 / 131,072 tokens
K / V cache: K8V4
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Mistral Small 3.2 24B Instruct 2506 Q6_K

View this profile in the benchmark explorer

KV cache · Qwen MTP on/off

Benchmark edition: KV-cache retrieval study; deterministic exact-match grading

Context capacity: 32K test bucket; allocations differ
Thinking: Off
MTP: On versus off; per result
K / V cache: F16 / F16

Versions below cover 2 result records. Open each result for its exact combination and capture dates. Reused answers and summaries are separate records, not additional test runs.

Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
Selected runtime extension: Not recorded · 2/2 records
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded · 2/2 records
GPU driver version: 617.14 · 2/2 records
CUDA runtime version: Windows CUDA 13.4 build · 2/2 records

Per-result dates, versions & settings (2)

Qwen3.8-27B UD-Q6_K · F16 · mtp-on · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 04:18:50.195643 +00:00
Capture finished: 2026-10-14 · 04:21:54.297332 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 65,536 tokens
K / V cache: F16
MTP: On, maximum 3 draft tokens; draft KV F16
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

Qwen3.8-27B UD-Q6_K · F16 · mtp-off · 14 Oct 2026

Test captured: 14 Oct 2026
Date precision: Range of accepted question request envelopes; intervals can include downtime.
Capture started: 2026-10-14 · 10:00:47.396557 +00:00
Capture finished: 2026-10-14 · 10:03:21.433251 +00:00
Capture timezone: UTC
Application / version: llama-server b11580
Inference engine / build: llama.cpp b11580 (4413a1924)
OS / environment: Windows; RTX 5090 32 GB, display-attached as reported by Casey; Intel i9-13900K; 128 GB DDR5; benchmark-era Windows build not recorded
GPU driver version: 617.14
CUDA runtime version: Windows CUDA 13.4 build
Thinking: Off
Sampling: Greedy; temperature 0
Output cap: 48 tokens
Flash attention: Enabled
GPU offload: Full for accepted runs
Batch / microbatch: 2048 / 512
Parallel requests: 1
Automatic fit / context shifting: Disabled
Context capacity: 36,864 tokens
K / V cache: F16
MTP: Off
Prompt reuse: One document prefix reused across questions; Qwen primed before questions
Weight package: Qwen3.8-27B UD-Q6_K

View this profile in the benchmark explorer

I’d start with Q8 KV cache on the two models I tested. It reduced the main cache allocation by about 47%, with no detectable accuracy difference after correcting for multiple comparisons. Q4 saved about 72% of the cache memory, but changed more individual answers. That’s a useful trade when context is running into your VRAM limit, provided you check the behavior your workflow depends on.

I graded 8,000 answers on one RTX 5090. The result wasn’t that every setting behaved the same. Scores were close, individual answers moved, and one mixed-cache setting ran noticeably slower. Those are three different things to consider before changing a dropdown.

I changed the cache while keeping each model’s weights fixed

The KV cache stores the keys and values an attention layer needs from text the model has already read. Keeping them avoids recalculating that work for each new token. The cache grows with context, while the model weights stay a fixed cost.

Cache quantization stores those keys and values with fewer bits. It changes a different part of the system from weight quantization. I kept both models’ main weights at Q6_K and varied their main KV cache format. Some runs used different allocated context sizes or boot sessions, which limits what the speed comparisons can isolate.

Main KV cache formats and memory savings
Setting	Keys / values	Main cache memory saved versus F16
F16	16-bit floating point / 16-bit floating point	Reference
Q8	Q8 / Q8	46.9%
Q4	Q4 / Q4	71.9%
K8V4	Q8 / Q4	59.4%

The models were Unsloth’s Qwen3.8-27B UD-Q6_K and Mistral Small 3.2 24B Instruct 2506 Q6_K. Qwen is a hybrid: 16 of its 64 layers use full attention, with recurrent layers making up the rest. Mistral uses attention in all 40 layers. That gives them very different cache costs.

I used llama.cpp b11580, the Windows CUDA 13.4 build, on a display-attached RTX 5090 with 32 GB of VRAM, an i9-13900K and 128 GB of RAM. Flash attention and full GPU offload were enabled. Sampling was greedy, thinking was off, and answers were capped at 48 tokens.

Qwen also used an MTP draft head, which proposes tokens ahead for the main model to check. Its draft cache stayed F16 throughout the main comparison. Mistral ran without it.

The tests measured precise lookup and tracking the latest value

I used two synthetic document tasks. The asset registry asks the model to retrieve a specific field or identify a record from a site and room. Similar records and near-collision identifiers make guessing risky. The config change log asks for the final value of a setting after several updates, with older values and similar settings as distractions.

Every cache setting for a given model received the same documents and questions. Document lengths were measured with that model’s tokenizer. Each model/cache/context/task cell had 100 questions, spread evenly across five depth zones. At 4K, that meant 20 documents with five questions each; longer buckets used four documents with 25 questions each.

The buckets were 4K, 16K, 32K, 64K and 128K. Those labels describe the test sizes: the 128K bucket targeted a document of 120,000 tokens, leaving room for the instructions and answer. In the context figures below, K means 1,024 tokens.

Grading used a fixed parser and exact-match comparison, with no AI judge. The totals were 7,800 core answers plus 200 answers in an MTP-off check. Mistral F16 at 128K remained unavailable because its allocation exceeded this card’s VRAM. Priming requests, fit checks and repeatability checks weren’t counted as scored answers.

I found no detectable accuracy difference after correction

Across 62 paired accuracy comparisons, zero survived Holm correction. That correction accounts for running many comparisons, so isolated small p-values don’t become a headline just because I tested enough combinations. Two comparisons were below the usual threshold before correction; neither remained there afterward.

I also pooled the paired results over the buckets where F16 was available. These intervals resample whole documents, so questions sharing a document aren’t treated as completely independent observations.

Pooled paired accuracy differences versus F16
Model	Cache versus F16	Paired questions	Accuracy difference, percentage points	95% document-bootstrap interval
Qwen3.8-27B	Q8	1,000	-0.70	-1.76 to +0.33
Qwen3.8-27B	Q4	1,000	+0.40	-1.07 to +1.91
Qwen3.8-27B	K8V4	1,000	+0.10	-0.87 to +1.08
Mistral Small 3.2	Q8	800	-0.13	-0.83 to +0.63
Mistral Small 3.2	Q4	800	+0.38	-2.00 to +2.57
Mistral Small 3.2	K8V4	800	-0.13	-1.56 to +1.33

Qwen includes 4K through 128K; Mistral includes 4K through 64K. Every interval includes zero. That supports no detectable difference on these tests, while leaving room for small effects. It doesn’t establish equivalence, and pooling can conceal differences on a particular task or context size.

For a concrete long-context example, Qwen’s 128K asset accuracy was 97% with F16, 95% with Q8, 98% with Q4 and 97% with K8V4. On config tracking, those scores were 67%, 64%, 68% and 66%. There were 100 questions behind each percentage, so I wouldn’t read a one-point gap as a practical ranking.

The much larger descriptive change was the difficulty of the task at longer context. Mistral F16 config accuracy fell from 78% at 4K to 36% at 16K. Choosing a different cache format didn’t resolve that drop.

Format failures in the two concentrated task cells
Concentrated format failures	F16	Q8	Q4	K8V4
Qwen 4K config	12	11	2	7
Mistral 32K asset	32	30	8	33

The cache savings are substantial, but total VRAM savings are smaller

At an allocated context of 131,072 tokens, the main cache requires the following memory, calculated from the verified per-token cache sizes. The Mistral F16 entry was unavailable for the accepted benchmark. Allocations here use GiB, with 1 GiB equal to 1,024 MiB; the graphics card is sold as a 32 GB model.

Main KV cache requirement at 131,072 tokens
Cache	Qwen3.8-27B main cache	Mistral Small 3.2 main cache
F16	8.0 GiB	20.0 GiB
Q8	4.3 GiB	10.6 GiB
Q4	2.3 GiB	5.6 GiB
K8V4	3.3 GiB	8.1 GiB

Qwen’s separate F16 draft cache isn’t included in that table. The weights, recurrent state and compute workspace also need memory. That’s why a 72% reduction in the main cache doesn’t mean 72% less total VRAM.

For Qwen at that context allocation, the logged CUDA buffer total fell from 29.8 GiB with F16 to 24.5 GiB with Q4, a reduction of about 17.8%. Those are allocated model/runtime buffers, excluding desktop use; they aren’t the GPU’s sampled peak usage.

Mistral made the context trade especially clear. Q4 at 131,072 tokens allocated about 23.8 GiB, compared with 27.8 GiB for F16 at 65,536 tokens. That’s twice the allocated context with about 4.0 GiB less in CUDA buffers.

Quantization gave me more room for context

These are the verified practical context allocations under my guarded setup. They are not exhaustive hardware limits, and the larger allocations were checked with a short input rather than a full document at their limit.

Verified practical context allocations under this setup
Model	Cache	Verified context	Limiting condition
Mistral Small 3.2	F16	77,824 tokens / 76K	VRAM budget
Mistral Small 3.2	Q8	131,072 tokens / 128K	Model's trained limit
Mistral Small 3.2	Q4	131,072 tokens / 128K	Model's trained limit
Mistral Small 3.2	K8V4	131,072 tokens / 128K	Model's trained limit
Qwen3.8-27B	F16	131,072 tokens / 128K	VRAM budget
Qwen3.8-27B	Q8	208,896 tokens / 204K	VRAM budget
Qwen3.8-27B	Q4	262,144 tokens / 256K	Model's trained limit
Qwen3.8-27B	K8V4	253,952 tokens / 248K	VRAM budget

Scope: practical max under this setup (display-attached, fresh-reboot baseline); allocation + 4K speed verified; long-context accuracy not tested beyond 128K.

Each accepted capacity launch retained at least 1,640 MiB of logged buffer-plus-baseline headroom and passed the short-input speed screen. Compute workspace also grew with context, so predicting capacity from weights and cache alone was insufficient.

The desktop mattered too. An early Qwen F16/128K launch loaded but failed the speed screen with only 46.6 MiB of logged headroom. After a fresh reboot, the unloaded desktop baseline was 423 MiB and the configuration passed qualification. Loading successfully wasn’t enough to make a run usable. Memory pressure was a plausible explanation for the earlier slowdown, but I didn’t isolate its cause.

Mixed K8V4 hit a slower generation path in this build

Reading the documents and generating the answers showed different behavior. For Mistral through 64K, cold document-processing medians stayed within roughly 3.1% of F16. Generation exposed a much larger K8V4 penalty as context grew.

Mistral generation rate in tokens per second
Mistral context bucket	F16 generation	Q8 generation	Q4 generation	K8V4 generation
4K	72.3 tok/s	69.6 tok/s	69.6 tok/s	65.7 tok/s
16K	66.4 tok/s	65.1 tok/s	65.1 tok/s	54.4 tok/s
32K	59.8 tok/s	59.5 tok/s	59.2 tok/s	48.1 tok/s
64K	49.7 tok/s	51.0 tok/s	50.6 tok/s	34.1 tok/s
128K	Unavailable	39.6 tok/s	38.7 tok/s	21.5 tok/s

These are median rates from short answers, not a sustained long-output benchmark. At 128K, K8V4 was about 46% slower than Q8. That comparison spans boot sessions. At the 64K test bucket, K8V4 was about 33% slower than Q8 in the same session, but K8V4 allocated 131,072 tokens while Q8 allocated 65,536.

The tested b11580 CUDA build logged that its mixed Q8-key/Q4-value vector flash-attention kernel was missing and that keys and values were being converted to F16 instead. Flash attention remained enabled and every model layer stayed on the GPU.

That extra conversion is a likely major contributor to the slowdown. I didn’t profile its exact share of latency or test a rebuilt binary, so I can’t promise how much speed compiling the mixed kernel would recover. This finding applies to the tested build, not every current llama.cpp package or app bundling it.

Qwen’s matched Q8 and Q4 caches had native kernels, yet their 64K generation medians were about 103 tok/s against F16’s 116 tok/s. F16 allocated 65,536 tokens; Q8 and Q4 allocated 131,072, so this also isn’t an isolated cache-precision comparison. The missing mixed-pair kernel doesn’t explain these matched-cache rates. The 128K F16 comparison also came from a later boot session. I wouldn’t turn these measurements into a claim that quantized cache always preserves generation speed.

MTP was useful, with a comparison caveat

On the Qwen F16/32K questions, the MTP-on run generated at a median 124.4 tok/s, versus 55.6 tok/s with MTP off: an observed 2.24x ratio. All 200 paired answers agreed at both parsed and byte level.

MTP on allocated 65,536 tokens; MTP off allocated 36,864, and the runs came from different boot sessions. That makes the ratio an observed comparison rather than a clean measurement of MTP alone. It also doesn’t establish that MTP will preserve every answer on other workloads.

What I’d use, and where I’d test again

For these models, Q8 is my starting point: substantial cache savings and relatively little answer churn. Q4 is my choice when I need more room, with a check of the specific responses I rely on. F16 remains useful when VRAM allows it and I want the reference behavior. I’d avoid K8V4 on the tested build unless I had verified its kernel support and performance.

The limits of this result matter. I tested two Q6_K model packages, one GPU and one runtime build, using synthetic retrieval tasks with short answers. This doesn’t establish behavior on coding, extended reasoning, long outputs, other weight formats, other runtimes or accuracy above 128K. Four documents per longer-context task cell also provide limited coverage, even with 100 questions. The repeat checks establish observed agreement on their samples, not a universal zero-noise guarantee.

For my use, the memory savings are worth testing. I’d start with Q8, move to Q4 when context needs the space, and check both correctness and answer stability on the work I actually do.

In the companion KV-cache video, I walk through the results and memory tradeoffs. Tell me in the YouTube comments which model and cache setting you’re using, and subscribe if you’d like more local-model tests like this one.

BRING YOUR SETUP TO THE CONVERSATION

What are you running?

Watch the comparison, then share your hardware and workload in the YouTube comments.

Watch & discuss on YouTube

Test capture & runtime

KV cache · 4K · Asset registry

KV cache · 4K · Config tracking

KV cache · 16K · Asset registry

KV cache · 16K · Config tracking

KV cache · 32K · Asset registry

KV cache · 32K · Config tracking

KV cache · 64K · Asset registry

KV cache · 64K · Config tracking

KV cache · 128K · Asset registry

KV cache · 128K · Config tracking

KV cache · 4K · short-answer speed

KV cache · 16K · short-answer speed

KV cache · 32K · short-answer speed

KV cache · 64K · short-answer speed

KV cache · 128K · short-answer speed

KV cache · practical context capacity

KV cache · main-cache allocation at 128K

KV cache · paired accuracy and answer churn

KV cache · pooled differences versus F16

KV cache · Qwen MTP on/off

I changed the cache while keeping each model’s weights fixed

The tests measured precise lookup and tracking the latest value

I found no detectable accuracy difference after correction

Similar scores still hid different answers

The cache savings are substantial, but total VRAM savings are smaller

Quantization gave me more room for context

Mixed K8V4 hit a slower generation path in this build

MTP was useful, with a comparison caveat

What I’d use, and where I’d test again

What are you running?