THE RESULT 392.2 output tokens per second at the engine. 387.6 through the full Rafay governance layer — quotas, metering, and multi-tenant controls included. That's 52.6% more throughput than a strong FP8 baseline, with no measurable drop in quality.
GPU capacity doesn't become valuable the moment a model loads. It becomes valuable once developers can call it reliably, and once an operator can control, measure, price, and reproduce the service built around it. Closing that gap, between raw performance and an operable service, was the whole point of this joint experiment. It's really two problems: making inference efficient, and making it governable.
To test both at once, we deployed the language-model path of Qwen3.6-27B on a 96 GB NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. Minima took on the first problem, reducing the memory and compute required inside the worker. Rafay Token Factory took on the second, publishing that same worker as an OpenAI-compatible, multi-tenant endpoint, complete with API access, quotas, rate limits, metrics, and token-level usage records.
The important result is that both problems were solved at the same time, in the same stack: a governed inference SKU an operator could offer to customers or internal teams, running at the full efficiency the hardware can deliver without building the service layer from scratch.
We tested the service, not only the engine
The comparison used the same model revision, prompts, scheduler settings, and GPU. The strong baseline used FP8 weights, activations, and KV cache with optimized Blackwell kernels. The Minima configuration used Blackwell-native NVFP4 W4A4 execution, Qwen-specific kernels including Gated DeltaNet paths, and a staged attention KV cache: recent and anchor pages in FP8, stale pages in TurboQuant 3-bit.
For the interactive load point, we sent 32 requests at concurrency eight with approximately 512 input tokens and up to 256 output tokens. Each scored point was warmed and repeated five times. We also ran long-context memory tests, a fixed quality suite, tenant-policy checks, token reconciliation, and a 24-hour mixed-tenant soak.
Compression basis The 3.2x weight and 3.5x attention-KV ratios are both measured against BF16. Against the strong FP8 baseline, Minima used 37.5% less weight memory and 42.9% less attention-KV memory. The ratios apply to different memory pools and are not multiplied.
The result on one Blackwell GPU
| Metric | FP8 through Rafay | Minima through Rafay | Outcome |
|---|---|---|---|
| Resident weight payload | 25.15 GiB | 15.72 GiB | 37.5% less |
| Attention KV at 1M live tokens | 30.52 GiB | 17.44 GiB | 42.9% less |
| Weights + 1M live KV | 55.67 GiB | 33.16 GiB | 40.4% less |
| Resident 32K context equivalents | 53 | 98 | 1.85x |
| Aggregate output throughput | 254.0 tok/s | 387.6 tok/s | 52.6% more |
| TTFT p50 / p95 | 1.21 / 2.29 s | 0.91 / 1.73 s | 24-25% lower |
| E2E p50 / p95 | 7.48 / 8.99 s | 5.03 / 5.87 s | 33-35% lower |
| GPU-hours per 1M output tokens | 1.094 | 0.717 | 34.5% less |
| 30-day output at 70% busy | 461M tokens | 703M tokens | 242M more |
Resident-context figures use an 86.4 GiB serving envelope and include the model's fixed DeltaNet sequence state. They are memory ceilings, not a claim that all sequences can decode concurrently at the same SLO.
The Minima worker delivered 392.2 aggregate output tokens per second when addressed directly. The same worker delivered 387.6 tokens per second through the Rafay-published endpoint. The matched FP8 service delivered 254.0 tokens per second through the same path. End to end, Minima increased throughput by 52.6% and cut GPU time per million output tokens by 34.5%.
Quality stayed flat. Across MMLU-Pro, GPQA Diamond, HumanEval+, IFEval, and long-context retrieval, the combined configuration averaged 0.03 percentage points below the BF16 reference, inside the pre-agreed plus-or-minus 0.5 point non-inferiority gate. Passkey retrieval matched BF16 at 8K, 32K, 128K, and 262K context.
Rafay closed the gap between benchmark and business
Minima changed the economics inside the worker. Rafay changed what the operator could do with that worker. Token Factory bound the pinned Minima image and Qwen artifact to the Blackwell GPU, published the deployment through an OpenAI-compatible endpoint, and exposed it to three test tenants with separate API keys, quotas, and rate limits.
| Operational proof | Measured result | What Rafay made possible |
|---|---|---|
| Serving-path overhead | 392.2 to 387.6 tok/s (-1.17%) | Governance and metering with minimal effect on engine goodput |
| Token reconciliation | 99.97% across 10,000 requests | Attributable, billing-ready input and output usage records |
| Tenant-policy tests | 600 / 600 passed | API-key revocation, quotas, and rate limits affected only the intended tenant |
| 24-hour mixed-tenant soak | 99.99% request success | No monotonic memory growth or unexplained usage drift |
| Operations view | TTFT, ITL, E2E, throughput, and KV pressure visible | One place to operate the service against latency and capacity SLOs |
Developers still saw an endpoint, an API key, and familiar calls. Platform teams saw service health, latency, throughput, and KV-cache pressure. Operators received token usage that could feed pricing, chargeback, showback, or external billing workflows. That is Rafay's value in this stack: it converted a runtime efficiency gain into a service that could be governed, observed, and monetized.
More efficient tokens create more strategic options
At the same hourly GPU cost, the joint service reduced the infrastructure requirement from 1.094 to 0.717 GPU-hours per million output tokens. At 70% sustained utilization, one GPU moved from about 461 million to 703 million output tokens per 30-day month.
- Protect margin. Keep the token price stable while reducing the infrastructure cost of each successful output.
- Compete on price. Pass part of the efficiency gain to customers without surrendering the entire margin improvement.
- Serve more demanding workloads. Use the memory headroom for more tenants, longer contexts, or distinct shared and dedicated service tiers.
- Make economics visible. Use Rafay's token attribution and pricing layer to connect production cost with each tenant, business unit, or external customer.
For a neocloud, telco, or sovereign AI provider, the gain becomes a differentiated model SKU instead of cheaper raw compute. For an enterprise platform team, it becomes a governed internal service with auditable consumption. In both cases, Rafay turns Minima's performance gain into an operating and economic advantage.
Performance becomes valuable when it becomes operable
Minima made each Blackwell worker produce more useful token capacity. Rafay made that capacity discoverable, governable, observable, measurable, and sellable.
Together, we moved Qwen3.6-27B from an optimized runtime result to a production-shaped token service: 52.6% more output per GPU than the matched FP8 service, 1.85x the resident 32K context capacity, quality parity, and billing-ready multi-tenant operations.
For operators building AI clouds, that is the end state worth aiming for: not simply a faster model, but a better token business.
Ready to turn Blackwell capacity into a higher-margin token service? Talk with Rafay and Minima about deploying the validated Qwen3.6-27B reference stack.
About the partners
Rafay Systems. Rafay provides the operational and economic control plane for modern AI infrastructure. Rafay Token Factory enables enterprises, neoclouds, telecommunications providers, and sovereign AI operators to publish models as governed, OpenAI-compatible, token-metered services with multi-tenancy, observability, pricing, and billing-ready usage data.
Minima. Minima develops compressed model formats, KV-cache technology, and a Blackwell-native inference runtime that reduce the hardware and memory required to serve open-weight models. Its Qwen stack combines NVFP4 W4A4 execution, model-specific kernels, and staged KV compression behind an OpenAI-compatible interface.
Methodology and references
Benchmark scope: text-generation path only; vision encoder excluded; tensor parallelism 1. Throughput is aggregate completion-token throughput. Compression ratios are versus BF16, and the staged-KV result reflects the observed page-age mix in the long-context test. 'Quality parity' means no statistically or practically material regression under the stated non-inferiority gate; it does not mean bit-identical arithmetic.
Rafay AI Token Factory - OpenAI-compatible endpoints, multi-tenancy, token metering, pricing, quotas, and billing-ready usage data
Rafay model deployment metrics - TTFT, inter-token latency, E2E latency, throughput, and KV-cache visibility
Qwen3.6-27B model card - model architecture and context support
NVIDIA RTX PRO 6000 Blackwell Server Edition - 96 GB GDDR7 Blackwell GPU