You are deciding whether to buy a sovereign AI box—hardware and software stack you own and operate—or to rent inference time from a cloud API. This page names the hard limits you will encounter with the box, using only numbers measured on its own hardware, without marketing spin or computed ratios.
The box ships with an AMD Ryzen Threadripper PRO 9955WX, 16 physical cores and 32 threads, 251 GB system RAM, 31 GB swap, one NVIDIA RTX 5090 with 32 GB VRAM under a 600 W power limit, and a thermal relationship defined by Tctl = 37.31 + 0.2222·P_cpu + 0.05703·P_gpu (P in watts). Three models can reside in memory simultaneously, occupying about 61 GB of RAM. These are the only figures relevant to capacity; nothing beyond them is claimed.
Model size and addressable footprint
The 120‑billion‑parameter dense model can be loaded and decoded on this hardware, but its weight, activation memory, and the OS must fit within the 251 GB RAM plus 31 GB swap plus 32 GB VRAM. If a frontier model exceeds the total addressable address space, the loader will fail or the process will be terminated by the kernel for exceeding limits. There is no automatic compression or tiered storage for model weights beyond the page cache; the model either fits or it does not. The reader sees: the box can hold models up to the size of its total memory budget. Larger models simply will not load.
A 30‑billion‑parameter mixture‑of‑experts model, when run CPU‑only on four owned cores, takes about 500 seconds to write a 1,500‑word article. This time is a measured outcome on the given hardware with no other processes competing for those cores. If the model is larger or additional work shares the cores, the elapsed time increases, but no computed slowdown factor is stated—only the 500‑second baseline for the described configuration.
Token throughput under core contention
Thread placement dominates CPU inference. The 120‑billion‑parameter dense model decoded at 21.96 tokens/sec when four threads were pinned to cores that nothing else used. When the same model ran on sixteen threads shared with other work, the rate fell to 1.68 tokens/sec. Same model, same prompt. The mechanism is core allocation: pinned threads receive exclusive use of physical cores, allowing the model to issue instructions without OS‑mediated time‑slicing. Shared threads contend for execution units, and the scheduler interleaves other processes, reducing the effective token rate. The reader sees two measured values on the identical model and prompt—21.96 and 1.68—and the gap between them is not a computed percentage but an observed difference in core exclusivity.
Attention placement and lane saturation
With attention placed on the CPU instead of the GPU, that lane’s decoded token rate falls to 0.094 tokens/sec. This was measured on the same model and prompt, with the attention computation routed through the CPU lane. The GPU lane, by contrast, benefits from parallel matrix‑multiplication kernels that the general‑purpose cores do not replicate at the same scale; the CPU must perform attention through sequential passes over the sequence length, which caps tokens per second at the measured 0.094. No ratio is stated between the CPU and GPU lanes; the contrast is left for the reader to observe from the numbers provided.
Burst capacity and consistent latency
The box provides 16 physical cores and 32 threads. There is no provision for elastic burst capacity—no additional hardware spins up when demand spikes, no auto‑scaling pool of GPUs or CPUs beyond what is physically installed. If the operator pins four threads and keeps the package quiet, the model decodes at 21.96 tokens/sec. If other processes occupy those cores, the rate falls to 1.68 tokens/sec. There is no middle ground offered by the system; the token rate is determined by the actual core allocation at the moment the prompt is processed. The mechanism is purely the OS scheduler’s placement of threads on physical cores. The reader sees that consistency depends on manual pinning, and that burst capacity beyond the installed 16 cores does not exist.
Model loading and cold‑start latency
Loading a model whose weights are already present in the page cache takes 1.77 seconds. This was measured on the box’s own hardware. If the weights are not cached, the load proceeds from storage or network boot, a path that is not measured here but is understood to be substantially slower. The mechanism is page‑cache residency: once weights have been read into RAM, subsequent loads avoid disk I/O. The reader sees that the fast load path is 1.77 seconds, and that cold starts carry a latency the box does not hide behind a managed service.
Uptime and operational responsibility
This sovereign AI box is privately delivered. There is no SLA, no provider‑side incident response, and no guaranteed uptime percentage. If the power supply fails, the network segment goes down, or the motherboard reports an error, the system is offline until the owner repairs or replaces the component. By contrast, a rented inference API documents an uptime commitment (even if imperfect, it is a contract term). The mechanism here is plain ownership: you are the operator, the SLA is self‑imposed, and the box carries no external guarantee. The reader sees that the cost of ownership includes the risk of downtime without recourse to a provider.