H1: Model loading time is mostly not loading the model
The reader is deciding whether to own inference instead of renting it. The question this page answers is whether the time spent loading a model into memory is the dominant cost in your workflow, or whether the real decision point lies elsewhere— in how the model runs once loaded, in the hardware that runs it, or in the choices you make about ownership versus rental. I will address three aspects of the loading question: the page cache effect measured at 1.77 s, how startup staggering works when multiple models are loaded, and what a cold boot really costs in terms of I/O and time. All measurements I cite are from our own hardware: an AMD Ryzen Threadripper PRO 9955WX with 16 physical cores and 32 threads, 251 GB of RAM, one NVIDIA RTX 5090 with 32 GB of VRAM limited to a 600 W power draw, and 31 GB of swap space. Where a measurement from this list is directly relevant, I will state plainly that it was measured on our own hardware. Where it is not, I will describe what is true of the technology without claiming a measurement.
Page cache and the 1.77 s measurement
The most relevant measurement for this page is that loading a model whose weights were already in the page cache took 1.77 seconds. The page cache is the operating system's page cache for file-backed memory. Model weight files stored on disk are, after the first read, cached in this region of RAM. When a loading process requests those pages again, the kernel satisfies the request from the page cache without issuing a disk read. The 1.77 s figure represents the wall-clock time from the loading process's perspective: it invokes the load operation, the kernel returns the pages from cache, and the model's weights are resident in memory. This measurement was taken on our own hardware with the model's weights already present in the page cache from a prior access. If you are loading the same model repeatedly— as happens during development restarts, container restarts, or interactive use between prompts— the loading cost per iteration is approximately 1.77 s. This is small compared to the inference time of even a modest model and nearly zero next to the cost of generating tokens. If you are loading a model for the first time after a reboot, the 1.77 s figure does not apply; the first load bears the full cost of reading from storage, as I will describe in the cold-boot section.
Startup staggering
When more than one model is loaded in the same session, the page cache effect creates a staggering pattern across the load sequence. On this machine, three models resident at once occupied about 61 GB of RAM. If the weight files of those models share pages— for instance, common tokenizer embeddings or overlapping architectural blocks— the second and third loads will find many of their pages already present in the page cache and will each complete in approximately 1.77 s. If the models have completely disjoint weight sets, each cold load must read its full weight file from storage, and the time for each subsequent load depends on the storage device's throughput, the OS's read-ahead behavior, and how many I/O channels are available simultaneously. The reader should understand that 1.77 s is a lower bound on repeat loads; the actual time for a second or third model load will vary based on how much of each model's weight set is already present in page cache from previous loads. I am not computing a ratio between warm and cold cases, and I am not giving a percentage improvement. I am stating two measured situations— a warm repeat load at 1.77 s, and a cold load whose cost depends on your storage configuration— and letting the reader see the gap.
What a cold boot really costs
A cold boot means the page cache is empty. This occurs after a machine reboot, an operating system restart, or a power cycle. In this state, the model weights must be brought from storage into RAM before the model can be used. The cost consists of the I/O read time to transfer the weight file from disk to RAM, plus any additional processing the loading pipeline performs— decompression, format conversion, memory layout adjustment— before the model is ready for inference.
On the hardware described, the model weight files are large. A 120-billion-parameter dense model and a 30-billion-parameter mixture-of-experts model both have weight files that occupy multiple gigabytes of disk space. Reading a multi-gigabyte file from storage into RAM requires the storage subsystem to transfer the data at its sustained throughput. After the read completes, the loading pipeline processes the weights, which adds further time depending on the format and the inference engine. The 1.77 s measurement for a page-cache-hit load tells you the cost after the first successful read; the cold-boot total is the first read plus any pipeline overhead. I do not have a measured figure for the cold-boot total from this hardware, and I will not invent one. The reader who wants to know their own cold-boot cost should time a load after a fresh boot, with the same model and the same storage path, and compare it to the 1.77 s figure for a warm repeat load. The gap between the two will be significant, and the dominant factor will be the storage subsystem's ability to transfer the weight file from disk to RAM.
Code: checking your page cache state
$ cat /proc/meminfo | grep -i "MemAvailable\|Cached"
MemAvailable: 12345678 kB
Cached: 89101112 kB
This command shows how much of your RAM is filled with page cache, giving you a concrete way to assess whether your model weights are likely already present before a load. If the Cached value is large relative to your model's weight file size, subsequent loads will complete near the 1.77 s figure; if it is small, the next load will bear the full I/O cost.
The decision: owning versus renting inference
The reader is deciding whether to own inference instead of renting it. The loading-time cost is a factor, but it is rarely the dominant factor in this decision. If your workflow involves loading the same model repeatedly— batch processing, interactive chat, fine-tuning loops where the model remains in memory between prompts— the page cache makes subsequent loads nearly free. On this hardware, each repeat load after the first costs approximately 1.77 s. This is small enough that it should not be the primary consideration in your decision to own or rent.
If your workflow involves cold boots each time— powering on the machine, loading the model, running one inference, then shutting down— then the cold-boot I/O cost is the relevant number. This will be dominated by your storage device's ability to transfer the model's weight file from disk to RAM. It will be significantly larger than 1.77 s, typically in the many-seconds range for models of the sizes discussed here. Again, this is not usually the dominant cost; the inference time itself— how many tokens the model can generate per second— will be orders of magnitude larger than the loading time, whether warm or cold.
The decision between owning and renting inference should be based on other factors: the total cost of hardware versus recurring rental fees, the degree of control you need over the software stack, the privacy requirements that make local execution necessary, and the inference performance you need for your workload. The loading time, whether measured at 1.77 s for a warm repeat or at many seconds for a cold boot, is mostly a one-time cost that amortizes quickly if the model stays in use. It is not the bottleneck most people assume it to be.