CPU and Memory Capacity

Budget the whole process

The model file is only one part of memory use. The runtime also needs loaded weights, temporary working areas called buffers, the Python process, vector indexes, and operating-system pages. It keeps a key-value cache (KV cache) of earlier token calculations so it does not redo them for every new token. A longer context or more simultaneous requests makes that cache larger.

On this shared platform, Python-only AI labs receive a 1 gibibyte (GiB) memory ceiling. Warmup and the generation lab receive 2 GiB because they can load Qwen2.5:0.5b or SmolLM2-135M. All tiers use half of one CPU and one model request at a time. These ceilings stop one learner's experiment from taking memory needed by the PostgreSQL database, Redis cache, or other learners.

Warning: Never solve an out-of-memory incident by silently switching to a larger model. Reduce model size, context, concurrency, or dependency footprint and measure again.