The resource limits in a container spec are the numbers most teams copy from a template and defend with a shrug, and the shrug costs money twice. The limits set too low throttle the application invisibly, the CPU quota that slices the response time into latency spikes nobody's dashboard explains, the memory cap that kills the pod at the worst moment of the day. The limits set too high waste the cluster's capacity, the node's schedulable memory consumed by the requests nobody uses, the bin packing that starves while it hoards. The practice that fixes both directions is the same one this series keeps teaching, measure the actual behavior, set the numbers from the distribution, and monitor the gap. This article builds the resource model properly, the requests and the limits as two different promises, the throttling and the OOM that the misconfigured numbers produce, the measurement workflow that produces honest numbers, and the JVM and runtime specifics that make the memory question harder than it looks.
Requests and limits as two different promises
The container spec's two numbers are two contracts with two different readers. The request is the scheduler's promise, the resource the pod is guaranteed, and the scheduler packs the node by the requests, the bin packing that fits the pods whose guarantees sum to the node's capacity. The limit is the runtime's ceiling, the maximum the container may use, and the two numbers create the three QoS classes the orchestrator uses for its eviction ordering, Guaranteed, the requests equal to the limits, Burstable, the requests below the limits, and BestEffort, the naked pod with neither, the first class evicted last and the last evicted first:
resources:
requests: { cpu: 250m, memory: 512Mi }
limits: { cpu: 1000m, memory: 1Gi }The request's honesty is the cluster's efficiency, the request set at the p50 of the actual usage packs the nodes tightly, and the request set at the p99 wastes the capacity the pod never touches. The limit's honesty is the application's isolation, the limit set above the p99.9 absorbs the spikes, and the limit set at the p99 turns every spike into the throttling or the kill that the next two sections describe. The working insight that summarizes the model, the request is for the scheduler and the limit is for the runtime, and the two numbers serve different masters with different math.
CPU throttling, the invisible latency tax
The CPU limit is enforced as a quota over a period, the cgroup's CFS bandwidth control, the container that exhausts its quota within the period waits until the next, and the waiting is the throttling. The throttled application's symptoms are the ones that confuse teams for weeks, the p99 latency that spikes in patterns, the application's own profiling that shows nothing wrong, the service that is fast on the developer's laptop and slow in the cluster, because the laptop has no quota and the cluster slices:
# the throttled container's own view of time
period: 100ms, quota: 50ms # the 500m limit
# 60ms of work in a 100ms window: 10ms spent waitingThe throttling detection is the cgroup's own counters, container_cpu_cfs_throttled_periods_total, the metric that names the invisible tax, and the dashboards that watch it are the ones that find the pods whose limit is the p99 of a bursty workload. The throttling fix is the measurement's conclusion, the limit raised to absorb the burst, or the code changed to smooth it, and the fix that deserves the emphasis is the second, because the limit that is raised to hide the garbage collection spike or the batch loop is the limit that raises the whole fleet's cost by one application's misbehavior.
The CPU request's counterpart failure is the noisy neighbor, the request set below the usage on the fully packed node, the pod competing for the cycles it was not guaranteed, and the node's overcommit that the scheduler permitted because the requests permitted it. The honest request is the usage's p50 to p70, the limit is the p99.9 of the burst, and the pair prices the application's real behavior rather than the template's folklore.
Memory limits and the OOMKill the logs never explain
The memory limit's failure is louder and simpler, the container that exceeds its cap is killed, the OOMKilled status, the exit code 137, the restart that the orchestrator performs and the application's own logs say nothing about, because the kernel's decision did not consult the process. The memory accounting includes what the application's own tools undercount, the page cache that the container's file reads accumulate, the memory mapped regions, the runtime's own overhead, and the memory usage that the application's profiler reports is the subset that the kernel actually bills.
# the killed pod's only testimony
LAST STATE: OOMKilled, exit code 137The memory request's failure is the scheduling starvation, the request set generously by the template, the node's allocatable memory consumed by requests the pods never use, and the cluster with the capacity headroom that cannot schedule because the packing is lying to it. The memory numbers' honest derivation is the same measurement discipline, the working set's p99 over the representative window, the request at the p50, the limit at the p99 plus the headroom for the spikes, and the numbers revisited with the application's changes, because the memory profile is not a constant but a property of the code's evolution.
The JVM and the runtimes that make memory hard
The memory question has a hard case the memory limit discussion must name, the JVM and the runtimes with the managed heaps, because the default memory sizing ignores the container and the runtime's own view of the machine's memory is the host's, the four gigabyte heap on the one gigabyte limit, the OOMKill on the first warm up. The container awareness is the modern default, the UseContainerSupport flag on the current JVMs reading the cgroup's limit, the MaxRAMPercentage that sizes the heap to the limit's fraction:
java -XX:MaxRAMPercentage=70 -jar app.jar
# the heap is 70% of the container limit, the rest for the metaspace,
# the threads, the direct buffers the heap accounting missesThe percentage's remainder is the honest engineering, the native memory that the heap ignores, the metaspace, the thread stacks, the direct buffers, the code cache, and the OOMKill that arrives despite the healthy heap is the native spike that the percentage left no room for, which is why the practice's number is seventy and not ninety. The GC logging and the native memory tracking belong in the diagnostic toolbox for exactly this class, the OOMKill whose heap dump shows a healthy heap, and the runtimes with the managed heaps all carry the same shape of the problem, the runtime's view and the kernel's view of the same number.
One throttled service diagnosed from the counter up
The throttling story makes the counter's value concrete. The search service's p99 latency spikes every hour, on the half hour, a pattern the application's profiling cannot reproduce and the database's slow log does not explain, and the investigation's first number is the throttled periods counter, the metric that spikes on the same schedule, the container that exhausts its quota every hour and waits. The second number is the usage trace, the CPU's consumption over the same window, and the spike's shape is the scheduled indexer, the batch job that runs inside the service on the hour, and the search traffic's own bursty baseline riding beside it. The arithmetic is the diagnosis, the 500m limit against the burst's 900m demand, every batch window a queue of throttled periods, and the user facing latency the batch's invisible tax.
The fix that the measurement suggests is the code's, the batch throttled at the application layer, the indexer yielding between its items, the batch that fits beside the traffic rather than competing with it, and the limit untouched because the traffic's p99 never needed the raise. The fix that the measurement would have hidden is the limit raised to 1200m, the batch's cost paid by the whole fleet's capacity forever, and the story's lesson is the counter's direction, the throttling metric is not only the limit's feedback, it is the code's, the number that names the misbehavior the latency dashboard only suffers.
The node and the overcommit policy, the cluster's side of the contract
The limits and requests compose into a cluster level policy, and the overcommit ratio is the operator's dial. The CPU overcommit, the sum of the node's CPU limits against its capacity, prices the burst tolerance, the 2 to 1 ratio that lets the average fleet absorb its spikes, the tighter ratio for the latency critical services, and the request packing that decides the ratio is the scheduler's arithmetic. The memory overcommit is the dangerous one, the limits that sum above the capacity are the OOMKill's booking, the node's memory that cannot be overcommitted the way the CPU is throttled, because the kernel's OOM is the eviction without the queue.
The QoS classes complete the policy, the Guaranteed pods that request what they limit, the Burstable majority, and the BestEffort tier that the node's pressure evicts first, and the policy that a cluster operator writes down is the composition, which workloads run Guaranteed, the databases and the critical paths, which run Burstable, the stateless majority, and the eviction ordering that the pressure event follows. The policy's honest form is the admission control, the policy engine that rejects the spec whose requests the node cannot honor, and the cluster that enforces the policy at admission is the cluster whose pressure events follow the ordering the operator wrote rather than the ordering the scheduler improvised.## The measurement workflow that produces honest numbers
The limits from measurement arrive from a workflow rather than a guess, and the steps are few. Load the application at the production shape, the traffic mix, the data volume, the concurrency, in the staging cluster, and observe the resource usage's distribution over the representative window, the metrics that the container runtime exports, the working set, the CPU's consumption over the periods. Read the percentiles rather than the averages, the p50 that sets the request, the p99.9 that informs the limit, the burst behavior that the max over the window reveals. Set the numbers, deploy, and watch the gap in production, the throttling counter that says the limit is tight, the OOMKill's absence that says the memory is honest, and the vertical pod autoscaler's recommendations that automate the same reading continuously for the workloads that change.
The working rules compress the practice, and the list is short:
- Set the requests from the measured p50 and the limits from the measured p99.9, never from the template;
- Watch the throttling counter and the OOMKill status as the limits' feedback loop;
- Size the managed runtimes to the container, the percentage flag and the native headroom;
- Revisit the numbers with the application's changes, the memory profile evolves with the code;
- Treat the requests as the cluster's efficiency contract and the limits as the isolation contract;
- Use the VPA's recommendations for the drift, and audit the fleet's requests quarterly.
The resource numbers are the application's handshake with the cluster, and the handshake from measurement is the one that holds, the CPU that bursts without the tax, the memory that spikes without the kill, and the node capacity that serves the fleet it actually promised to serve. The numbers from the template are the folklore's last stand in the cluster, and the measurement workflow retires it the same way this series retired every other guess, with the distribution, the percentile and the monitor on the gap.
The measured numbers, the enforced policy and the monitored gap together turn the resource spec from folklore into contract, and the contract is the cluster's efficiency and the application's isolation arriving in the same document, which was always the promise the two numbers were making.