Concepts

Edit This Page

Limit Ranges

By default, containers run with unbounded compute resources on a Kubernetes cluster. With Resource quotas, cluster administrators can restrict the resource consumption and creation on a namespace basis. Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace’s resource quota. There is a concern that one Pod or Container could monopolize all of the resources. Limit Range is a policy to constrain resource by Pod or Container in a namespace.

A limit range, defined by a LimitRange object, provides constraints that can:

Enabling Limit Range

Limit Range support is enabled by default for many Kubernetes distributions. It is enabled when the apiserver --enable-admission-plugins= flag has LimitRanger admission controller as one of its arguments.

A limit range is enforced in a particular namespace when there is a LimitRange object in that namespace.

Overview of Limit Range:

Examples of policies that could be created using limit range are:

In the case where the total limits of the namespace is less than the sum of the limits of the Pods/Containers, there may be contention for resources; The Containers or Pods will not be created.

Neither contention nor changes to limitrange will affect already created resources.

Limiting Container compute resources

The following section discusses the creation of a LimitRange acting at Container Level. A Pod with 04 containers is first created; each container within the Pod has a specific spec.resource configuration
each container within the pod is handled differently by the LimitRanger admission controller.

Create a namespace limitrange-demo using the following kubectl command

kubectl create namespace limitrange-demo

To avoid passing the target limitrange-demo in your kubectl commands, change your context with the following command

kubectl config set-context --current --namespace=limitrange-demo

Here is the configuration file for a LimitRange object:

admin/resource/limit-mem-cpu-container.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: limit-mem-cpu-per-container
spec:
  limits:
  - max:
      cpu: "800m"
      memory: "1Gi"
    min:
      cpu: "100m"
      memory: "99Mi"
    default:
      cpu: "700m"
      memory: "900Mi"
    defaultRequest:
      cpu: "110m"
      memory: "111Mi"
    type: Container

This object defines minimum and maximum Memory/CPU limits, default cpu/Memory requests and default limits for CPU/Memory resources to be apply to containers.

Create the limit-mem-cpu-per-container LimitRange in the limitrange-demo namespace with the following kubectl command.

kubectl create -f https://k8s.io/examples/admin/resource/limit-mem-cpu-container.yaml -n limitrange-demo
 kubectl describe limitrange/limit-mem-cpu-per-container -n limitrange-demo
Type        Resource  Min   Max   Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---   ---   ---------------  -------------  -----------------------
Container   cpu       100m  800m  110m             700m           -
Container   memory    99Mi  1Gi   111Mi            900Mi          -

Here is the configuration file for a Pod with 04 containers to demonstrate LimitRange features :

admin/resource/limit-range-pod-1.yaml
apiVersion: v1
kind: Pod
metadata:
  name: busybox1
spec:
  containers:
  - name: busybox-cnt01
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt01; sleep 10;done"]
    resources:
      requests:
        memory: "100Mi"
        cpu: "100m"
      limits:
        memory: "200Mi"
        cpu: "500m"
  - name: busybox-cnt02
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt02; sleep 10;done"]
    resources:
      requests:
        memory: "100Mi"
        cpu: "100m"
  - name: busybox-cnt03
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt03; sleep 10;done"]
    resources:
      limits:
        memory: "200Mi"
        cpu: "500m"
  - name: busybox-cnt04
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt04; sleep 10;done"]

Create the busybox1 Pod :

kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-1.yaml -n limitrange-demo

Container spec with valid CPU/Memory requests and limits

View the busybox-cnt01 resource configuration

kubectl get  po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].resources"
{
  "limits": {
    "cpu": "500m",
    "memory": "200Mi"
  },
  "requests": {
    "cpu": "100m",
    "memory": "100Mi"
  }
}

Container spec with a valid CPU/Memory requests but no limits

View the busybox-cnt02 resource configuration

kubectl get  po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].resources"
{
  "limits": {
    "cpu": "700m",
    "memory": "900Mi"
  },
  "requests": {
    "cpu": "100m",
    "memory": "100Mi"
  }
}

Container spec with a valid CPU/Memory limits but no requests

View the busybox-cnt03 resource configuration

kubectl get  po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].resources"
{
  "limits": {
    "cpu": "500m",
    "memory": "200Mi"
  },
  "requests": {
    "cpu": "500m",
    "memory": "200Mi"
  }
}

Container spec with no CPU/Memory requests/limits

View the busybox-cnt04 resource configuration

kubectl get  po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].resources"
{
  "limits": {
    "cpu": "700m",
    "memory": "900Mi"
  },
  "requests": {
    "cpu": "110m",
    "memory": "111Mi"
  }
}

All containers defined in the busybox Pod passed LimitRange validations, this the Pod is valid and create in the namespace.

Limiting Pod compute resources

The following section discusses how to constrain resources at Pod level.

admin/resource/limit-mem-cpu-pod.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: limit-mem-cpu-per-pod
spec:
  limits:
  - max:
      cpu: "2"
      memory: "2Gi"
    type: Pod

Without having to delete busybox1 Pod, create the limit-mem-cpu-pod LimitRange in the limitrange-demo namespace

kubectl apply -f https://k8s.io/examples/admin/resource/limit-mem-cpu-pod.yaml -n limitrange-demo

The limitrange is created and limits CPU to 2 Core and Memory to 2Gi per Pod.

limitrange/limit-mem-cpu-per-pod created

Describe the limit-mem-cpu-per-pod limit object using the following kubectl command

kubectl describe limitrange/limit-mem-cpu-per-pod
Name:       limit-mem-cpu-per-pod
Namespace:  limitrange-demo
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Pod         cpu       -    2    -                -              -
Pod         memory    -    2Gi  -                -              -

Now create the busybox2 Pod.

admin/resource/limit-range-pod-2.yaml
apiVersion: v1
kind: Pod
metadata:
  name: busybox2
spec:
  containers:
  - name: busybox-cnt01
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt01; sleep 10;done"]
    resources:
      requests:
        memory: "100Mi"
        cpu: "100m"
      limits:
        memory: "200Mi"
        cpu: "500m"
  - name: busybox-cnt02
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt02; sleep 10;done"]
    resources:
      requests:
        memory: "100Mi"
        cpu: "100m"
  - name: busybox-cnt03
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt03; sleep 10;done"]
    resources:
      limits:
        memory: "200Mi"
        cpu: "500m"
  - name: busybox-cnt04
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt04; sleep 10;done"]
kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-2.yaml -n limitrange-demo

The busybox2 Pod definition is identical to busybox1 but an error is reported since Pod’s resources are now limited

Error from server (Forbidden): error when creating "limit-range-pod-2.yaml": pods "busybox2" is forbidden: [maximum cpu usage per Pod is 2, but limit is 2400m., maximum memory usage per Pod is 2Gi, but limit is 2306867200.]
kubectl get  po/busybox1  -n limitrange-demo -o json | jq ".spec.containers[].resources.limits.memory" 
"200Mi"
"900Mi"
"200Mi"
"900Mi"

busybox2 Pod will not be admitted on the cluster since the total memory limit of its container is greater than the limit defined in the LimitRange. busybox1 will not be evicted since it was created and admitted on the cluster before the LimitRange creation.

Limiting Storage resources

You can enforce minimum and maximum size of storage resources that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange.

admin/resource/storagelimits.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: storagelimits
spec:
  limits:
  - type: PersistentVolumeClaim
    max:
      storage: 2Gi
    min:
      storage: 1Gi

Apply the YAML using kubectl create.

kubectl create -f https://k8s.io/examples/admin/resource/storagelimits.yaml -n limitrange-demo 
limitrange/storagelimits created

Describe the created object,

kubectl describe limits/storagelimits  

the output should look like

Name:                  storagelimits
Namespace:             limitrange-demo
Type                   Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----                   --------  ---  ---  ---------------  -------------  -----------------------
PersistentVolumeClaim  storage   1Gi  2Gi  -                -              -
admin/resource/pvc-limit-lower.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-limit-lower
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Mi
kubectl create -f https://k8s.io/examples/admin/resource//pvc-limit-lower.yaml -n limitrange-demo

While creating a PVC with requests.storage lower than the Min value in the LimitRange, an Error thrown by the server

Error from server (Forbidden): error when creating "pvc-limit-lower.yaml": persistentvolumeclaims "pvc-limit-lower" is forbidden: minimum storage usage per PersistentVolumeClaim is 1Gi, but request is 500Mi.

Same behaviour is noted if the requests.storage is greater than the Max value in the LimitRange

admin/resource/pvc-limit-greater.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-limit-greater
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
kubectl create -f https://k8s.io/examples/admin/resource/pvc-limit-greater.yaml -n limitrange-demo
Error from server (Forbidden): error when creating "pvc-limit-greater.yaml": persistentvolumeclaims "pvc-limit-greater" is forbidden: maximum storage usage per PersistentVolumeClaim is 2Gi, but request is 5Gi.

Limits/Requests Ratio

If LimitRangeItem.maxLimitRequestRatio if specified in th LimitRangeSpec, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value

the following LimitRange enforces memory limit to be at most twice the amount of the memory request for any pod in the namespace.

admin/resource/limit-memory-ratio-pod.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: limit-memory-ratio-pod
spec:
  limits:
  - maxLimitRequestRatio:
      memory: 2
    type: Pod
kubectl apply -f https://k8s.io/examples/admin/resource/limit-memory-ratio-pod.yaml

Describe the LimitRange with the following kubectl command:

$ kubectl describe limitrange/limit-memory-ratio-pod
Name:       limit-memory-ratio-pod
Namespace:  limitrange-demo
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Pod         memory    -    -    -                -              2

Let’s create a pod with requests.memory=100Mi and limits.memory=300Mi

admin/resource/limit-range-pod-3.yaml
apiVersion: v1
kind: Pod
metadata:
  name: busybox3
spec:
  containers:
  - name: busybox-cnt01
    image: busybox
    resources:
      limits:
        memory: "300Mi"
      requests:
        memory: "100Mi"
kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-3.yaml

The pod creation failed as the ratio here (3) is greater than the enforced limit (2) in limit-memory-ratio-pod LimitRange

Error from server (Forbidden): error when creating "limit-range-pod-3.yaml": pods "busybox3" is forbidden: memory max limit to request ratio per Pod is 2, but provided ratio is 3.000000.

Clean up

Delete the limitrange-demo namespace to free all resources

kubectl delete ns limitrange-demo

Examples

What's next

See LimitRanger design doc for more information.

Feedback