Linux – Is there a way to retrieve temporary storage usage details for kubernetes containers?

I

created some pods with containers for which I set ephemeral-storage requests and limits, for example: (10GB here).

Unfortunately, for some containers, ephemeral-storage will be completely filled for unknown reasons. I want to know which directories/files are responsible for populating everything, but I didn’t find a solution.

I tried using df -h, but unfortunately it gives statistics for the entire node, not just specific pods/containers.

Is there a way to retrieve ephemeral-storage usage details for kubernetes containers?

Pods use temporary local storage for scratch space, cache, and logs. The kubelet can use local temporary storage to mount the emptyDir volume into a container to provide scratch space for the pod.

Depending on your Kubernetes platform,

you may not be able to easily determine where these files are written to, and any file system may fill up, but rest assured that the disk is being consumed somewhere (or worse, memory – depending on the specific configuration of your emptyDir and/or Kubernetes platform.

Quoting this SO about how default and allocable temporary storage in a standard Kubernetes environment is installed from the file system (mount to /var/ lib/kubelet).

Also refer to the Kubernetes documentation to learn how ephemeral storage can be managed & Ephemeral storage consumption management works .

I’m assuming you’re a GCP user and you can get an idea of how your temporary storage is being used:
Menu > Monitoring > Metrics Explorer >
Resource type: Kubernetes node & metric: Temporary storage

Try the following command to learn the details of temporary storage usage for Kubernetes pods/containers:

1. Trying du -sh/[run inside container]:d u -sh will provide the space occupied by the container file. It simply returns the amount of disk space used by the current directory and all these things in it as a whole, for example: 2.4G.

You can also use the du -h someDir command to check the full file size.

1. Inspecting container filesystems: You can use /bin/df as a monitor ephemeral The storage usage tool is on the same volume as the temporary container data, i.e. /var/lib/kubelet and /var/lib/containers.