KUBERNETES: How To Enable core_pattern On the Linux Host To Generate Coredump File For Pods/Containers Crash Issue

KUBERNETES: How To Enable core_pattern On the Linux Host To Generate Coredump File For Pods/Containers Crash Issue

A core_pattern defines the name and path for the core dump file if there is a crash of pods or containers due to segfault or system calls. Coredump file contains an image of the process’s memory at the time of termination. 

Below are steps for enabling core_pattern on the Linux hosts to generate coredump if the pod or container crashes.

1) Make a note of the default core pattern setting using below command.

 cat /proc/sys/kernel/core_pattern

Default value will be core.

2) Change the core pattern as follows:

sysctl -w "kernel.core_pattern=|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h"

3) Verify that the core handler changed to systemd core handler by runnign below command.

cat /proc/sys/kernel/core_pattern

You should see output as follows:

|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h

How to Check and collect the coredump in case of pods/containers crash

1) If the pods are in crashloopbackoff state or error state due to segfault or system calls from OS, check and identify pods corefile PID using below command.

coredumpctl list

Below is sample output of above command which shows testpod under EXE column. The pid for the pod is 1459 in this case

# coredumpctl list

TIMEPID   UID   GID SIG PRESENT EXE

Tue 2024-01-09 20:51:56 GMT1459 0 0   3 * /usr/local/bin/testpod

2) Once the PID is identified dump the core to a file using below command. In below command replace 1459 PID with the identified coredns pid.

coredumpctl dump 1459 > /tmp/coredump.out

/tmp/coredump.out will be your coredump file.

In case you want to revert back the core_pattern to default value which was there before, for e.g. to set it back to default core, run below command.

sysctl -w "kernel.core_pattern=core"