Linux查看程序被Killed的原因及分析

Linux查看程序被Killed原因

1.查看信息,输出最近被killed的进程

查看被Killed简短信息列表 egrep -i -r 'killed process' /var/log

[root@ecs-bcb3 ~]# egrep -i -r 'killed process' /var/log 
/var/log/messages:Jul 12 15:00:24 ecs-bcb3 kernel: Killed process 26399 (python3), UID 1002, total-vm:27455044kB, anon-rss:24933916kB, file-rss:712kB, shmem-rss:0kB
/var/log/messages:Jul 12 15:00:24 ecs-bcb3 kernel: Killed process 29412 (python3), UID 1002, total-vm:27455044kB, anon-rss:24934124kB, file-rss:780kB, shmem-rss:0kB
/var/log/messages:Jul 12 15:18:39 ecs-bcb3 kernel: Killed process 25017 (chromedriver), UID 1002, total-vm:218888kB, anon-rss:2312kB, file-rss:184kB, shmem-rss:0kB
/var/log/messages:Jul 12 15:18:39 ecs-bcb3 kernel: Killed process 30252 (chromedriver), UID 1002, total-vm:201472kB, anon-rss:2032kB, file-rss:104kB, shmem-rss:0kB

查看被Killed详细信息列表 dmesg | egrep -i -B100 'killed process'

[root@ecs-bcb3 ~]# dmesg | egrep -i -B100 'killed process'
[19095552.852409] [31573]  1002 31573   131893     2522     166        0             0 chrome
[19095552.852585] [31597]  1002 31597   144656     4247     168        0           200 chrome
[19095552.852756] [31599]  1002 31599   148317     2766     183        0             0 chrome

[19095552.869403] [26287]  1004 26287  1975489   186919     547        0             0 java
[19095552.869548] [27054]   999 27054     1154       23       8        0             0 sh
[19095552.869700] [27058]   999 27058     1110       16       8        0             0 df
[19095552.869842] Out of memory: Kill process 7429 (python3) score 357 or sacrifice child
[19095552.869991] Killed process 7429 (python3), UID 1002, total-vm:26420076kB, anon-rss:23508276kB, file-rss:920kB, shmem-rss:0kB
...

查看被Killed简短信息列表 journalctl -xb | egrep -i 'killed process'

12月 21 15:33:18 ecs-kms kernel: Killed process 2580 (chrome), UID 0, total-vm:862648kB, anon-rss:53044kB, file-rss:156kB, shmem-rss:2692kB
12月 21 15:33:18 ecs-kms kernel: Killed process 3000 (chrome), UID 0, total-vm:862136kB, anon-rss:49508kB, file-rss:16kB, shmem-rss:2508kB
12月 21 15:33:18 ecs-kms kernel: Killed process 3165 (chrome), UID 0, total-vm:859320kB, anon-rss:48372kB, file-rss:112kB, shmem-rss:2492kB
...

2.设定kill优先度

  • 2.1 完全关掉oom
sysctl vm.overcommit_memory=2
echo “vm.overcommit_memory=2” >> /etc/sysctl.conf
  • 2.2 出现 Out Of Memory 之后重启
sysctl vm.panic_on_oom=1
sysctl kernel.panic=X
echo “vm.panic_on_oom=1” >> /etc/sysctl.conf
echo “kernel.panic=X” >> /etc/sysctl.conf
  • 2.3 优先考虑kill这个进程
sudo echo 10>/proc/[PID]/oom_adj
  • 2.4 尽量不去kill这个进程
sudo echo -15>/proc/[PID]/oom_adj
  • 2.5 不会kill这个进程
sudo echo -17>/proc/[PID]/oom_adj

备注:

  • oom_adj的值在-16 到 +15之间,值越高被kill的优先度越高。
  • 当该值为-17时,系统将不会杀死指定pid的进程,而-16~15则会使得进程的 /proc/[pid]/oom_adj 值呈指数形式递增,即它们被杀掉的可能性呈指数递增。
  • 针对init(进程号为1)这个进程,无论该值设为多少都不会被杀

3.若上述命令被拒绝访问

命令修改成以下方式

bash -c "echo '10' | tee /proc/[PID]/oom_adj"
bash -c "echo '-15' | tee /proc/[PID]/oom_adj"
bash -c "echo '-17' | tee /proc/[PID]/oom_adj"

总结

以上为个人经验,希望能给大家一个参考,也希望大家多多支持恩蓝小号。

原创文章,作者:VUMAA,如若转载,请注明出处:http://www.wangzhanshi.com/n/777.html

(0)
VUMAA的头像VUMAA
上一篇 2024年12月17日 17:59:29
下一篇 2024年12月17日 17:59:32

相关推荐

发表回复

登录后才能评论