vm.swappiness and OOM in RHEL6

Posted in: Open Source, Site Reliability Engineering, Technical Track

The behavior of vm.swappiness was always a bit confusing for novice linux users, as setting vm.swappiness to 0 would not completely disable swapping in the system during a memory crunch. vm.swappiness would only affect the agressiveness of swapping.

Following upstream commit tried to give more control to parameter. This commit tried to avoid swapping completely when vm.swappiness is set to 0.

commit fe35004fbf9eaf67482b074a2e032abb9c89b1dd
Author: Satoru Moriya
Date: Tue May 29 15:06:47 2012 -0700
mm: avoid swapping out with swappiness==0

With above commit, setting vm.swappiness to “0” instructs the kernel not to initiate swapping until the amount of free and file-backed pages is less than the high water mark in a memory zone. In other words, it tries to reclaim as much memory that can be reclaimed, before swapping starts.
This greatly reduced the chances of swapping.

When this commit was included in RHEL6, it caused OOM kill situations in servers with some memory pressure, even when there was enough reclaimable memory available in the system.

 

* Mon Aug 27 2012 Jarod Wilson [2.6.32-303.el6] 
[..] 
– [mm] avoid swapping out with swappiness==0 (Satoru Moriya) [787885]

This happened as kernel was not able to reclaim pagecache fast enough in zones. This blocked the scanning and swapping of anonymous pages, as the scanning of anonymous pages would only start if free and file-backed pages is less than the high water mark in a memory zone as per above commit. OOM kill is the result.

This behavior is fixed with following patch. This means that we can set vm.swappiness to 0 without issues, if RHEL kernel is above 2.6.32-642.el6.

* Tue Jan 12 2016 Aristeu Rozanski [2.6.32-600.el6] 
[..] 
– [mm] vmscan: avoid OOM killing when swappiness == 0 (Jerome Marchand) [1126228]

This patch avoids OOM kill by forcing the scan of anonymous pages when an OOM kill is imminent, while avoiding the swapping out of anonymous pages under moderate memory pressure.

In summary, the bug which causes vm.swappiness=0 to OOM in RHEL6 kernels is only limited to kernels between 2.6.32-303.el6 and 2.6.32-600.el6 .

The next publicly released versions of above kernels are 2.6.32-642.el6 and 2.6.32-358.el6.

email

Interested in working with Minto? Schedule a tech call.

About the Author

Lead Site Reliability Consultant
Minto Joseph is an expert in opensource technologies with a deep understanding of Linux. This allows him to troubleshoot issues from kernel to the application layer. He also has extensive experience in debugging Linux performance issues. Minto uses his skills to architect, implement and debug enterprise environments.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *