What to do if Slab cache grows and overall server performance is bad

The issue description:

Say, you have a large partition (1Tb or higher), for example /home, which is almost full. This means that there are a lot of user’s files on it, and when some software like backup scans all these files, Slab cache starts to grow. Here is an example of /proc/meminfo:

# cat /proc/meminfo
MemTotal:       131822716 kB
MemFree:         8396188 kB
MemAvailable:   125324428 kB
Buffers:           18040 kB
Cached:            94444 kB
SwapCached:       741444 kB
Active:           873220 kB
Inactive:         972620 kB
Active(anon):     818524 kB
Inactive(anon):   925616 kB
Active(file):      54696 kB
Inactive(file):    47004 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      31999996 kB
SwapFree:       20412420 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:       1228792 kB
Mapped:            40932 kB
Shmem:              8124 kB
Slab:           120366508 kB
SReclaimable:   117102628 kB
SUnreclaim:      3263880 kB

Slab shows over 100Gb !

In this case, the overall server performance could decrease, especially when some heavy IO software is running.

There are a few settings to control the Slab cache size.

First, we recommend to set vm.vfs_cache_min_ratio to 0.

Important notice: setting vm.vfs_cache_min_ratio to 0 on the kernel version prior to 3.10.0-614.10.2.lve1.4.46 will crash it. So, you can safely set vm.vfs_cache_min_ratio to 0 if your server is running the kernel 3.10.0-614.10.2.lve1.4.46 or higher.

This parameter controls the minimum amount (in percentage) of dentry\inode cache which can not be reclaimed. The default value is 2, which is rather small, but with a lot of cgroups used in the system, this two percents can be a very big value.

The second parameter is vfs_cache_pressure, the default value is 100. Basically, this parameter controls how aggressively the kernel will try to shrink the dentry/inode cache, so setting it to a bigger value could help to reduce the Slab cache size. The value can be set to 500 or even 1000.