MDS hangs on scanning

Issue

MDS stuck when scanning against large databases with the following symptoms:

• Fatal error: Allowed memory size of XXXXX bytes exhausted (tried to allocate XXXX bytes) in /opt/ai-bolit/imunify_dbscan.php 
• Imunify scan process stuck and can't progress.

Environment

• Imunify360 6.5.3.* and lower
• Malware Database Scanner
• Any supported panel
• Any supported OS

Solution

This is a known problem (internal id DEF-19230 can be tracked in changelog) that was fixed in Imunify360 6.4. It is enough to update Imunify360 on the server to the latest stable release to get MDS working correctly again: https://docs.imunify360.com/update/#gradual-roll-out 

If there is no possibility to install updates for Imunify360, the following workaround can be applied for version 6.3.* and lower:

1. Remove the immutable bit from the file /opt/ai-bolit/imunify_dbscan.php:

# chattr -i /opt/ai-bolit/imunify_dbscan.php

2. Increase the memory limit to 1024M in the /opt/ai-bolit/imunify_dbscan.php file, after the first <?php add the following and save the file.

ini_set('memory_limit', '1024M');

3. After the changes back immutable bit again:

# chattr +i /opt/ai-bolit/imunify_dbscan.php

Also, up to version 6.5.3, there is a similar issue might appear with the Allowed memory limit that also can be checked with a manual scan:

/opt/ai-bolit/wrapper /opt/ai-bolit/imunify_dbscan.php --scan --path /home/user11 \
--avdb /var/imunify360/files/sigs/v1/aibolit/mds-ai-bolit-hoster.db \
--report-file `pwd`/report.json --procudb /var/imunify360/files/sigs/v1/aibolit/mds-procu2.db \
--memory 2048M

It results in:

PHP Warning: PHP Startup: Unable to load dynamic library 'sqlite3.so' 
(tried: /opt/alt/php-internal/usr/lib64/php/modules/sqlite3.so 
(/opt/alt/php-internal/usr/lib64/php/modules/sqlite3.so: cannot open shared object file: 
No such file or directory), 
/opt/alt/php-internal/usr/lib64/php/modules/sqlite3.so.so 
(/opt/alt/php-internal/usr/lib64/php/modules/sqlite3.so.so: 
cannot open shared object file: No such file or directory)) in Unknown on line 0

We decreased the limit of SQL query from 3000 to 2000 so that on each scan iteration we get fewer entries to store in memory and therefore consume less memory. It will result in a few more interactions with MySQL Database, although with these changes it is anticipated Databases will be scanned with less or no Allowed memory size issues:

grep QUERY_DATA_PART_LIMIT /opt/ai-bolit/imunify_dbscan.php
const QUERY_DATA_PART_LIMIT = 2000;

The patch will be released in 6.7.

Useful links

• https://docs.imunify360.com/update/#gradual-roll-out
• https://blog.imunify360.com/release-notes-imunify360-v.6.4