Website not loading because of conntrack limit

Issue

Initially, the problem appeared as port 443 being filtered from external scans (e.g. nmap), while HTTP (port 80) remained accessible.

After further analysis, we identified that the server was hitting nf_conntrack table exhaustion, confirmed by kernel messages such as:

nf_conntrack: table full, dropping packet

At the time:

nf_conntrack_count was very close to nf_conntrack_max

New incoming connections (including HTTPS) were being dropped at the kernel level

This resulted in connection timeouts that externally appeared as a filtered port

We have taken the following actions:

Increased net.netfilter.nf_conntrack_max to 524288

Began adjusting connection timeouts (the default nf_conntrack_tcp_timeout_established was set to 432000 seconds)

Despite these changes, the issue is still intermittently affecting some domains over HTTPS.

We would like your assistance to verify if any CloudLinux-specific components could be contributing to this behavior, such as:

• LVE limits impacting network handling
• Integration with iptables / iptables-legacy
• Any known issues with conntrack handling under CloudLinux 8/9 kernels

Environment

• CloudLinux

Solution

Add the following items to /etc/sysctl.conf:

# 1. Enable SYN Cookies (CRITICAL)

# This allows the kernel to handle SYN floods natively and efficiently, preventing CPU spikes from CSF's iptables-based SYNFLOOD protection.

net.ipv4.tcp_syncookies = 1

# 2. Increase the SYN Backlog

# Gives the kernel a larger queue for half-open connections before it has to start relying on SYN cookies or dropping packets.

net.ipv4.tcp_max_syn_backlog = 4096

# 3. Increase the Socket Listen Queue

# The default is often 128. Increasing this helps Apache/LiteSpeed handle sudden spikes in legitimate traffic.

net.core.somaxconn = 4096

# 4. Drop Idle Established Connections

# Reduces the time a connection can sit idle in the established state from the default (often 5 days) to 10 minutes (600s). This prevents inactive connections from unnecessarily filling up the conntrack table.

net.netfilter.nf_conntrack_tcp_timeout_established = 600

# 5. Reduce TIME_WAIT Timeout

# Quickly clears connections that have been closed but are lingering in the TIME_WAIT state. Dropping this from the default 120s frees up slots faster.

net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30

# 6. Reduce CLOSE_WAIT Timeout

# Accelerates the removal of connections where the remote end has closed the connection, but the local server is waiting for the application to finish.

net.netfilter.nf_conntrack_tcp_timeout_close_wait = 30

# 7. Reduce FIN_WAIT Timeout

# Drops connections faster when the server has sent a FIN packet to close the connection but is waiting for the client's acknowledgment.

net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 30

Once you add these, just remember to run sysctl -p to apply the changes immediately.