Practical sysctl and kernel performance tuning

sysctl provides runtime access to many Linux kernel controls, including network queues, socket buffers, memory pressure, file-descriptor ceilings, and scheduling settings. Useful tuning begins with a workload and a measurable symptom; copying large values from another server is neither a diagnosis nor a safe default.

Australian teams commonly combine on-premises RHEL or CentOS systems with AWS, Azure, and Google Cloud instances, often backed by local reseller support and Red Hat subscriptions. Sydney, Melbourne, and Perth offices can present very different WAN paths to overseas services. During end-of-financial-year processing in June, when an arvo batch gets a proper squiz, network latency may be confused with a host that needs larger kernel limits.

Establish a baseline before changing values

Record the kernel version, CPU model, memory size, workload profile, and relevant service limits. Useful evidence includes application latency, CPU saturation, memory pressure, socket counts, packet drops, and I/O wait. On RHEL, CentOS Stream, and Fedora, compare the live values with the files under /etc/sysctl.d and /usr/lib/sysctl.d.

sysctl -w key=value changes a live value, while /etc/sysctl.d/99-performance.conf provides a persistent override on RHEL-family systems. Record the original value before applying it. Containers may expose a restricted or namespaced view, so a permission error does not necessarily indicate a faulty host configuration.

Choose workload-specific settings rather than generic “optimisation” bundles. Legacy software associated with DOS4GW compatibility has different memory and I/O assumptions from a native Linux daemon, so its historical limits should not dictate host-wide values.

Tune network queues and socket memory

High connection rates can exhaust the accept queue before an application accepts traffic. Reasonable starting points for testing include net.core.somaxconn=4096, net.ipv4.tcp_max_syn_backlog=4096, and net.core.netdev_max_backlog=16384. These are candidates, not universal defaults, and an application’s own listen backlog must match the intended capacity.

Use ss -s, nstat -az, sar -n DEV,TCP,ETCP, and packet-drop counters to confirm where pressure occurs. Raising queue sizes without checking CPU and scheduler latency can merely move congestion elsewhere. Persistent settings should remain in the same reviewed sysctl.d file rather than being scattered across scripts.

TCP socket memory is controlled by net.ipv4.tcp_rmem and net.ipv4.tcp_wmem, whose maximum values are bounded by net.core.rmem_max and net.core.wmem_max. Linux autotuning often works well, but high-latency paths to Europe or South-East Asia may justify testing another congestion control. Check net.ipv4.tcp_available_congestion_control and benchmark candidates such as BBR under realistic traffic.

Match Nginx and file-descriptor limits

Linux permits a process to open descriptors up to the applicable service limit, but the system also has a per-process ceiling controlled by fs.nr_open and a global allocation controlled by fs.file-max. Raising only fs.file-max does not fix an exhausted LimitNOFILE value in a systemd unit.

For Nginx, set a realistic LimitNOFILE in the service override and restart the service after reloading systemd. The relationship between the daemon, worker processes, and operating system is covered in the site's Nginx tuning notes, which are useful alongside the unit configuration.

Budget Nginx worker_connections conservatively because proxied requests can require both a client and upstream descriptor. The configured listen backlog should also remain within net.core.somaxconn. Leaving headroom protects administration tasks, logging, health checks, and the workers themselves when traffic approaches the limit.

Tune memory and write-heavy workloads

vm.swappiness controls how aggressively the kernel reclaims page cache in favour of anonymous memory. A low value, often between 1 and 10, can reduce swap latency on latency-sensitive servers, but it may also make memory pressure appear sooner. Check vmstat, sar -B, and memory-pressure metrics before changing it.

Transparent Huge Pages can improve some workloads but add latency for others. RHEL commonly uses transparent_hugepage=madvise, allowing applications to request huge pages when useful. Database servers and latency-sensitive services should verify their actual behaviour rather than relying on a global “always” or “never” setting.

Write-heavy hosts may require tuning of vm.dirty_background_ratio and vm.dirty_ratio, although newer RHEL systems can provide better control through cgroup writeback settings. Avoid setting vm.overcommit_memory=1 unless a specific application requires it, because unrestricted overcommit can defer failure until an out-of-memory event. Measure with iostat -x 1 and the I/O pressure information under /proc/pressure/io.

Handle routing, firewalls, and legacy rules

Strict reverse-path filtering, expressed through rp_filter=1, suits single-homed hosts. Multihomed routers and asymmetric return paths may need the effective interface policy set to the loose value of 2. Because the kernel considers both global and per-interface settings, changing only one may not produce the intended behaviour.

Enable net.ipv4.ip_forward only when the host genuinely routes traffic. Forwarding, NAT, redirects, and firewall policy must be considered together. On older RHEL systems that still rely on iptables, the legacy iptables guidance helps explain how older rule chains interact with routing changes.

Connection tracking also has a memory limit. Compare net.netfilter.nf_conntrack_count with net.netfilter.nf_conntrack_max when the module is loaded, and account for each tracked connection before increasing the ceiling. Settings such as accept_redirects and send_redirects affect network behaviour and security; they should not be changed merely because a generic tuning list recommends disabling them.

Apply, verify, and retain changes

Place reviewed overrides in /etc/sysctl.d/99-performance.conf, then test them with sysctl -p /etc/sysctl.d/99-performance.conf. Use sysctl --system when the full configured set must be reloaded, and verify individual keys with sysctl. After a reboot, confirm that the values survived and were not replaced by an image, cloud-init, or service-specific configuration.

Use the kernel command line only for genuine boot-time settings that are not exposed through sysctl. Such changes belong in the GRUB configuration under /etc/default/grub, require the boot configuration to be regenerated, and normally need a reboot. Arbitrary kernel parameters can cause boot failures or alter isolation in ways that are difficult to diagnose.

Compare metrics before and after under representative load, document the expected benefit, and retain a tested rollback value for every change. The practical takeaway is to treat every sysctl value as a capacity decision: change one measured limit, verify its effect under real traffic, and retain only settings you can explain and reverse.