Log Analysis on Linux with Grep and Regular Expressions

When an admin in Sydney or Melbourne opens a shell late on a Sunday arvo to chase a misbehaving service, the first tool out of the kit is almost always grep. Logs across RHEL, CentOS and Fedora systems quickly outgrow anything a human can read line by line, and regular expressions turn grep into a surgical instrument for tracing user behaviour and spotting break-ins.

The walkthrough below builds patterns step by step, starting with basic switches and finishing with a workflow that runs across multiple servers. The examples assume a Red Hat-family host, but the techniques apply to any systemd-based distribution and to logs forwarded from cloud instances in the Sydney or Melbourne regions where most local production workloads now sit.

Core grep syntax for log hunting

The simplest invocation, grep sshd /var/log/secure, prints every line that mentions the SSH daemon. Adding -i ignores case, and -n prints line numbers for cross-referencing with a helpdesk ticket.

For richer output, -E enables extended regular expressions without escaping each metacharacter. Combined with -c, grep returns a count, a fast way to confirm a service is generating traffic on a quiet Sunday morning. Local sysadmins supporting ATO workloads often wrap these switches into short scripts before paging through the noise.

Anchors, character classes and quantifiers

Anchors give a pattern a position. ^ matches the start of a line, so grep '^Oct 12' /var/log/messages isolates everything from a specific date. The $ anchor catches lines that end with a particular string, handy for spotting truncated proxy requests.

Character classes such as [0-9] narrow the search further. A pattern like grep -E 'fail[0-9]+' auth.log matches one or more digits after "fail", surfacing error codes without dragging in unrelated lines. Quantifiers control repetition, and parentheses group sub-patterns for reuse. These same regex building blocks also power the query syntax used in Android's storage layers, as the SharedPreferences preferences guide shows when applying anchoring logic to user-preference keys.

Hunting failed SSH attempts

Authentication logs are a daily concern for any internet-facing host. The Australian Cyber Security Centre regularly reminds teams to monitor SSH, and a quick pattern reveals the offenders. grep -E 'Failed password.*from [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' /var/log/secure matches every failed attempt with the source IP attached.

To isolate unique sources, use grep -oE '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$' and feed it into sort -u. If the same address appears thousands of times, it is often a Telstra or Optus customer behind a compromised modem rather than a coordinated attacker, worth checking before raising an incident.

Pulling web server data from Nginx and Apache

Web logs follow predictable formats, which makes them ideal regex targets. For Nginx's default access log, grep -E 'GET /api/v[0-9]+/' /var/log/nginx/access.log isolates API traffic, while grep -E ' 5[0-9]{2} ' catches any five-hundred response code regardless of path. Adding -B2 -A2 shows two lines of context either side, confirming whether a 502 came from upstream or the local listener.

On CentOS Stream hosts running Apache, the same patterns apply to the combined log format. Engineers at Canva or Atlassian often pipe these greps into small scripts that produce hourly summaries for Slack, all without a dedicated log aggregator. When an endpoint misbehaves, narrowing by user-agent with grep -E 'Mozilla|curl|wget' distinguishes browsers from automation, the first step before any blocklist action.

Following process activity with lsof

Sometimes grep tells you what is happening, but not which process is responsible. Pairing it with lsof closes the loop. After spotting repeated writes from a service, run lsof | grep -E '/var/log/app\.log' to identify the file descriptor and PID. The approach is laid out in the lsof guide for open files and processes.

This pairing is useful on production boxes in Perth or Brisbane where the only access is via a bastion host and the file system is read-only for most paths. Knowing the owning PID makes it easier to decide whether to restart the service, rotate the log, or escalate to the application team.

Persistent storage for rotated logs

Once the right entries have been found, preserving them somewhere durable matters. Local disks fill quickly, especially on hosts shipping logs to a central collector. Mounting an NFS share through systemd keeps the workflow consistent and survives reboots, exactly the kind of arrangement described in the systemd mount unit guide.

Pairing that mount with a cron job that greps, gzips and copies matched lines onto the share keeps audit trails available for the seven-year retention many Australian organisations follow. The same pattern supports PCI or IRAP evidence, where regulators expect original log lines to remain untouched.

Practical habits for repeatable log reviews

A reliable workflow combines the previous steps into a single script. Start with a date range using grep -E '^Oct (1[0-9]|2[0-9])', then layer in service filters and response-code filters, finishing with an IP or user extraction piped through sort | uniq -c | sort -rn so the most frequent matches sit at the top.

Saving the script as /usr/local/bin/logcheck.sh and running it daily from a central jump host gives the whole crew the same view. Consultancies supporting government clients in Canberra often standardise on this pattern so findings are reproducible across engagements, and storing the script in version control keeps a record of what was checked, which auditors appreciate.

A short checklist to fold into a daily routine:

• Start with grep -E rather than plain grep to keep regex syntax readable.
• Anchor patterns with ^ and $ whenever the log format is structured.
• Use -oE with sort | uniq -c to rank offenders without writing a parser.
• Pair grep with lsof whenever the source process is unclear.
• Store matched snippets on a systemd-managed mount for retention.
• Reuse the same patterns in staging before touching live systems.

A good way to start is to pick one log file on a local server today, run grep -E '^Oct' /var/log/secure | head -20 to see what the format looks like, and begin building a pattern from those first twenty lines.