How to Use curl and wget for HTTP Requests

Linux administrators use curl and wget to inspect websites, download files, test APIs, and troubleshoot application servers. Both tools work well from a terminal on RHEL, CentOS Stream, Fedora, and other Unix-like systems, but they suit different tasks.

curl is designed for flexible HTTP requests and detailed protocol testing. wget is especially convenient for downloading files, following links, and recovering interrupted transfers. Knowing a few reliable options makes everyday server work faster and easier, whether a system runs in a Sydney data centre or on a workstation connected through the NBN.

Install And Check The Tools

On Fedora, RHEL, and CentOS Stream, install both packages with DNF:

sudo dnf install -y curl wget

Check the installed versions before troubleshooting a command copied from documentation:

curl --version
wget --version

A basic request displays the response body in the terminal:

curl https://example.com
wget -qO- https://example.com

The -qO- combination tells wget to run quietly, write the response to standard output, and avoid creating a local file. This is useful when checking a health endpoint from a server in Melbourne or Brisbane.

Inspect HTTP Responses With curl

Use curl -I to request headers without downloading the complete page:

curl -I https://example.com

The output can reveal the status code, redirect location, content type, cache settings, and server software. For verbose connection details, including TLS negotiation and request headers, use:

curl -v https://example.com

To follow redirects and impose sensible connection limits, combine -L, --connect-timeout, and --max-time:

curl -L --connect-timeout 5 --max-time 20 https://example.com

A status-only check can be placed in a monitoring script:

curl -sS -o /dev/null -w '%{http_code}\n' https://example.com

Australian sites may route users through regional CDNs, so a response tested during AEST business hours can differ from one returned by an overseas host. The status code and timing fields help separate an application failure from a network path problem.

Send Data And Headers

For an API request, specify the method, content type, and JSON payload explicitly:

curl -X POST https://api.example.com/users \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d '{"name":"Alex","role":"admin"}'

Credentials should not be placed directly in shell history. Prefer an environment variable, a protected configuration file, or an authentication mechanism documented by the application:

curl -H "Authorization: Bearer $API_TOKEN" \
  -H 'Accept: application/json' \
  https://api.example.com/profile

For form submissions, --data-urlencode safely handles spaces and special characters:

curl --data-urlencode 'search=network maintenance' \
  https://example.com/search

When testing services subject to the Australian Privacy Act 1988, avoid sending real customer details in diagnostic requests. Use synthetic records and remove tokens from logs before sharing output with a vendor or an internal support team.

Download Files Reliably With wget

wget saves a remote object using its original filename:

wget https://downloads.example.com/releases/app-2.4.1.tar.gz

Choose the destination name with -O, or continue an interrupted transfer with -c:

wget -O app.tar.gz https://downloads.example.com/releases/app-2.4.1.tar.gz
wget -c https://downloads.example.com/releases/large-backup.tar.gz

For unattended jobs, use retries, a timeout, and a log file:

wget --tries=4 --timeout=15 \
  --waitretry=5 \
  -o download.log \
  https://downloads.example.com/patch.iso

Always verify downloads when a project publishes a checksum:

sha256sum app.tar.gz

A checksum comparison is particularly valuable when a large file travels over a congested NBN connection or across a long route to an Australian mirror.

Practical Options And Safety Checks

Use these options frequently when building scripts or diagnosing a web server:

• curl -sS suppresses progress while retaining useful errors.
• curl -f returns failure for HTTP 4xx and 5xx responses.
• wget --spider checks a URL without downloading its content.
• wget --user-agent='Linux-admin-check' identifies a controlled test client.

For repeatable automation, keep commands small and make failures visible:

if curl -fsS --max-time 10 https://example.com/health >/dev/null; then
  echo "service is healthy"
else
  echo "service check failed" >&2
  exit 1
fi

Use --limit-rate when downloading during a shared office connection, and respect robots.txt, authentication rules, and the owner’s acceptable-use policy when retrieving multiple pages. For broader Nginx and Apache troubleshooting, the site’s web server guides provide related configuration context.

Common Mistakes To Avoid

• Treating a successful TCP connection as proof of an HTTP 200 response.
• Following redirects without checking the final hostname.
• Storing passwords or bearer tokens in shell history.
• Downloading unverified packages into a privileged directory.

Useful Testing Patterns

• Check headers with curl -I before changing Nginx configuration.
• Save verbose output with curl -v 2>curl-debug.log.
• Preview a download using wget --spider.
• Compare a downloaded file with its published SHA-256 value.

Troubleshoot Permissions And Automated Tests

A request can work from an interactive shell but fail inside a service because of SELinux, proxy settings, DNS, or a restricted environment. Check the effective user, environment variables, and audit records before disabling security controls. This SELinux boolean guide explains how to permit specific application behaviour while keeping enforcement enabled.

For an application test, capture the request and response separately:

curl -sS -D response-headers.txt \
  -o response-body.json \
  https://example.com/api/health

This makes failures easier to compare in CI systems running in Sydney or Perth. Browser-oriented projects can also use a browser test framework when a check needs JavaScript rendering rather than a direct HTTP client.

Build A Repeatable HTTP Check

Start with a harmless public or internal health endpoint, then record the expected status, content type, and maximum response time. A simple first test is:

curl -fsS -o /dev/null \
  -w 'status=%{http_code} time=%{time_total}s\n' \
  --max-time 10 \
  https://example.com/health

Run it manually, then place it in a controlled cron job or systemd timer with a log destination and a non-privileged account. Before deploying the check, execute that exact command now and confirm it prints status=200 within the chosen timeout.