Using jq to parse JSON in shell scripts

JSON is common in Linux administration, from cloud APIs and monitoring tools to configuration files generated by deployment systems. Shell scripts can read this data with basic text utilities, but those approaches become unreliable when objects contain nested fields, escaped characters, or arrays.

jq provides a dependable command-line JSON parser for RHEL, CentOS, Fedora, and other Unix-like systems. It can select values, filter records, reshape documents, and produce plain text suitable for further shell processing.

Why jq is safer than text parsing

Tools such as grep, cut, and awk are useful for line-oriented data, but JSON has no requirement to place each value on a separate line. A formatted document, a compact document, and an object with reordered keys should all produce the same result. Parsing by text layout can fail in every case.

jq understands JSON types, including strings, numbers, booleans, arrays, and null values. Administrators already familiar with Linux command-line guide techniques can add it to pipelines without changing the surrounding shell workflow.

Installing jq on Linux

On Fedora and modern RHEL-based systems, install the package with:

sudo dnf install -y jq

Older CentOS systems using yum can use:

sudo yum install -y jq

Check the installation before adding it to an automation job:

jq --version

In controlled environments, package availability may depend on enabled repositories. A server in Sydney, Melbourne, or a smaller regional office should use the organisation’s approved repository mirror rather than downloading arbitrary binaries.

Reading objects and nested values

Consider this file named server.json:

{
  "hostname": "web-01",
  "region": "sydney",
  "resources": {
    "cpu": 4,
    "memory_gb": 16
  },
  "roles": ["nginx", "php-fpm"]
}

Read a top-level value with:

jq -r '.hostname' server.json

The -r option removes JSON quotation marks, producing web-01 instead of "web-01". Nested properties use a dot expression:

jq -r '.resources.memory_gb' server.json

When a field may be absent, provide a default:

jq -r '.resources.memory_gb // 0' server.json

This is useful for reports scheduled in AEST, where a missing metric should be recorded as zero rather than causing a later command to process the literal value null.

Filtering arrays and selecting records

Many API responses contain an array of objects. Given:

{
  "services": [
    {"name": "nginx", "enabled": true, "port": 443},
    {"name": "sshd", "enabled": true, "port": 22},
    {"name": "test-app", "enabled": false, "port": 8080}
  ]
}

List enabled services with:

jq -r '.services[] | select(.enabled == true) | .name' services.json

Select both a condition and a formatted result:

jq -r '
  .services[]
  | select(.enabled and .port == 443)
  | "\(.name):\(.port)"
' services.json

Filters are especially useful when checking firewall or Nginx inventory across Australian cloud workloads. A Sydney production account and a Melbourne disaster-recovery account can return different records while the same expression remains valid.

Passing shell variables safely

Avoid constructing JSON expressions by concatenating untrusted shell variables. Use --arg for strings and --argjson for values that should remain JSON numbers or booleans:

service_name="nginx"

jq -r --arg name "$service_name" '
  .services[]
  | select(.name == $name)
  | .port
' services.json

For generated JSON, jq -n creates a document without requiring an input file:

jq -n \
  --arg host "web-01" \
  --arg region "sydney" \
  '{hostname: $host, region: $region, status: "active"}'

This approach prevents quotation errors in names containing spaces or punctuation. It also suits scripts that build deployment payloads for local providers, including systems operating around NBN maintenance windows or end-of-financial-year change freezes.

Producing shell-friendly output

Use @csv when exporting records for spreadsheets or operational reports:

jq -r '
  .services[]
  | [.name, .port, .enabled]
  | @csv
' services.json

For one value per line, -r is usually sufficient. Avoid unquoted command substitution when values can contain whitespace:

hostname=$(jq -r '.hostname' server.json)
printf 'Host: %s\n' "$hostname"

When a script must process multiple fields, emit tab-separated output and use a controlled read loop:

jq -r '.services[] | [.name, .port] | @tsv' services.json |
while IFS=$'\t' read -r name port; do
  printf '%s listens on %s\n' "$name" "$port"
done

For API integrations and payload checks, concise JSON testing notes can complement local command-line validation before data reaches a production job.

Validating input and handling failures

Check whether a file contains valid JSON before using its fields:

if ! jq empty response.json; then
  printf 'Invalid JSON input\n' >&2
  exit 1
fi

A missing field and a malformed document are different problems. Test required values explicitly:

hostname=$(jq -er '.hostname' response.json) || {
  printf 'hostname is missing or invalid\n' >&2
  exit 1
}

The -e option makes jq return a failure status for false or null results. Combine it with set -Eeuo pipefail in important scripts, and log the original response when an API call fails. That record is valuable when tracing a supplier outage, an EOFY deployment issue, or a failed scheduled job.

Building a complete shell workflow

A small script can download JSON, validate it, and produce an administrator-friendly summary:

#!/usr/bin/env bash
set -Eeuo pipefail

input=${1:-services.json}

jq -e '.services | type == "array"' "$input" >/dev/null

jq -r '
  .services[]
  | select(.enabled == true)
  | "\(.name) listening on port \(.port)"
' "$input"

Keep the jq filter readable with a multiline string, and test it against empty arrays, missing keys, Unicode text, and unexpected data types. Store filters in separate .jq files when they become larger than a few lines.

The practical rule is simple: validate JSON first, use -r only when plain text is required, pass variables with --arg, and let jq handle structure instead of treating JSON as ordinary text.