Stop scrolling through raw JSON lines. Master command-line log querying using jq pipelines to diagnose errors, isolate slow queries, and monitor replication events instantly.
Starting in version 4.4, MongoDB moved to a fully structured JSON logging format. While this makes ingestion into enterprise platforms like ELK and Splunk seamless, it presents a challenge for database administrators using classic terminal tools. Commands like grep, awk, and sed struggle to handle nested properties or filter based on object values.
This is where jq comes in. Often described as the “Swiss Army knife” for JSON in shell scripts, jq is a lightweight command-line processor that lets you slice, query, filter, and transform JSON data on the fly. Let’s look at how to leverage it to extract logs, isolate slow queries, and solve issues.
- Pretty-Printing & Basic Structure
By default, MongoDB logs are flat single-line JSON records. Reading them raw in a standard terminal is practically impossible. The simplest way to inspect them is using jq’s basic identity filter (.), which pretty-prints the output with indentation and color coding.
|
cat mongod.log | jq ‘.’ | head -n 20 |
When formatted, you will notice that every MongoDB log contains a set of standard keys:
- t: Timestamp object (e.g., {“$date”: “…”})
- s: Severity level (e.g., “I” for Info, “W” for Warning, “E” for Error, “D” for Debug)
- c: Component (e.g., “COMMAND”, “NETWORK”, “STORAGE”, “REPL”)
- id: Unique numeric event identifier
- ctx: Context identifier (the specific connection thread, e.g., “conn1”)
- msg: Descriptive string message
- attr: A dynamic document containing message-specific attributes (the key area for detailed troubleshooting)
- Filtering by Severity Level
If you’re troubleshooting a server crash or a replication mismatch, you don’t care about Info messages. You only want to see Warnings (“W”) or Errors (“E”). We can achieve this using the select() function inside jq.
|
jq ‘select(.s == “E”)’ mongod.log |
This command evaluates each line and outputs the entire JSON log object only if its s property matches “E”. To filter by multiple levels, say Warnings and Errors, use logical operators:
|
jq ‘select(.s == “E” or .s == “W”)’ mongod.log |
- Isolating Specific Components
MongoDB divides operations into distinct logical subsystems. For instance, network issues live under the “NETWORK” component, command parsing under “COMMAND”, and disk writes under “STORAGE”. We can query specific areas of concern:
|
jq ‘select(.c == “REPL”)’ mongod.log |
| Tip:
Instead of displaying the entire JSON object, we can project only the fields we care about. For example, to read a timeline of replica set events, we can extract just the timestamp, context, and message: jq ‘select(.c == “REPL”) | [.t.$date, .ctx, .msg]’ mongod.log |
- Detecting and Tracking Slow Queries
One of the most valuable aspects of MongoDB logs is database performance diagnostics. MongoDB logs any query that takes longer than the configured threshold (default: 100ms) with the message “Slow query”. Inside the attr document, MongoDB dumps parameters like query command details, plan summary (e.g. COLLSCAN or IXSCAN), keys examined, and durationMillis.
|
jq ‘select(.attr.durationMillis > 500)’ mongod.log |
To extract only the query namespace (database and collection) and execution time for quick performance reports, use a mapping pipeline:
|
jq ‘select(.msg == “Slow query”) | {ns: .attr.ns, ms: .attr.durationMillis}’ mongod.log |
- Aggregating & Frequency Counting
If your database is throwing errors or generating heavy logs, you can compile occurrences of specific message types to pinpoint repeating problems. We do this by feeding jq’s raw outputs (using the -r flag to output strings without surrounding double-quotes) into native shell aggregation utilities: sort and uniq.
|
jq -r ‘.msg’ mongod.log | sort | uniq -c | sort -rn |
This prints a list showing count and message text, sorted with the most frequent occurrences at the top. It is the quickest way to diagnose repeating issues (e.g. repeated connection authentications or authentication failures).
Log Querying Cheat Sheet Reference
| Objective | jq Filter Expression | Full Terminal Command |
| Pretty Print Logs | . | jq ‘.’ mongod.log |
| Errors Only | select(.s == “E”) | jq ‘select(.s == “E”)’ mongod.log |
| Warnings & Errors | select(.s == “E” or .s == “W”) | jq ‘select(.s == “E” or .s == “W”)’ mongod.log |
| Slow Queries | select(.msg == “Slow query”) | jq ‘select(.msg == “Slow query”)’ mongod.log |
| Query Thresholds | select(.attr.durationMillis > 200) | jq ‘select(.attr.durationMillis > 200)’ mongod.log |
| Extract Target Fields | [.t.$date, .s, .msg] | jq ‘[.t.$date, .s, .msg]’ mongod.log |
| Aggregate Messages | .msg | jq -r ‘.msg’ mongod.log | sort | uniq -c | sort -rn |
