Learn how to build lightweight, production-ready Bash and PowerShell scripts using jq to stream database logs and monitor server metrics in real-time.
In our previous blog, we discussed using jq to parse individual logs and query database metrics manually. While this is great for troubleshooting past issues, production databases require active monitoring. We can build scripts that leverage jq to stream logs and server metrics dynamically, giving database administrators a lightweight, command-line monitor without bloated dashboard software.
We will create two types of automation monitoring scripts in both Bash (for Linux) and PowerShell (for Windows).
- Live Log & Performance Monitor
MongoDB logs events continuously. By tailing the log file and using jq with the –unbuffered flag, we can isolate slow queries (duration > 100ms), Warnings (W), and Errors (E), printing clean, color-coded status messages immediately as they occur.
Unix Bash Script (monitor_logs.sh)
| #!/bin/bash LOG_FILE=${1:-“/var/log/mongodb/mongod.log”}echo “Monitoring MongoDB Logs: $LOG_FILE” tail -f “$LOG_FILE” | jq –unbuffered -r ‘ select(.msg == “Slow query” or .s == “E” or .s == “W”) | if .msg == “Slow query” then “⚡ SLOW QUERY [\((.t.”$date” // .t)[:19])] ctx=\(.ctx) ns=\(.attr.ns) duration=\(.attr.durationMillis)ms msg=\(.attr.planSummary // “no-plan”)” elif .s == “E” then “🚨 ERROR [\((.t.”$date” // .t)[:19])] ctx=\(.ctx) comp=\(.c) id=\(.id) msg=\”\(.msg)\” attr=\(.attr | tostring)” elif .s == “W” then “⚠️ WARNING [\((.t.”$date” // .t)[:19])] ctx=\(.ctx) comp=\(.c) id=\(.id) msg=\”\(.msg)\” attr=\(.attr | tostring)” else empty end ‘ |
Windows PowerShell Script (monitor_logs.ps1)
| param ( [string]$LogPath = “C:\data\log\mongod.log” )Write-Host “Monitoring MongoDB Logs: $LogPath” Get-Content -Path $LogPath -Tail 0 -Wait | jq –unbuffered -r ‘ select(.msg == “Slow query” or .s == “E” or .s == “W”) | if .msg == “Slow query” then “⚡ SLOW QUERY [\((.t.”$date” // .t)[:19])] ctx=\(.ctx) ns=\(.attr.ns) duration=\(.attr.durationMillis)ms msg=\(.attr.planSummary // “no-plan”)” elif .s == “E” then “🚨 ERROR [\((.t.”$date” // .t)[:19])] ctx=\(.ctx) comp=\(.c) id=\(.id) msg=\”\(.msg)\” attr=\(.attr | tostring)” elif .s == “W” then “⚠️ WARNING [\((.t.”$date” // .t)[:19])] ctx=\(.ctx) comp=\(.c) id=\(.id) msg=\”\(.msg)\” attr=\(.attr | tostring)” else empty end ‘ |
- Live Server Status Metrics Dashboard
Monitoring log files handles event-based diagnostic issues. But how do we track memory usage, cache exhaustion, active client connections, and networking traffic over time? MongoDB stores all of this metadata in the db.serverStatus() administrative document. By querying this periodically in a loop, we can format a clean live statistics table using jq.
PowerShell Metrics Monitor (monitor_metrics.ps1)
| param ( [string]$ConnectionUri = “mongodb://localhost:27017”, [int]$Interval = 5 )while ($true) { $statusJson = mongosh $ConnectionUri –quiet –eval “printjson(db.serverStatus())” 2>$null $statusJson | jq -r ‘ if .connections then “📊 MongoDB Live Statistics – ” + (.localTime // “N/A”) + “\n” + “———————————————————-\n” + “👥 Connections: \(.connections.current) active / \(.connections.available) available (Total: \(.connections.totalCreated // 0))\n” + “💾 Memory: \(.mem.resident // 0) MB (Resident) / \(.mem.virtual // 0) MB (Virtual)\n” + “📈 Operations: Inserts: \(.opcounters.insert // 0) | Queries: \(.opcounters.query // 0) | Updates: \(.opcounters.update // 0) | Deletes: \(.opcounters.delete // 0)\n” + “🌐 Network: In: \(((.network.bytesIn // 0) / 1024 / 1024 * 10 | round) / 10) MB | Out: \(((.network.bytesOut // 0) / 1024 / 1024 * 10 | round) / 10) MB\n” + (if .wiredTiger then “🗄️ WiredTiger Cache Pressure:\n” + ” • Configured Max: \(((.wiredTiger.cache[“maximum bytes configured”] // 0) / 1024 / 1024 / 1024 * 10 | round) / 10) GB\n” + ” • Bytes in Cache: \(((.wiredTiger.cache[“bytes currently in the cache”] // 0) / 1024 / 1024 / 1024 * 10 | round) / 10) GB\n” + ” • Cache Fill %: \((((.wiredTiger.cache[“bytes currently in the cache”] // 0) / (.wiredTiger.cache[“maximum bytes configured”] // 1) * 100) * 10 | round) / 10)%\n” else “🗄️ WiredTiger Cache metrics unavailable\n” end) + “==========================================================\n” else “Error querying server status.” end ‘ Start-Sleep -Seconds $Interval } |
| ℹ️ Understanding WiredTiger Cache Pressure
WiredTiger cache statistics are vital. If the ‘Cache Fill %’ exceeds 80%, MongoDB is writing to disk heavily, which can slow down client operations. Keeping an eye on this in your terminal dashboard helps proactively warn you about memory configuration limitations. |
Conclusion
By coupling MongoDB’s administrative CLI commands and tailing properties with the power of jq, you can construct lightweight performance tooling that runs natively in your terminal. This is an efficient alternative to heavyweight tracking packages, especially during direct ssh debugging session.
