Nginx Access Logs: What Every Field Means and How to Analyze Them

Nginx access logs record every request your web server handles. They are the first place to look when debugging 404s, tracking down slow endpoints, or investigating suspicious traffic. Yet many developers skim the raw log lines without fully understanding each field. This guide breaks down the default Nginx log format, shows how to customize it, and walks through practical analysis techniques using standard command-line tools.

Default Nginx Access Log Format

Out of the box, Nginx uses the combined log format, which looks like this:

log_format combined '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

A typical log line:

192.0.2.1 - - [10/Oct/2023:13:55:36 +0000] "GET /api/users HTTP/1.1" 200 1234 "https://example.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"

Field-by-Field Explanation

Customizing the Log Format

You can define your own format to capture additional details like request time, upstream response time, or gzip ratio. Add a log_format directive in the http block and reference it in access_log.

http {
    log_format detailed '$remote_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent" '
                        'rt=$request_time uct="$upstream_connect_time" '
                        'uht="$upstream_header_time" urt="$upstream_response_time"';

    access_log /var/log/nginx/access.log detailed;
}

Common extra variables:

• $request_time – total time from first byte received to last byte sent.
• $upstream_response_time – time spent waiting for upstream (e.g., PHP-FPM, Node.js).
• $gzip_ratio – compression ratio if gzip is enabled.
• $http_x_forwarded_for – original client IP when behind a proxy or load balancer.

Analyzing Nginx Logs with Command-Line Tools

You don't need expensive software to extract insights. Standard Unix tools like grep, awk, sort, and uniq are powerful enough for most tasks.

1. Find the Most Frequent IP Addresses

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -10

This helps identify heavy hitters or potential DDoS sources.

2. Identify Top Requested URLs

awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -10

Useful for understanding popular content or spotting unusual patterns.

3. Count HTTP Status Codes

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -nr

Quickly see how many 404s or 500s are occurring.

4. Find Requests with Slow Response Times

If you added $request_time to your log format, you can filter for slow requests:

awk '{ if ($NF > 1) print }' /var/log/nginx/access.log

Adjust the threshold (1 second here) as needed.

5. Detect Suspicious User Agents

awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -10

Look for known bad bots or unusual clients.

Practical Use Cases

• Troubleshooting: Correlate 500 errors with specific endpoints or upstream response times.
• Performance tuning: Identify slow pages and optimize database queries or caching.
• Security monitoring: Spot brute-force attempts (many 401/403 from same IP) or SQL injection patterns in URLs.
• Capacity planning: Track request volume over time to anticipate scaling needs.

Automating Analysis with Log Parsers

While command-line one-liners work for ad-hoc queries, regular reporting benefits from dedicated tools. You can use open-source log analyzers like GoAccess or AWStats, or build custom scripts. For quick, browser-based parsing without installing anything, the Nginx Log Analyzer lets you paste log lines and get instant summaries of status codes, top IPs, and requested paths.

Best Practices for Log Management

• Rotate logs regularly: Use logrotate to prevent disk space issues.
• Centralize logs: Ship to a log management system (e.g., ELK, Graylog) for long-term storage and search.
• Include request IDs: Add $request_id to correlate with application logs.
• Respect privacy: Anonymize IPs if required by GDPR or other regulations.

FAQ

What is the difference between access logs and error logs in Nginx?

Access logs record every request handled by Nginx, including client IP, request method, status code, and user agent. Error logs capture diagnostic messages about server issues, such as failed connections, configuration errors, or upstream timeouts. Both are essential for different troubleshooting scenarios.

How can I see the real client IP when Nginx is behind a load balancer?

When Nginx sits behind a proxy or load balancer, $remote_addr shows the balancer's IP. To log the original client IP, configure the real_ip module to trust the balancer's IP and use $http_x_forwarded_for or $realip_remote_addr in your log format.

Can I analyze Nginx logs without installing software?

Yes. You can use standard Unix commands like grep, awk, and sort directly in the terminal. For a graphical, browser-based approach, online tools like the Nginx Log Analyzer provide quick insights without any installation.

Ready to dive into your logs? Try the Nginx Log Analyzer to parse and visualize your access logs in seconds—no setup required.