6 Nginx configuration pitfalls that bit me in the ass

Non-obvious Nginx configuration mistakes that take tons of time to debug when things go south

nginx add_header not working (part 1)

Having separate add_header directives for different HTTP headers in server and location blocks causes the headers defined in the server block to be dropped from the request response.

There could be several add_header directives. These directives are inherited from the previous level if and only if there are no add_header directives defined on the current level. (http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header)

Make sure you copy your add_header directives to your location blocks.

nginx add_header not working (part 2)

By default, HTTP headers with underscores are dropped by Nginx. To fix this, set: underscores_in_headers off;

server level return directives take precedence over location blocks

server {
    root /my/path;
    location /file.txt {
        try_files $uri =404;    
    }

    # ...1000 lines of Nginx config below...

    return 301 https://www.example.com$request_uri;
}

If you request /file.txt you’ll get an HTTP 301. Instead, you should do:

server {
    root /my/path;
    location /file.txt {
        try_files $uri =404;
    }
    location / {
        return 301 https://www.example.com$request_uri;
    }
}

Don’t use hostnames in your Nginx upstream blocks

Hostnames in your upstream block are resolved only once on Nginx start/reload, they are not dynamically resolved.

• If the hostnames can’t be resolved, Nginx will fail to start/reload.

• If their IPs change, Nginx won’t notice the change until you reload.

For these reasons, always use IPs instead of hostnames in your upstream blocks.

Failed POSTs are retried on old versions of Nginx

If you use an old Nginx (<1.9.13), please upgrade it ASAP. Below this Nginx version, non-idempotent HTTP requests such as POSTs are retried when they fail. This is dangerous as it could lead to sensitive requests such as payment requests being sent more than once.

Unwanted request rebalancing

The ip_hash algorithm rebalances your requests if you remove servers or reorder servers in the upstream server list and reload your Nginx config.

If one of the servers needs to be temporarily removed, it should be marked with the down parameter in order to preserve the current hashing of client IP addresses. (http://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash)

(If you liked this, you might like A few tips about Nginx caching)