Welcome! Log In Create A New Profile

Advanced

location blocks, and if conditions in server context

Posted by Lucas Rolff 
Lucas Rolff
location blocks, and if conditions in server context
March 07, 2018 06:00PM
Hi guys,

I have a few hundred nginx zones, where I try to remove as much duplicate code as possible, and inherit as much as possible to prevent nginx from consuming memory (and also to keep things clean).

However I came across something today, that I don’t know how to get my head around without duplicating code, even within a single server context.

I have a set of distributed nginx servers, all these requires SSL certificates, where I use Let’s Encrypt to do this.
When doing the Let’s Encrypt validation, it uses a path such as /.well-known/acme-challenge/<hash>

For this, I made a location block such as:

location ~* /.well-known {
proxy_pass http://letsencrypt.validation.backend.com$request_uri;
}

Basically, I proxy_pass to the backend where I actually run the acme client – works great.

However, I have an option to force a redirect from http to https, and I’ve implemented that by doing an if condition on the server block level (so not within a location):

if ($sslproxy_protocol = "http") {
return 301 https://$host$request_uri;
}

This means I have something like:

1: location ~* /.well-known
2: if condition doing redirect if protocol is http
3: location /
4: location /api
5: location /test

All my templates include 1 to 3, and *might* have additional locations.
I’ve decided to not put e.g. location /api inside the location / - because there’s things I don’t want to inherit, thus keeping them at the same “level”, and not a location context inside a location context.
Things I don’t want to inherit, is stuff such as headers, max_ranges directive etc.

My issue is – because of this if condition that does the redirect to https – it also applies to my location ~* /.well-known – thus causing a redirect, and I want to prevent this, since it breaks the Let’s Encrypt validation (they do not accept 301 redirects).

A solution would be to move the if condition into each location block that I want to have redirected, but then I start repeating myself 1, 2 or even 10 times – which I don’t wanna do.

Is there a smart way without adding too much complexity, which is still super-fast (I know if is evil) ?

A config example is seen below:

server {
listen 80;
listen 443 ssl http2;

server_name secure.domain.com;

access_log /var/log/nginx/secure.domain.com main;

location ~* /.well-known {
proxy_pass http://letsencrypt.validation.backend.com$request_uri;
}

if ($sslproxy_protocol = "http") {
return 301 https://$host$request_uri;
}

location / {

expires 10m;
etag off;

proxy_ignore_client_abort on;
proxy_intercept_errors on;
proxy_next_upstream error timeout invalid_header;
proxy_ignore_headers Set-Cookie Vary X-Accel-Expires Expires Cache-Control;
more_clear_headers Set-Cookie Cookie Upgrade;

proxy_cache one;
proxy_cache_min_uses 1;
proxy_cache_lock off;
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;

proxy_cache_valid 200 10m;
proxy_cache_valid any 1m;

proxy_cache_revalidate on;
proxy_ssl_server_name on;

include /etc/nginx/server.conf;

proxy_set_header Host backend-host.com;

proxy_cache_key "http://backend-host.com-1-$request_uri";;
proxy_pass http://backend-host.com$request_uri;

proxy_redirect off;
}
}

Thank you in advance!

Best Regards,
Lucas Rolff
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
I agree that avoiding if is a good thing. But avoiding duplication isn’t always good.

Have you considered a model where your configuration file is generated with a templating engine? The input file that you modify to add/remove/change configurations could be free of duplication but the conf file that nginx reads could be concrete and verbose

Sent from my iPhone

> On Mar 7, 2018, at 11:55, Lucas Rolff <[email protected]> wrote:
>
> Hi guys,
>
> I have a few hundred nginx zones, where I try to remove as much duplicate code as possible, and inherit as much as possible to prevent nginx from consuming memory (and also to keep things clean).
>
> However I came across something today, that I don’t know how to get my head around without duplicating code, even within a single server context.
>
> I have a set of distributed nginx servers, all these requires SSL certificates, where I use Let’s Encrypt to do this.
> When doing the Let’s Encrypt validation, it uses a path such as /.well-known/acme-challenge/<hash>
>
> For this, I made a location block such as:
>
> location ~* /.well-known {
> proxy_pass http://letsencrypt.validation.backend.com$request_uri;
> }
>
> Basically, I proxy_pass to the backend where I actually run the acme client – works great.
>
> However, I have an option to force a redirect from http to https, and I’ve implemented that by doing an if condition on the server block level (so not within a location):
>
> if ($sslproxy_protocol = "http") {
> return 301 https://$host$request_uri;
> }
>
> This means I have something like:
>
> 1: location ~* /.well-known
> 2: if condition doing redirect if protocol is http
> 3: location /
> 4: location /api
> 5: location /test
>
> All my templates include 1 to 3, and *might* have additional locations.
> I’ve decided to not put e.g. location /api inside the location / - because there’s things I don’t want to inherit, thus keeping them at the same “level”, and not a location context inside a location context.
> Things I don’t want to inherit, is stuff such as headers, max_ranges directive etc.
>
> My issue is – because of this if condition that does the redirect to https – it also applies to my location ~* /.well-known – thus causing a redirect, and I want to prevent this, since it breaks the Let’s Encrypt validation (they do not accept 301 redirects).
>
> A solution would be to move the if condition into each location block that I want to have redirected, but then I start repeating myself 1, 2 or even 10 times – which I don’t wanna do.
>
> Is there a smart way without adding too much complexity, which is still super-fast (I know if is evil) ?
>
> A config example is seen below:
>
> server {
> listen 80;
> listen 443 ssl http2;
>
> server_name secure.domain.com;
>
> access_log /var/log/nginx/secure.domain.com main;
>
> location ~* /.well-known {
> proxy_pass http://letsencrypt.validation.backend.com$request_uri;
> }
>
> if ($sslproxy_protocol = "http") {
> return 301 https://$host$request_uri;
> }
>
> location / {
>
> expires 10m;
> etag off;
>
> proxy_ignore_client_abort on;
> proxy_intercept_errors on;
> proxy_next_upstream error timeout invalid_header;
> proxy_ignore_headers Set-Cookie Vary X-Accel-Expires Expires Cache-Control;
> more_clear_headers Set-Cookie Cookie Upgrade;
>
> proxy_cache one;
> proxy_cache_min_uses 1;
> proxy_cache_lock off;
> proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
>
> proxy_cache_valid 200 10m;
> proxy_cache_valid any 1m;
>
> proxy_cache_revalidate on;
> proxy_ssl_server_name on;
>
> include /etc/nginx/server.conf;
>
> proxy_set_header Host backend-host.com;
>
> proxy_cache_key "http://backend-host.com-1-$request_uri";;
> proxy_pass http://backend-host.com$request_uri;
>
> proxy_redirect off;
> }
> }
>
> Thank you in advance!
>
> Best Regards,
> Lucas Rolff
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Hi peter,

I generate configs already using a template engine (more specific Laravel Blade), so creating the functionality in the template is easy, however, I generally don’t like having server blocks that can be 100s of lines because of repeating things

I don’t know the internals of nginx fully, how it uses memory when storing configs, but I would assume that inheritance is better than duplication in terms of memory usage.

I’m just wondering if there’s a way I can avoid the if condition within the location blocks.

- lucas

Get Outlook for iOShttps://aka.ms/o0ukef
________________________________
From: nginx <[email protected]> on behalf of Peter Booth <[email protected]>
Sent: Wednesday, March 7, 2018 11:08:40 PM
To: nginx@nginx.org
Subject: Re: location blocks, and if conditions in server context

I agree that avoiding if is a good thing. But avoiding duplication isn’t always good.

Have you considered a model where your configuration file is generated with a templating engine? The input file that you modify to add/remove/change configurations could be free of duplication but the conf file that nginx reads could be concrete and verbose

Sent from my iPhone

On Mar 7, 2018, at 11:55, Lucas Rolff <[email protected]<mailto:[email protected]>> wrote:

Hi guys,

I have a few hundred nginx zones, where I try to remove as much duplicate code as possible, and inherit as much as possible to prevent nginx from consuming memory (and also to keep things clean).

However I came across something today, that I don’t know how to get my head around without duplicating code, even within a single server context.

I have a set of distributed nginx servers, all these requires SSL certificates, where I use Let’s Encrypt to do this.
When doing the Let’s Encrypt validation, it uses a path such as /.well-known/acme-challenge/<hash>

For this, I made a location block such as:

location ~* /.well-known {
proxy_pass http://letsencrypt.validation.backend.com$request_uri;
}

Basically, I proxy_pass to the backend where I actually run the acme client – works great.

However, I have an option to force a redirect from http to https, and I’ve implemented that by doing an if condition on the server block level (so not within a location):

if ($sslproxy_protocol = "http") {
return 301 https://$host$request_uri;
}

This means I have something like:

1: location ~* /.well-known
2: if condition doing redirect if protocol is http
3: location /
4: location /api
5: location /test

All my templates include 1 to 3, and *might* have additional locations.
I’ve decided to not put e.g. location /api inside the location / - because there’s things I don’t want to inherit, thus keeping them at the same “level”, and not a location context inside a location context.
Things I don’t want to inherit, is stuff such as headers, max_ranges directive etc.

My issue is – because of this if condition that does the redirect to https – it also applies to my location ~* /.well-known – thus causing a redirect, and I want to prevent this, since it breaks the Let’s Encrypt validation (they do not accept 301 redirects).

A solution would be to move the if condition into each location block that I want to have redirected, but then I start repeating myself 1, 2 or even 10 times – which I don’t wanna do.

Is there a smart way without adding too much complexity, which is still super-fast (I know if is evil) ?

A config example is seen below:

server {
listen 80;
listen 443 ssl http2;

server_name secure.domain.comhttp://secure.domain.com;

access_log /var/log/nginx/secure.domain.comhttp://secure.domain.com main;

location ~* /.well-known {
proxy_pass http://letsencrypt.validation.backend.com$request_uri;
}

if ($sslproxy_protocol = "http") {
return 301 https://$host$request_uri;
}

location / {

expires 10m;
etag off;

proxy_ignore_client_abort on;
proxy_intercept_errors on;
proxy_next_upstream error timeout invalid_header;
proxy_ignore_headers Set-Cookie Vary X-Accel-Expires Expires Cache-Control;
more_clear_headers Set-Cookie Cookie Upgrade;

proxy_cache one;
proxy_cache_min_uses 1;
proxy_cache_lock off;
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;

proxy_cache_valid 200 10m;
proxy_cache_valid any 1m;

proxy_cache_revalidate on;
proxy_ssl_server_name on;

include /etc/nginx/server.conf;

proxy_set_header Host backend-host.comhttp://backend-host.com;

proxy_cache_key "http://backend-host.com-1-$request_uri";;
proxy_pass http://backend-host.com$request_uri;

proxy_redirect off;
}
}

Thank you in advance!

Best Regards,
Lucas Rolff
_______________________________________________
nginx mailing list
[email protected]<mailto:[email protected]>
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Francis Daly
Re: location blocks, and if conditions in server context
March 08, 2018 09:50AM
On Wed, Mar 07, 2018 at 04:55:15PM +0000, Lucas Rolff wrote:

Hi there,

> This means I have something like:
>
> 1: location ~* /.well-known
> 2: if condition doing redirect if protocol is http
> 3: location /
> 4: location /api
> 5: location /test
>
> All my templates include 1 to 3, and *might* have additional locations.

> My issue is – because of this if condition that does the redirect to https – it also applies to my location ~* /.well-known – thus causing a redirect, and I want to prevent this, since it breaks the Let’s Encrypt validation (they do not accept 301 redirects).

> Is there a smart way without adding too much complexity, which is still super-fast (I know if is evil) ?

As phrased, I think the short answer to your question is "no".

However...

You optionally redirect things from http to https. Is that "you want
to redirect *everything* from http to https, apart from the letsencrypt
thing"? If so, you could potentially have just one

server {
listen 80;
location / { return 301 https://$host$uri; }
location /.well-known/ { proxy_pass http://letsencrypt.validation.backend.com; }
}

and a bunch of

server {
listen 443;
}

blocks.

Or: you use $sslproxy_protocol. Where does that come from?

If it is a thing that you create to decide whether or not to redirect
to https, then could you include a check for whether the request starts
with /.well-known/, and if so set it to something other than "http"?

f
--
Francis Daly francis@daoine.org
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Hi Francis,

I indeed thought about having a separate server {} block in case there’s the http to https redirect for a specific domain.
Since it depends on the domain, I can’t make a general one to match everything.

> Or: you use $sslproxy_protocol. Where does that come from?

$sslproxy_protocol is a simple map doing:

map $https $sslproxy _protocol {
default "http";
SSL "https";
on "https";
}

Best Regards,
Lucas Rolff

On 08/03/2018, 09.44, "nginx on behalf of Francis Daly" <[email protected] on behalf of [email protected]> wrote:

On Wed, Mar 07, 2018 at 04:55:15PM +0000, Lucas Rolff wrote:

Hi there,

> This means I have something like:
>
> 1: location ~* /.well-known
> 2: if condition doing redirect if protocol is http
> 3: location /
> 4: location /api
> 5: location /test
>
> All my templates include 1 to 3, and *might* have additional locations.

> My issue is – because of this if condition that does the redirect to https – it also applies to my location ~* /.well-known – thus causing a redirect, and I want to prevent this, since it breaks the Let’s Encrypt validation (they do not accept 301 redirects).

> Is there a smart way without adding too much complexity, which is still super-fast (I know if is evil) ?

As phrased, I think the short answer to your question is "no".

However...

You optionally redirect things from http to https. Is that "you want
to redirect *everything* from http to https, apart from the letsencrypt
thing"? If so, you could potentially have just one

server {
listen 80;
location / { return 301 https://$host$uri; }
location /.well-known/ { proxy_pass http://letsencrypt.validation.backend.com; }
}

and a bunch of

server {
listen 443;
}

blocks.

Or: you use $sslproxy_protocol. Where does that come from?

If it is a thing that you create to decide whether or not to redirect
to https, then could you include a check for whether the request starts
with /.well-known/, and if so set it to something other than "http"?

f
--
Francis Daly francis@daoine.org
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Francis Daly
Re: location blocks, and if conditions in server context
March 09, 2018 12:40AM
On Thu, Mar 08, 2018 at 08:57:29AM +0000, Lucas Rolff wrote:

Hi there,

> I indeed thought about having a separate server {} block in case there’s the http to https redirect for a specific domain.
> Since it depends on the domain, I can’t make a general one to match everything.

So, if I read this correctly, the new "requirement statement" is: some
domains want to redirect everything (apart from the letsencrypt piece)
from http to https; and some domains do not want to redirect anything
from http to https.

In that case, the one server with "listen 80 default;" and the two
locations, one with "return 301" and the other with "proxy_pass"; plus
the multiple servers with "listen 443" should Just Work.

If you do want the to-https redirect for this domain, do not add "listen
80" in the 443 server. If you do not want the to-https redirect for that
domain, do add "listen 80" in the 443 server.

Am I missing something?

> > Or: you use $sslproxy_protocol. Where does that come from?
>
> $sslproxy_protocol is a simple map doing:
>
> map $https $sslproxy _protocol {
> default "http";
> SSL "https";
> on "https";
> }

Because I don't know what else you use that variable for, perhaps you
could make a new variable $redirect_to_https, like so (untested):

map $https$uri $redirect_to_https {
default "yes";
~^SSL "no";
~^on "no";
~^/.well-known/ "no";
}

and then redirect based on the value of that variable, where it might matter.

(I presume that $https is empty in http-mode, per http://nginx.org/r/$https)

I prefer the first solution, without the extra variable-and-if; but it's
not my server.

Good luck with it,

f
--
Francis Daly francis@daoine.org
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login