Welcome! Log In Create A New Profile

Advanced

weight and balancing in upstream proxy

Posted by Frank Liu 
Frank Liu
weight and balancing in upstream proxy
April 13, 2017 12:00AM
Hi,

How does nginx balances traffic to upstream with different weight? If I
have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy,
will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3,
3? If I have two servers with both weight 50, will nginx will 50 requests
to server 1, and then 50 to server 2, or will it calculate the ration to be
1:1 and send one after another?

Thanks!
Frank
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
B.R. via nginx
Re: weight and balancing in upstream proxy
April 13, 2017 10:20AM
That is an interesting questions as intuitively, people could think the
former behavior applies.

If I got the source code
<https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_http_upstream_round_robin.c#L507>;
right, and as the docs
<https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>;
state, nginx is following a weighted round-robin
https://en.wikipedia.org/wiki/Weighted_round_robin algorithm.
It thus means it will go over the same list of servers everytime a peer
needs to be chosen (ie for every request), and pick the first not having
depleted its weight allocation.

To me, it would use the latter of your proposals.
​Please correct me if I am wrong, so incorrect information does not
propagate too much. :o)​
---
*B. R.*

On Wed, Apr 12, 2017 at 11:50 PM, Frank Liu <[email protected]> wrote:

> Hi,
>
> How does nginx balances traffic to upstream with different weight? If I
> have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy,
> will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3,
> 3? If I have two servers with both weight 50, will nginx will 50 requests
> to server 1, and then 50 to server 2, or will it calculate the ration to be
> 1:1 and send one after another?
>
> Thanks!
> Frank
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Maxim Dounin
Re: weight and balancing in upstream proxy
April 13, 2017 04:40PM
Hello!

On Thu, Apr 13, 2017 at 10:09:16AM +0200, B.R. via nginx wrote:

> That is an interesting questions as intuitively, people could think the
> former behavior applies.
>
> If I got the source code
> <https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_http_upstream_round_robin.c#L507>;
> right, and as the docs
> <https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>;
> state, nginx is following a weighted round-robin
> https://en.wikipedia.org/wiki/Weighted_round_robin algorithm.
> It thus means it will go over the same list of servers everytime a peer
> needs to be chosen (ie for every request), and pick the first not having
> depleted its weight allocation.
>
> To me, it would use the latter of your proposals.
> ​Please correct me if I am wrong, so incorrect information does not
> propagate too much. :o)​

The Wikipedia link in question doesn't seem to be related to what
nginx does.

--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Frank Liu
Re: weight and balancing in upstream proxy
April 14, 2017 01:10AM
Hi Maxim,
Thanks for pointing out the link is not related. Do you have the answer to the original question or a related link?
Thanks
Frank

> On Apr 13, 2017, at 7:34 AM, Maxim Dounin <[email protected]> wrote:
>
> Hello!
>
>> On Thu, Apr 13, 2017 at 10:09:16AM +0200, B.R. via nginx wrote:
>>
>> That is an interesting questions as intuitively, people could think the
>> former behavior applies.
>>
>> If I got the source code
>> <https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_http_upstream_round_robin.c#L507>;
>> right, and as the docs
>> <https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>;
>> state, nginx is following a weighted round-robin
>> https://en.wikipedia.org/wiki/Weighted_round_robin algorithm.
>> It thus means it will go over the same list of servers everytime a peer
>> needs to be chosen (ie for every request), and pick the first not having
>> depleted its weight allocation.
>>
>> To me, it would use the latter of your proposals.
>> Please correct me if I am wrong, so incorrect information does not
>> propagate too much. :o)
>
> The Wikipedia link in question doesn't seem to be related to what
> nginx does.
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Francis Daly
Re: weight and balancing in upstream proxy
April 14, 2017 01:50AM
On Wed, Apr 12, 2017 at 02:50:08PM -0700, Frank Liu wrote:

Hi there,

> How does nginx balances traffic to upstream with different weight? If I
> have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy,
> will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3,
> 3?

If you want to know what your current nginx version does, it should not
be too difficult to test:

One nginx.conf. One http section. One upstream{} listing multiple
ip:ports. One server{} which proxy_pass:es to that upstream. Multiple
server{}s, each of which listens on one ip:port, writes to a different
access_log, and does something like "return 200 ok;".

Then for increasing numbers X, "GET /X" on the main server. Look at
the individual access log files to see which server handled /1, which
handled /2, etc. By the time you get to 700, you'll either see that
there is a reliably repeating pattern, or you'll see that it is probably
randomish.


If you actually care about what pattern is used; or if you want to
guarantee that the same pattern will be used in future nginx versions;
then get your preferred code written and use that instead of whatever
nginx uses.


If you want to know what guarantee there is that the behaviour will not
change in the future: I'd say "none", except that there is a good chance
that what is written in the documentation will be honoured. Paraphrasing
that, for the above case: for 7 requests, 1 will go to the first server;
2 to the second; and 4 to the third.

> If I have two servers with both weight 50, will nginx will 50 requests
> to server 1, and then 50 to server 2, or will it calculate the ration to be
> 1:1 and send one after another?

Same answer: it does not seem to difficult to test, if you don't want
to read the available source.

Good luck with it!

f
--
Francis Daly francis@daoine.org
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Frank Liu
Re: weight and balancing in upstream proxy
April 14, 2017 07:10AM
Hi Francis,

Thanks for confirming that there is no document, and any results observed
through testing or reviewing code will not be guaranteed. I guess it is
purposely undocumented so that people won't rely on one behavior and we are
free to change.

Regards,
Frank


On Thu, Apr 13, 2017 at 4:49 PM, Francis Daly <[email protected]> wrote:

> On Wed, Apr 12, 2017 at 02:50:08PM -0700, Frank Liu wrote:
>
> Hi there,
>
> > How does nginx balances traffic to upstream with different weight? If I
> > have 3 servers in upstream, with weight 1, 2, 4, assuming all are
> healthy,
> > will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3,
> 3,
> > 3?
>
> If you want to know what your current nginx version does, it should not
> be too difficult to test:
>
> One nginx.conf. One http section. One upstream{} listing multiple
> ip:ports. One server{} which proxy_pass:es to that upstream. Multiple
> server{}s, each of which listens on one ip:port, writes to a different
> access_log, and does something like "return 200 ok;".
>
> Then for increasing numbers X, "GET /X" on the main server. Look at
> the individual access log files to see which server handled /1, which
> handled /2, etc. By the time you get to 700, you'll either see that
> there is a reliably repeating pattern, or you'll see that it is probably
> randomish.
>
>
> If you actually care about what pattern is used; or if you want to
> guarantee that the same pattern will be used in future nginx versions;
> then get your preferred code written and use that instead of whatever
> nginx uses.
>
>
> If you want to know what guarantee there is that the behaviour will not
> change in the future: I'd say "none", except that there is a good chance
> that what is written in the documentation will be honoured. Paraphrasing
> that, for the above case: for 7 requests, 1 will go to the first server;
> 2 to the second; and 4 to the third.
>
> > If I have two servers with both weight 50, will nginx will 50 requests
> > to server 1, and then 50 to server 2, or will it calculate the ration to
> be
> > 1:1 and send one after another?
>
> Same answer: it does not seem to difficult to test, if you don't want
> to read the available source.
>
> Good luck with it!
>
> f
> --
> Francis Daly francis@daoine.org
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
B.R. via nginx
Re: weight and balancing in upstream proxy
April 14, 2017 09:20AM
Please, enlighten us then.
---
*B. R.*

On Thu, Apr 13, 2017 at 4:34 PM, Maxim Dounin <[email protected]> wrote:

> Hello!
>
> On Thu, Apr 13, 2017 at 10:09:16AM +0200, B.R. via nginx wrote:
>
> > That is an interesting questions as intuitively, people could think the
> > former behavior applies.
> >
> > If I got the source code
> > <https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_
> http_upstream_round_robin.c#L507>
> > right, and as the docs
> > <https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>;
> > state, nginx is following a weighted round-robin
> > https://en.wikipedia.org/wiki/Weighted_round_robin algorithm.
> > It thus means it will go over the same list of servers everytime a peer
> > needs to be chosen (ie for every request), and pick the first not having
> > depleted its weight allocation.
> >
> > To me, it would use the latter of your proposals.
> > ​Please correct me if I am wrong, so incorrect information does not
> > propagate too much. :o)​
>
> The Wikipedia link in question doesn't seem to be related to what
> nginx does.
>
> --
> Maxim Dounin
> http://nginx.org/
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Francis Daly
Re: weight and balancing in upstream proxy
April 14, 2017 09:40AM
On Thu, Apr 13, 2017 at 10:08:21PM -0700, Frank Liu wrote:

Hi there,

> Thanks for confirming that there is no document, and any results observed
> through testing or reviewing code will not be guaranteed. I guess it is
> purposely undocumented so that people won't rely on one behavior and we are
> free to change.

The guarantee is written in the licence, available at
http://nginx.org/LICENSE.

Something similar is true for every aspect of every piece of behaviour
of every piece of software.

If there is specific behaviour you want, you always have the option
of finding someone who will commit to providing and preserving that
behaviour in future versions of the nginx that you use.

Or, if you know through testing and code review that the behaviour of
1.10.0 is exactly what you want, you always have the option of continuing
to use that version. (Compiled with your version of a compiler and
support libraries, and running with your version of an operating system
and runtime libraries.)


I suspect that the upstream round-robin implementation will not change in
the future without good reason, just like I expect that HTTP/1.0 support
will not change without good reason. But if things do change, and if
my comfort depends on the old behaviour persisting, I have the freedom
to ensure that the old behaviour does persist on my systems.

So I'd say that if it matters, you can see what the current code does,
and base your system on that.

Note that if your system relies on all upstreams remaining always
available, your system is probably fragile -- if you need a specific
pattern of access, and one of your upstreams is unavailable when nginx
wants it, the current nginx "recovery" behaviour may or may not be what
you want. And *that* behaviour (I think) depends on timings of incoming
requests, so may not be fully under your control anyway.

Good luck with it,

f
--
Francis Daly francis@daoine.org
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Aleksandar Lazic
Re: weight and balancing in upstream proxy
April 14, 2017 10:50AM
Hi.

Am 12-04-2017 23:50, schrieb Frank Liu:

> Hi,
>
> How does nginx balances traffic to upstream with different weight?
> If I have 3 servers in upstream, with weight 1, 2, 4, assuming all are
> healthy, will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2,
> 2, 3, 3, 3, 3?
> If I have two servers with both weight 50, will nginx will 50 requests
> to server 1, and then 50 to server 2, or will it calculate the ration
> to be 1:1 and send one after another?

You can find a explanation of the algorithm in the commit messages.

http://hg.nginx.org/nginx/rev/c90801720a0c
http://hg.nginx.org/nginx/rev/d05ab8793a69
http://hg.nginx.org/nginx/rev/0811376954e4

I have found this with this query.

http://hg.nginx.org/nginx/log?rev=weight

Have you seen this doc?
http://nginx.org/en/docs/http/load_balancing.html

Best Regards
Aleks
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Frank Liu
Re: weight and balancing in upstream proxy
April 14, 2017 10:10PM
Hi Aleks,

Those information are extremely helpful. Much appreciated!

Regards,
Frank


On Fri, Apr 14, 2017 at 1:47 AM, Aleksandar Lazic <[email protected]> wrote:

> Hi.
>
> Am 12-04-2017 23:50, schrieb Frank Liu:
>
> Hi,
>>
>> How does nginx balances traffic to upstream with different weight?
>> If I have 3 servers in upstream, with weight 1, 2, 4, assuming all are
>> healthy, will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2,
>> 3, 3, 3, 3?
>> If I have two servers with both weight 50, will nginx will 50 requests to
>> server 1, and then 50 to server 2, or will it calculate the ration to be
>> 1:1 and send one after another?
>>
>
> You can find a explanation of the algorithm in the commit messages.
>
> http://hg.nginx.org/nginx/rev/c90801720a0c
> http://hg.nginx.org/nginx/rev/d05ab8793a69
> http://hg.nginx.org/nginx/rev/0811376954e4
>
> I have found this with this query.
>
> http://hg.nginx.org/nginx/log?rev=weight
>
> Have you seen this doc?
> http://nginx.org/en/docs/http/load_balancing.html
>
> Best Regards
> Aleks
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login