Welcome! Log In Create A New Profile


failover to the next upstream server if one of the servers is slow

Posted by ebondar 

i'm trying to setup a failover configuration between two upstream servers,
all works as expected. But i want to cover the case if one of the upstream
servers is became a very slow and i want to remove this servers from
rotation and move all requests to the second upstream server.

upstream rubyfe {
server qa-vmf01.int:443;
server qa-vmf02.int:443;

server {
listen 443 http2;
server_name qa-www.example.com;
gzip on;
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;
proxy_buffering on;
location / {
proxy_read_timeout 1;
proxy_pass https://rubyfe;
proxy_next_upstream error timeout invalid_header
http_500 http_502 http_503 http_504;
proxy_next_upstream_timeout 3;

when i tried to load the first server
#stress --cpu 80 --io 8 --vm 4 --vm-bytes 300M --timeout 180s

# wrk -t2 -c20 -d30s https://qa-www.example.com

in log files i catched request from "wrk" - - [22/Jun/2018:17:59:10 -0400] "GET / HTTP/1.1" 200 50393 "-"
"-" "-" ", [ 1.001, 0.814 ]" "text/html;
charset=utf-8" "-" "582bd020659715d66afafad533f7ac5d"

but at this time i tried to get result via "curl" i seen

$ curl -IL https://qa-www.example.com/
HTTP/1.1 502 Bad Gateway

could you help me to understand how i can operate timeouts of requests to
the upstream servers and in case if one of the upstream server became is
slow to force requests switch to another server

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,280259,280259#msg-280259

nginx mailing list
You may have to resort to Lua (openresty) and periodically perform via a sub
request a query which should indicate how fast an upstream is and decide
weather to take it offline (which also can be done with Lua).

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,280259,280260#msg-280260

nginx mailing list
Thanks itpp2012, I'll look at LUA

if I understand correctly, we can not specify the timeout of the session to
upstream the server, initiate a timeout error, and force requests to move to
another server?

proxy_read_timeout 1;
-> proxy_next_upstream error timeout

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,280259,280261#msg-280261

nginx mailing list
Can't tell atm. if a timeout forces a node to become offline but 60s is
still along time to wait (and decide) without actually knowing if a node is
overloaded (it might be just busy which does not always mean overloaded).

There are tools for fi. edge routers which polls a status page to decide to
change routing or not, in your case you first need to determine what exactly
the conditions are for a node to be slow and then design a way to detect
this, then you can look for tooling to automate whatever you want happening.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,280259,280263#msg-280263

nginx mailing list
Sorry, only registered users may post in this forum.

Click here to login