Welcome! Log In Create A New Profile

Advanced

Health Checks not run before attempting to use backend

Posted by Dave Chiluk 
Dave Chiluk
Health Checks not run before attempting to use backend
April 13, 2018 01:10AM
Hi we're evaluating haproxy for use as the load balancer in front of our
mesos cluster. What we are finding is that even though we have requested
the check option in the server line, haproxy attempts to serve traffic to
the server on startup until the first healthcheck completes.

server slot1 10.40.40.2:7070 check inter 1000 rise 3 fall 3 maxconn 32

This is because we are adding servers to haproxy as they are started in
mesos, but before our backend application itself is ready to serve
connections. This results in spurious 503's being handed to clients as we
add backends via the admin socket or haproxy restart. I looked into
possibly forcing a healthcheck during the cfgparse constructors, but that
seems like it would require some significant rearchitecting.

Is there a way to force haproxy to not use a backend until it passes a
healthcheck? I'm also worried about the side affects this might cause as
requests start to queue up in the haproxy.

Thanks,
Dave
Jonathan Matthews
Re: Health Checks not run before attempting to use backend
April 13, 2018 07:40PM
On Fri, 13 Apr 2018 at 00:01, Dave Chiluk <[email protected]> wrote:

> Is there a way to force haproxy to not use a backend until it passes a
> healthcheck? I'm also worried about the side affects this might cause as
> requests start to queue up in the haproxy
>

I asked about this in 2014 ("Current solutions to the
soft-restart-healthcheck-spread problem?") and I don't recall seeing a fix
since then. Very interested in whatever you find out!


J

> --
Jonathan Matthews
London, UK
http://www.jpluscplusm.com/contact.html
Well after having read your thread that's disappointing. An alternative
solution to forcing healthchecks before the bind It would be nice to have
an option to initially start all servers in the down state unless
explicitly loaded as up via a "show servers state
/load-server-state-from-file" option.

Additionally, in a "seamless reload" configuration as we are using, would
it be possible for the new haproxy to complete a healthcheck on backends
after it has bound to the socket, but before it has signaled the old
haproxy, or am I missing another gotcha there.

Also we are doing all this using 1.8.7.

Thanks,
Dave



On Fri, Apr 13, 2018 at 12:35 PM Jonathan Matthews <[email protected]>
wrote:

> On Fri, 13 Apr 2018 at 00:01, Dave Chiluk <[email protected]>
> wrote:
>
>> Is there a way to force haproxy to not use a backend until it passes a
>> healthcheck? I'm also worried about the side affects this might cause as
>> requests start to queue up in the haproxy
>>
>
> I asked about this in 2014 ("Current solutions to the
> soft-restart-healthcheck-spread problem?") and I don't recall seeing a fix
> since then. Very interested in whatever you find out!
>
>
> J
>
>> --
> Jonathan Matthews
> London, UK
> http://www.jpluscplusm.com/contact.html
>
Sorry, only registered users may post in this forum.

Click here to login