Welcome! Log In Create A New Profile

Advanced

server DOWN and can not auto restore to MAINT state when use DNS SRV resolvers

Posted by slene 
I'm a new fisher of haproxy. I have some question when I use consul and
registrator for service discovery. And use haproxy as balancer.

My steps:

1. running two container and publish port 80 but not really listening to 80.
2. haproxy found two server failed status: 0/9 DOWN. (not tried times, only
once)
3. remove two container and SRV records already empty.
4. haproxy found two server failed and enters maintenance.
5. now two server are full DOWN and can not enter MAINT state. must be
restore manual by myself (set state to MAINT -> health force up -> set
state to READY)

I think the servers should change state to MAINT and ready for DNS SRV
records available instead of just DOWN and need restore manual. Am I right?

Is it a bug? Or I miss something. Thanks.

config and logs:
https://gist.github.com/slene/0feaa1e68adfaae1b7868871dd3c0196
Hi.

------ Originalnachricht ------
Von: "slene" <[email protected]>
An: haproxy@formilux.org
Gesendet: 05.12.2017 11:36:27
Betreff: server DOWN and can not auto restore to MAINT state when use
DNS SRV resolvers

>I'm a new fisher of haproxy. I have some question when I use consul and
>registrator for service discovery. And use haproxy as balancer.
>
>My steps:
>
>1. running two container and publish port 80 but not really listening
>to 80.
What does this mean?


>2. haproxy found two server failed status: 0/9 DOWN. (not tried times,
>only once)
>3. remove two container and SRV records already empty.
>4. haproxy found two server failed and enters maintenance.
>5. now two server are full DOWN and can not enter MAINT state. must be
>restore manual by myself (set state to MAINT -> health force up -> set
>state to READY)
>
>I think the servers should change state to MAINT and ready for DNS SRV
>records available instead of just DOWN and need restore manual. Am I
>right?
>
>Is it a bug? Or I miss something. Thanks.
>
>config and logs:
>https://gist.github.com/slene/0feaa1e68adfaae1b7868871dd3c0196

Cool you use the new server-template feature ;-)

As you writen in 3 there is "DNS NX" (=> no dns entry) for all 3
servers, what do you expect when the resolver can't find a ip or srv
entry?

Regards
aleks
>1. running two container and publish port 80 but not really listening
>to 80.
I simulate a service fault. (eg: port missed config).
And in step 2 the health check has layer4 failed: Connection refused.

> As you writen in 3 there is "DNS NX" (=> no dns entry) for all 3 servers,
what do you expect when the resolver can't find a ip or srv entry?

See the image in this comment:
https://gist.github.com/slene/0feaa1e68adfaae1b7868871dd3c0196#gistcomment-2277122

Now the server enter DOWN and can not return back to MAINT. Although it
said enters maintenance but not. I think http1/2 should be enable state
like http3 wait for SRV records available.





Aleksandar Lazic <[email protected]>于2017年12月6日周三 上午7:44写道:

> Hi.
>
> ------ Originalnachricht ------
> Von: "slene" <[email protected]>
> An: haproxy@formilux.org
> Gesendet: 05.12.2017 11:36:27
> Betreff: server DOWN and can not auto restore to MAINT state when use
> DNS SRV resolvers
>
> >I'm a new fisher of haproxy. I have some question when I use consul and
> >registrator for service discovery. And use haproxy as balancer.
> >
> >My steps:
> >
> >1. running two container and publish port 80 but not really listening
> >to 80.
> What does this mean?
>
>
> >2. haproxy found two server failed status: 0/9 DOWN. (not tried times,
> >only once)
> >3. remove two container and SRV records already empty.
> >4. haproxy found two server failed and enters maintenance.
> >5. now two server are full DOWN and can not enter MAINT state. must be
> >restore manual by myself (set state to MAINT -> health force up -> set
> >state to READY)
> >
> >I think the servers should change state to MAINT and ready for DNS SRV
> >records available instead of just DOWN and need restore manual. Am I
> >right?
> >
> >Is it a bug? Or I miss something. Thanks.
> >
> >config and logs:
> >https://gist.github.com/slene/0feaa1e68adfaae1b7868871dd3c0196
>
> Cool you use the new server-template feature ;-)
>
> As you writen in 3 there is "DNS NX" (=> no dns entry) for all 3
> servers, what do you expect when the resolver can't find a ip or srv
> entry?
>
> Regards
> aleks
>
>
Hi All,


On 12/06/2017 04:00 AM, slene wrote:
>>1. running two container and publish port 80 but not really listening
>>to 80.
> I simulate a service fault. (eg: port missed config).
> And in step 2 the health check has layer4 failed: Connection refused.
>
>> As you writen in 3 there is "DNS NX" (=> no dns entry) for all 3 servers, what do you expect when the resolver can't find a ip or srv entry?
>
> See the image in this comment:
> https://gist.github.com/slene/0feaa1e68adfaae1b7868871dd3c0196#gistcomment-2277122
>
> Now the server enter DOWN and can not return back to MAINT. Although it said enters maintenance but not. I think http1/2 should be enable state like http3 wait for SRV records available.
>
>
>
>
>
> Aleksandar Lazic <[email protected] <mailto:[email protected]>>于2017年12月6日周三 上午7:44写道:
>
> Hi.
>
> ------ Originalnachricht ------
> Von: "slene" <[email protected] <mailto:[email protected]>>
> An: haproxy@formilux.org <mailto:[email protected]>
> Gesendet: 05.12.2017 11:36:27
> Betreff: server DOWN and can not auto restore to MAINT state when use
> DNS SRV resolvers
>
> >I'm a new fisher of haproxy. I have some question when I use consul and
> >registrator for service discovery. And use haproxy as balancer.
> >
> >My steps:
> >
> >1. running two container and publish port 80 but not really listening
> >to 80.
> What does this mean?
>
>
> >2. haproxy found two server failed status: 0/9 DOWN. (not tried times,
> >only once)
> >3. remove two container and SRV records already empty.
> >4. haproxy found two server failed and enters maintenance.
> >5. now two server are full DOWN and can not enter MAINT state. must be
> >restore manual by myself (set state to MAINT -> health force up -> set
> >state to READY)
> >
> >I think the servers should change state to MAINT and ready for DNS SRV
> >records available instead of just DOWN and need restore manual. Am I
> >right?
> >
> >Is it a bug? Or I miss something. Thanks.
> >
> >config and logs:
> >https://gist.github.com/slene/0feaa1e68adfaae1b7868871dd3c0196
>
> Cool you use the new server-template feature ;-)
>
> As you writen in 3 there is "DNS NX" (=> no dns entry) for all 3
> servers, what do you expect when the resolver can't find a ip or srv
> entry?
>
> Regards
> aleks
>

Latest commit "BUG/MEDIUM: checks: a down server going to maint remains definitely stucked on down.."
should fix this issue.

R,
Emeric
Sorry, only registered users may post in this forum.

Click here to login