Welcome! Log In Create A New Profile

Advanced

Need some help in HAPROXY setup.

Posted by amit raj 
amit raj
Need some help in HAPROXY setup.
March 07, 2018 03:10PM
Hello Team,


We are facing some issue while running HAPROXY our environment.

1.Currently we are using the HAPROXY 1.8 build following are the problems
we need to solve basically.


a.> Connection closing should happen with FIN rather than RSTs
b.> Even load balancing based on both session and request in and request
out.


Please help us in achieving both these.


Currently we are using two configs in our HAPROXY.

in the frontend block we are using the "option http-tunnel" and in the
backend we are using the "option http keep-alive" and also "no http-tunnel"


With the current above setup we can achieve point a , but in point b we can
achieve session based even load balancing not request in and request out.


Any help would be deeply appreciated.


Thanks,
AMIT
Aleksandar Lazic
Re: Need some help in HAPROXY setup.
March 07, 2018 08:50PM
Hi amit.

Am 07.03.2018 um 14:44 schrieb amit raj:
> Hello Team,
>
>
> We are facing some issue  while running HAPROXY  our environment.
>
> 1.Currently we are using the HAPROXY 1.8 build following are the
> problems we need to solve basically.
>
> a.> Connection closing should happen with FIN rather than RSTs

When?
- at normal request flow
- at health check

> b.> Even load balancing based on both session and request in  and
> request out.

I don't understand what you mean?

The default behavior of haproxy when you setup the session stickiness
properly is to send the client request from a server back to the same
client.

> Please help us in achieving both these.
>
> Currently we are using two configs in our HAPROXY.
>
> in the frontend block we are using the "option http-tunnel" and in the
> backend  we are using the "option http keep-alive" and also "no http-tunnel"

Please attach the haproxy config so that we can understand you config
better.

> With the current above setup we can achieve point a , but in point b we
> can achieve session based even load balancing not  request in  and
> request out.
>
> Any help would be deeply appreciated.
>
> Thanks,
> AMIT

Regards
Aleks
Aleksandar Lazic
Re: Need some help in HAPROXY setup.
March 08, 2018 10:50PM
Hi Amit.

Please keep the mailing list in loop, thanks.

Am 08.03.2018 um 17:03 schrieb amit raj:
> Hello Alex,
>
> Two things we have to achieve with the HAPROXY.
>
> 1.There should be no RSTs for closing the connection.(which is happening
> when we are keeping option http-keep-alive in ur frontend and backend.)

Have you tried this options?

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-timeout%20server-fin

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-timeout%20client-fin

Why is the RST a problem?

> 2.Even load balancing not only with session load balancing also with the
> bytes in and bytes out , as you can see on the HAPROXy stats page.
> (We have used option-http tunnel for reducing the  RST ,but due to this
> our even load balancing is screwed up for bytes in and bytes out).

Let me describe what I have understood.

You want to use more the one balancing decision base.

* session (cookie, url, ...)
* bytes in. I assume from the client
* bytes out. I assume from the client

I this right?

Please can you describe in a ASCII flow how the decision flow should work..

What I have seen in the doc are this acl samples

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-table_bytes_in_rate
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-table_bytes_out_rate

But I'm not sure if this is what you need.

> Following our HA proxy config.
>
> frontend test_ha
>     bind.  :XXXX
>     option http-tunnel
>     default_backend test_back
> #------------------
> # backend instances
> #------------------
> backend test_back
>     balance roundrobin
>     option no-http-tunnel
>     option http-keep-alive
>     option http-reuse aggresssive

That's a short conf and not a working one.
Please post the current conf which works, thanks.

> Please help us .

Best regards
Aleks
Aleksandar Lazic
Re: Need some help in HAPROXY setup.
March 09, 2018 06:40PM
Hi Amit.

*Please keep the mailing list in loop, thanks.*

In case you haven't received my last mail here is the web link to the
maillinglist archive.

https://www.mail-archive.com/[email protected]/msg29252.html

Am 09.03.2018 um 07:43 schrieb amit raj:

> Hello Alex,
>
> Two things we have to achieve with the HAPROXY to be very clear.
>
> 1.The most important thing we want to achieve is uniform Load Balancing
> (with sessions and bytes in and bytes out parameter should be uniform
> across all the backend).

Let me describe what I have understood.

You want to use more the one balancing decision base.

* session (cookie, url, ...)
* bytes in. I assume from the client
* bytes out. I assume from the client

I this right? # <= Please answer this question, thanks.

Please can you describe in a ASCII flow how the decision flow should
work.

What I have seen in the doc are this acl samples

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-table_bytes_in_rate
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-table_bytes_out_rate

But I'm not sure if this is what you need.

> 2.When we have tried using keep-alive mode for achieving uniform load
> balancing , the HAPROXY is sending RST for closing the connection with
> the backend , this is what we really do not want .As per our use case
> HAPROXY should use FIN to close the backend connection.
>
> Currently we are using "option-http-tunnel" in the frontend of the
> application and "no-http-tunnel" and "keep-alive" in the backend this
> solves our RST issue and HAPROXY is closing the connection properly
> with
> FIN.But the uniform Load Balancing as I mentioned in point one got
> screwed up.
>
> Following are my basic HAPROXY config we are using for testing and
> fixing the issue.

Thank you for the full haproxy config.

> global
>
> log /dev/log local0
> log /dev/log local1 notice
> # lua-load /etc/haproxy/parseJsonValue.lua
> stats socket /var/run/haproxy/haproxy.sock mode 777 level admin
> expose-fd listeners
> chroot /var/lib/haproxy
> #lua-load /etc/haproxy/lua
> stats timeout 30s
> user haproxy
> group haproxy
> # daemon
> maxconn 2000
>
> defaults
>
> log global
> mode http
> retries 3
> option httplog
> #option http-server-close
> option dontlognull
> option forwardfor
> timeout connect 5000
> timeout client 50000
> timeout server 50000
> errorfile 400 /etc/haproxy/errors/400.http
> errorfile 403 /etc/haproxy/errors/403.http
> errorfile 408 /etc/haproxy/errors/408.http
> errorfile 500 /etc/haproxy/errors/500.http
> errorfile 502 /etc/haproxy/errors/502.http
> errorfile 503 /etc/haproxy/errors/503.http
> errorfile 504 /etc/haproxy/errors/504.http
>
> #------------------
> # internal statistics
> #------------------
>
> listen stats
> bind [email protected]:8999
> mode http
> stats enable
> stats uri /stats
> #------------------
> # frontend instances
> #------------------
>
> frontend test_ha
>
> bind [email protected]:8080
> option http-tunnel
> # http-request add-header X-Internal-Value %[lua.parseJsonValue]
> default_backend test_back
>
> #------------------
> # backend instances
> #------------------
>
> backend test_back
>
> # http-request set-header X-LB
> %[req.hdr(Host),lower]%[req.uri,lower]
> balance roundrobin
>
> no option http-tunnel
> option http-keep-alive> # option http-server-close
> #option http-buffer-request
>
> # stick-table type string size 30k expire 30m
> # stick on "lua.parseElement" table test_back
>
> server test2 test02.xyz.abc.com:80 http://test02.xyz.abc.com:80
> server test3 test03.xy.abc.com:80 http://test03.xy.abc.com:80
>
> Please let me know if you need any other info.

Ah I think I understand now what you mean.

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-stick-table

I haven't used stick-table very extensive so I hope a more experienced
Person will correct my suggestion.

--------------
frontend test_ha

bind [email protected]:8080
option http-tunnel
# http-request add-header X-Internal-Value %[lua.parseJsonValue]

stick-table type integer size 30k expire 30m store bytes_in_cnt

default_backend counter_out

backend counter_out
bind [email protected]:8081
option http-tunnel

stick-table type integer size 30k expire 30m store bytes_out_cnt
default_backend test_ha

backend be_test_ha

bind [email protected]:8082
option http-tunnel

# http-request add-header X-Internal-Value %[lua.parseJsonValue]

default_backend test_back

backend test_back

# http-request set-header X-LB %[req.hdr(Host),lower]%[req.uri,lower]
balance roundrobin>
no option http-tunnel
option http-keep-alive
# option http-server-close
#option http-buffer-request

# stick-table type string size 30k expire 30m
# stick on "lua.parseElement" table test_back

server test2 test02.xyz.abc.com:80 http://test02.xyz.abc.com:80
server test3 test03.xy.abc.com:80 http://test03.xy.abc.com:80
-----------------

Finally the question is, afaiu, how to balance based on test_ha AND
counter_out AND lua.parseElement

Best regards
Aleks
Aleksandar Lazic
Re: Need some help in HAPROXY setup.
March 16, 2018 02:20AM
Hi all.

I resend this just to highlight the question from below.

> Finally the question is, afaiu, how to balance based on test_ha AND
> counter_out AND lua.parseElement

This are different tables.

Best regards
Aleks

Am 09.03.2018 um 18:28 schrieb Aleksandar Lazic:
> Hi Amit.
>
> *Please keep the mailing list in loop, thanks.*
>
> In case you haven't received my last mail here is the web link to the
> maillinglist archive.
>
> https://www.mail-archive.com/[email protected]/msg29252.html
>
> Am 09.03.2018 um 07:43 schrieb amit raj:
>
>> Hello Alex,
>>
>> Two things we have to achieve with the HAPROXY to be very clear.
>>
>> 1.The most important thing we want to achieve is uniform Load Balancing
>> (with sessions and bytes in and bytes out parameter should be uniform
>> across all the backend).
>
> Let me describe what I have understood.
>
> You want to use more the one balancing decision base.
>
> * session (cookie, url, ...)
> * bytes in. I assume from the client
> * bytes out. I assume from the client
>
> I this right? # <= Please answer this question, thanks.
>
> Please can you describe in a ASCII flow how the decision flow should work.
>
> What I have seen in the doc are this acl samples
>
> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-table_bytes_in_rate
>
> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-table_bytes_out_rate
>
>
> But I'm not sure if this is what you need.
>
>> 2.When we have tried using keep-alive mode for achieving uniform load
>> balancing , the HAPROXY is sending RST for closing the connection with
>> the backend , this is what we really do not want .As per our use case
>> HAPROXY should use FIN to close the backend connection.
>>
>> Currently we are using "option-http-tunnel" in the frontend of the
>> application and "no-http-tunnel" and "keep-alive" in the backend this
>> solves our RST issue and HAPROXY is closing the connection properly with
>> FIN.But the uniform Load Balancing  as I mentioned in point one got
>> screwed up.
>>
>> Following are  my basic HAPROXY config we are using for testing and
>> fixing  the issue.
>
> Thank you for the full haproxy config.
>
>> global
>>
>>   log /dev/log    local0
>>   log /dev/log    local1 notice
>> #        lua-load /etc/haproxy/parseJsonValue.lua
>>   stats socket /var/run/haproxy/haproxy.sock mode 777 level admin
>>   expose-fd listeners
>>   chroot /var/lib/haproxy
>>   #lua-load /etc/haproxy/lua
>>   stats timeout 30s
>>   user haproxy
>>   group haproxy
>>   #        daemon
>>   maxconn 2000
>>
>> defaults
>>
>>   log     global
>>   mode    http
>>   retries 3
>>   option  httplog
>>   #option http-server-close
>>   option  dontlognull
>>   option forwardfor
>>   timeout connect 5000
>>   timeout client  50000
>>   timeout server  50000
>>   errorfile 400 /etc/haproxy/errors/400.http
>>   errorfile 403 /etc/haproxy/errors/403.http
>>   errorfile 408 /etc/haproxy/errors/408.http
>>   errorfile 500 /etc/haproxy/errors/500.http
>>   errorfile 502 /etc/haproxy/errors/502.http
>>   errorfile 503 /etc/haproxy/errors/503.http
>>   errorfile 504 /etc/haproxy/errors/504.http
>>
>> #------------------
>> # internal statistics
>> #------------------
>>
>> listen stats
>>   bind [email protected]:8999
>>   mode http
>>   stats enable
>>   stats uri  /stats
>> #------------------
>> # frontend instances
>> #------------------
>>
>> frontend test_ha
>>
>>   bind [email protected]:8080
>>   option http-tunnel
>>   #    http-request add-header X-Internal-Value %[lua.parseJsonValue]
>>   default_backend test_back
>>
>> #------------------
>> # backend instances
>> #------------------
>>
>> backend test_back
>>
>>   #  http-request set-header X-LB %[req.hdr(Host),lower]%[req.uri,lower]
>>   balance roundrobin
>>
>>   no option http-tunnel
>>   option http-keep-alive>    # option http-server-close
>>   #option http-buffer-request
>>
>> #        stick-table type string size 30k expire 30m
>> #        stick on "lua.parseElement" table test_back
>>
>>   server test2 test02.xyz.abc.com:80 http://test02.xyz.abc.com:80
>>   server test3 test03.xy.abc.com:80 http://test03.xy.abc.com:80
>>
>> Please let me know if you need any other info.
>
> Ah I think I understand now what you mean.
>
> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-stick-table
>
> I haven't used stick-table very extensive so I hope a more experienced
> Person will correct my suggestion.
>
> --------------
> frontend test_ha
>
>     bind [email protected]:8080
>     option http-tunnel
> #    http-request add-header X-Internal-Value %[lua.parseJsonValue]
>
>     stick-table type integer size 30k expire 30m store bytes_in_cnt
>
>     default_backend counter_out
>
> backend counter_out
>     bind [email protected]:8081
>     option http-tunnel
>
>     stick-table type integer size 30k expire 30m store bytes_out_cnt
>     default_backend test_ha
>
> backend be_test_ha
>
>      bind [email protected]:8082
>      option http-tunnel
>
>  #    http-request add-header X-Internal-Value %[lua.parseJsonValue]
>
>      default_backend test_back
>
> backend test_back
>
>   #  http-request set-header X-LB %[req.hdr(Host),lower]%[req..uri,lower]
>     balance roundrobin>
>     no option http-tunnel
>     option http-keep-alive
>     # option http-server-close
>     #option http-buffer-request
>
>  #        stick-table type string size 30k expire 30m
>  #        stick on "lua.parseElement" table test_back
>
>      server test2 test02.xyz.abc.com:80 http://test02.xyz.abc.com:80
>      server test3 test03.xy.abc.com:80 http://test03.xy.abc.com:80
> -----------------
>
> Finally the question is, afaiu, how to balance based on test_ha AND
> counter_out AND lua.parseElement
>
> Best regards
> Aleks
>
Sorry, only registered users may post in this forum.

Click here to login