Welcome! Log In Create A New Profile

Advanced

(haproxy) How-TO get HAPROXY to balanace 2 SSL encypted Webservers ?

Posted by Anonymous User 
So we have 2 webservers on the backend with SSL encryption.
We want to keep this the way it is.
Is there a way for HAPROXY to balance these 2 servers with sticky
sessions enabled?

how can this be done?

Currently when trying it this way;

defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
stats enable
stats uri /stats


frontend http-in
bind *:80
acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
use_backend ww2_test1_com if is_ww2_test1_com

backend ww2_test1_com
balance roundrobin
cookie SERVERID insert nocache indirect
option httpchk
option httpclose
option forwardfor
server Server1 10.10.10.11:80 cookie Server1
server Server1 10.10.10.12:80 cookie Server2

Since the 2 servers are encrypted on port 443 (with the main front
page on port 80 not encrypted),
the above setup works until it hits 443 where i get the error
"Error 310 (net::ERR_TOO_MANY_REDIRECTS): There were too many
redirects.".
Port 443 on the HAPROXY frontend is using Pound for the encryption.
However both backend servers have a Tomcat Keystore (signed through
thawte) which I doubt will be compatable with Pound. (and I don't
want to resign the cert or get a new cert)
Can I somehow get HAPROXY to balance these 2 servers with proper
sticky session handling?

TIA!
Where is the rest of your haproxy config - if you are talking to port 443 on
your tomcat servers...

If you have have the 2 backend servers and you want haproxy to talk to the
encrypted/ssl ports on them (and you want your end users to see the certs that
are on the tomcat servers) then the only thing haproxy can "see" is the source
IP and source port and try to create stickiness with the source IP. So you
have to think in those terms - what is unencrypted at the time each request
and response passes through haproxy.

In this case the end user sees the cert installed on pound and haproxy can use
all the layer 7/http capabilities:
ssl/443 -> pound -> non-ssl -> haproxy non-ssl -> tomcat(s)

you can't do (AFAIK):

ssl/443 -> pound -> non-ssl -> haproxy -> ssl -> tomcat(s)
because the user would still see only the pound cert and I don't think haproxy
can initiate ssl sessions on its own.

On 11/15/10 11:08 AM, toms@hush.com wrote:
> So we have 2 webservers on the backend with SSL encryption.
> We want to keep this the way it is.
> Is there a way for HAPROXY to balance these 2 servers with sticky
> sessions enabled?
>
> how can this be done?
>
> Currently when trying it this way;
>
> defaults
> log global
> mode http
> option httplog
> option dontlognull
> retries 3
> option redispatch
> maxconn 2000
> contimeout 5000
> clitimeout 50000
> srvtimeout 50000
> stats enable
> stats uri /stats
>
>
> frontend http-in
> bind *:80
> acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
> use_backend ww2_test1_com if is_ww2_test1_com
>
> backend ww2_test1_com
> balance roundrobin
> cookie SERVERID insert nocache indirect
> option httpchk
> option httpclose
> option forwardfor
> server Server1 10.10.10.11:80 cookie Server1
> server Server1 10.10.10.12:80 cookie Server2
>
> Since the 2 servers are encrypted on port 443 (with the main front
> page on port 80 not encrypted),
> the above setup works until it hits 443 where i get the error
> "Error 310 (net::ERR_TOO_MANY_REDIRECTS): There were too many
> redirects.".
> Port 443 on the HAPROXY frontend is using Pound for the encryption.
> However both backend servers have a Tomcat Keystore (signed through
> thawte) which I doubt will be compatable with Pound. (and I don't
> want to resign the cert or get a new cert)
> Can I somehow get HAPROXY to balance these 2 servers with proper
> sticky session handling?
>
> TIA!
>
>
Thanks.
Are there any config examples I can take a look at?
Specifically having HAPROXY load balance 2 backend SSL encrypted
tomcat servers.
As per your message, I would not be able to use POUND.
How can I configure HAPROXY to only balance the 2 servers' port 443
and apply stickiness to the source IP's?
are there any examples I can look at?

How can I modify the below config to also passthrough, balance and
create the sticky sessions for SSL also?
currently our port 80 load balancing looks like this: (entire
config)

global
log 127.0.0.1:514 local7 # only send important events
maxconn 4096
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
stats enable
stats uri /stats
frontend http-in
bind *:80
acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
use_backend ww2_test1_com if is_ww2_test1_com
backend ww2_test1_com
balance roundrobin
cookie SERVERID insert nocache indirect
option httpchk
option httpclose
option forwardfor
server Server1 10.10.10.11:80 cookie Server1
server Server2 10.10.10.12:80 cookie Server2

thanks again.

ts

On Mon, 15 Nov 2010 14:39:13 -0500 "Hank A. Paulson"
<[email protected]> wrote:
>Where is the rest of your haproxy config - if you are talking to
>port 443 on
>your tomcat servers...
>
>If you have have the 2 backend servers and you want haproxy to
>talk to the
>encrypted/ssl ports on them (and you want your end users to see
>the certs that
>are on the tomcat servers) then the only thing haproxy can "see"
>is the source
>IP and source port and try to create stickiness with the source
>IP. So you
>have to think in those terms - what is unencrypted at the time
>each request
>and response passes through haproxy.
>
>In this case the end user sees the cert installed on pound and
>haproxy can use
>all the layer 7/http capabilities:
>ssl/443 -> pound -> non-ssl -> haproxy non-ssl -> tomcat(s)
>
>you can't do (AFAIK):
>
>ssl/443 -> pound -> non-ssl -> haproxy -> ssl -> tomcat(s)
>because the user would still see only the pound cert and I don't
>think haproxy
>can initiate ssl sessions on its own.
>
>On 11/15/10 11:08 AM, toms@hush.com wrote:
>> So we have 2 webservers on the backend with SSL encryption.
>> We want to keep this the way it is.
>> Is there a way for HAPROXY to balance these 2 servers with
>sticky
>> sessions enabled?
>>
>> how can this be done?
>>
>> Currently when trying it this way;
>>
>> defaults
>> log global
>> mode http
>> option httplog
>> option dontlognull
>> retries 3
>> option redispatch
>> maxconn 2000
>> contimeout 5000
>> clitimeout 50000
>> srvtimeout 50000
>> stats enable
>> stats uri /stats
>>
>>
>> frontend http-in
>> bind *:80
>> acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
>> use_backend ww2_test1_com if is_ww2_test1_com
>>
>> backend ww2_test1_com
>> balance roundrobin
>> cookie SERVERID insert nocache indirect
>> option httpchk
>> option httpclose
>> option forwardfor
>> server Server1 10.10.10.11:80 cookie Server1
>> server Server1 10.10.10.12:80 cookie Server2
>>
>> Since the 2 servers are encrypted on port 443 (with the main
>front
>> page on port 80 not encrypted),
>> the above setup works until it hits 443 where i get the error
>> "Error 310 (net::ERR_TOO_MANY_REDIRECTS): There were too many
>> redirects.".
>> Port 443 on the HAPROXY frontend is using Pound for the
>encryption.
>> However both backend servers have a Tomcat Keystore (signed
>through
>> thawte) which I doubt will be compatable with Pound. (and I
>don't
>> want to resign the cert or get a new cert)
>> Can I somehow get HAPROXY to balance these 2 servers with proper
>> sticky session handling?
>>
>> TIA!
>>
>>
On Mon, Nov 15, 2010 at 03:38:58PM -0500, toms@hush.com wrote:
> Thanks.
> Are there any config examples I can take a look at?
> Specifically having HAPROXY load balance 2 backend SSL encrypted
> tomcat servers.
> As per your message, I would not be able to use POUND.

if you need to re-encrypt the traffic between haproxy and tomcat,
then you can't do that much easily. I've already done it with stunnel,
but the overall chain gets quite complicated :

client
|
| HTTPS/443
v
stunnel in server mode
|
| HTTP/localhost:8443
v
haproxy
|
| HTTP/localhost:8000+#server
v
stunnel in client mode
|
| HTTPS/server:443
v
server

> How can I configure HAPROXY to only balance the 2 servers' port 443
> and apply stickiness to the source IP's?

You can do that in plain TCP mode, so there won't be any HTTP processing.
Source IP stickiness can be configured using the stick-tables. An
alternative generally is to simply perform a source IP hash.

Version 1.5-dev3 makes it possible to use SSL-ID for stickiness, which
is more reliable than the IP address, but is limited in time by some
browsers. A solution could be to mix IP hashing with SSL-ID stickiness
in order to get the best of both worlds: as long as at least one of
them remains, stickiness is maintained.

> are there any examples I can look at?

There are a bit in the doc, but really not that much. Look for "stick-table".

> How can I modify the below config to also passthrough, balance and
> create the sticky sessions for SSL also?
> currently our port 80 load balancing looks like this: (entire
> config)
>
> global
> log 127.0.0.1:514 local7 # only send important events
> maxconn 4096
> user haproxy
> group haproxy
> daemon
> defaults
> log global
> mode http
> option httplog
> option dontlognull
> retries 3
> option redispatch
> maxconn 2000
> contimeout 5000
> clitimeout 50000
> srvtimeout 50000
> stats enable
> stats uri /stats
> frontend http-in
> bind *:80
> acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
> use_backend ww2_test1_com if is_ww2_test1_com
> backend ww2_test1_com
> balance roundrobin
> cookie SERVERID insert nocache indirect
> option httpchk
> option httpclose
> option forwardfor
> server Server1 10.10.10.11:80 cookie Server1
> server Server2 10.10.10.12:80 cookie Server2

For port 443, it would approximately look like this (untested) :

frontend https-in
mode tcp
bind :443
default_backend bk-https

backend bk-https
mode tcp
balance src
option ssl-hello-chk
server Server1 10.10.10.11:443 check
server Server2 10.10.10.12:443 check

But be careful, your servers will only log haproxy's IP address,
and this can clearly become an issue.

Regards,
Willy
What if I don't need to encrypt the traffic between the Haproxy
front and the 2 backend servers?
Is there a way just to have HAProxy passthrough any and all traffic
and balance them?
sort of like LVS works on Layer 4.

tia.



On Tue, 16 Nov 2010 06:18:03 -0500 Willy Tarreau <[email protected]> wrote:
>On Mon, Nov 15, 2010 at 03:38:58PM -0500, toms@hush.com wrote:
>> Thanks.
>> Are there any config examples I can take a look at?
>> Specifically having HAPROXY load balance 2 backend SSL encrypted
>
>> tomcat servers.
>> As per your message, I would not be able to use POUND.
>
>if you need to re-encrypt the traffic between haproxy and tomcat,
>then you can't do that much easily. I've already done it with
>stunnel,
>but the overall chain gets quite complicated :
>
>client
> |
> | HTTPS/443
> v
>stunnel in server mode
> |
> | HTTP/localhost:8443
> v
>haproxy
> |
> | HTTP/localhost:8000+#server
> v
>stunnel in client mode
> |
> | HTTPS/server:443
> v
>server
>
>> How can I configure HAPROXY to only balance the 2 servers' port
>443
>> and apply stickiness to the source IP's?
>
>You can do that in plain TCP mode, so there won't be any HTTP
>processing.
>Source IP stickiness can be configured using the stick-tables. An
>alternative generally is to simply perform a source IP hash.
>
>Version 1.5-dev3 makes it possible to use SSL-ID for stickiness,
>which
>is more reliable than the IP address, but is limited in time by
>some
>browsers. A solution could be to mix IP hashing with SSL-ID
>stickiness
>in order to get the best of both worlds: as long as at least one
>of
>them remains, stickiness is maintained.
>
>> are there any examples I can look at?
>
>There are a bit in the doc, but really not that much. Look for
>"stick-table".
>
>> How can I modify the below config to also passthrough, balance
>and
>> create the sticky sessions for SSL also?
>> currently our port 80 load balancing looks like this: (entire
>> config)
>>
>> global
>> log 127.0.0.1:514 local7 # only send important
>events
>> maxconn 4096
>> user haproxy
>> group haproxy
>> daemon
>> defaults
>> log global
>> mode http
>> option httplog
>> option dontlognull
>> retries 3
>> option redispatch
>> maxconn 2000
>> contimeout 5000
>> clitimeout 50000
>> srvtimeout 50000
>> stats enable
>> stats uri /stats
>> frontend http-in
>> bind *:80
>> acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
>> use_backend ww2_test1_com if is_ww2_test1_com
>> backend ww2_test1_com
>> balance roundrobin
>> cookie SERVERID insert nocache indirect
>> option httpchk
>> option httpclose
>> option forwardfor
>> server Server1 10.10.10.11:80 cookie Server1
>> server Server2 10.10.10.12:80 cookie Server2
>
>For port 443, it would approximately look like this (untested) :
>
>frontend https-in
> mode tcp
> bind :443
> default_backend bk-https
>
>backend bk-https
> mode tcp
> balance src
> option ssl-hello-chk
> server Server1 10.10.10.11:443 check
> server Server2 10.10.10.12:443 check
>
>But be careful, your servers will only log haproxy's IP address,
>and this can clearly become an issue.
>
>Regards,
>Willy
Does anyone have an answer to this?
Is there a way to balance 2 SSL encrypted (tomcat) webservers with
HAPROXY alone?
if so can someone please point out some config examples?
reading the documentation doesn't give this scenario.

tia.

On Tue, 16 Nov 2010 09:28:58 -0500 toms@hush.com wrote:
>What if I don't need to encrypt the traffic between the Haproxy
>front and the 2 backend servers?
>Is there a way just to have HAProxy passthrough any and all
>traffic
>and balance them?
>sort of like LVS works on Layer 4.
>
>tia.
>
>
>
>On Tue, 16 Nov 2010 06:18:03 -0500 Willy Tarreau <[email protected]> wrote:
>>On Mon, Nov 15, 2010 at 03:38:58PM -0500, toms@hush.com wrote:
>>> Thanks.
>>> Are there any config examples I can take a look at?
>>> Specifically having HAPROXY load balance 2 backend SSL
>encrypted
>>
>>> tomcat servers.
>>> As per your message, I would not be able to use POUND.
>>
>>if you need to re-encrypt the traffic between haproxy and tomcat,
>>then you can't do that much easily. I've already done it with
>>stunnel,
>>but the overall chain gets quite complicated :
>>
>>client
>> |
>> | HTTPS/443
>> v
>>stunnel in server mode
>> |
>> | HTTP/localhost:8443
>> v
>>haproxy
>> |
>> | HTTP/localhost:8000+#server
>> v
>>stunnel in client mode
>> |
>> | HTTPS/server:443
>> v
>>server
>>
>>> How can I configure HAPROXY to only balance the 2 servers' port
>
>>443
>>> and apply stickiness to the source IP's?
>>
>>You can do that in plain TCP mode, so there won't be any HTTP
>>processing.
>>Source IP stickiness can be configured using the stick-tables. An
>>alternative generally is to simply perform a source IP hash.
>>
>>Version 1.5-dev3 makes it possible to use SSL-ID for stickiness,
>>which
>>is more reliable than the IP address, but is limited in time by
>>some
>>browsers. A solution could be to mix IP hashing with SSL-ID
>>stickiness
>>in order to get the best of both worlds: as long as at least one
>>of
>>them remains, stickiness is maintained.
>>
>>> are there any examples I can look at?
>>
>>There are a bit in the doc, but really not that much. Look for
>>"stick-table".
>>
>>> How can I modify the below config to also passthrough, balance
>>and
>>> create the sticky sessions for SSL also?
>>> currently our port 80 load balancing looks like this: (entire
>>> config)
>>>
>>> global
>>> log 127.0.0.1:514 local7 # only send important
>>events
>>> maxconn 4096
>>> user haproxy
>>> group haproxy
>>> daemon
>>> defaults
>>> log global
>>> mode http
>>> option httplog
>>> option dontlognull
>>> retries 3
>>> option redispatch
>>> maxconn 2000
>>> contimeout 5000
>>> clitimeout 50000
>>> srvtimeout 50000
>>> stats enable
>>> stats uri /stats
>>> frontend http-in
>>> bind *:80
>>> acl is_ww2_test1_com hdr_end(host) -i ww2.test1.com
>>> use_backend ww2_test1_com if is_ww2_test1_com
>>> backend ww2_test1_com
>>> balance roundrobin
>>> cookie SERVERID insert nocache indirect
>>> option httpchk
>>> option httpclose
>>> option forwardfor
>>> server Server1 10.10.10.11:80 cookie Server1
>>> server Server2 10.10.10.12:80 cookie Server2
>>
>>For port 443, it would approximately look like this (untested) :
>>
>>frontend https-in
>> mode tcp
>> bind :443
>> default_backend bk-https
>>
>>backend bk-https
>> mode tcp
>> balance src
>> option ssl-hello-chk
>> server Server1 10.10.10.11:443 check
>> server Server2 10.10.10.12:443 check
>>
>>But be careful, your servers will only log haproxy's IP address,
>>and this can clearly become an issue.
>>
>>Regards,
>>Willy
Re: (haproxy) How-TO get HAPROXY to balanace 2 SSL encypted Webservers ?
February 10, 2012 09:42PM
Hi. Willy. I am interested in your haproxy and stunnel setting "about re-encrypt the traffic between haproxy and tomcat". could you post it, please?

Thanks

Jie
Hi,

Assuming you're running stunnel 4.50 and HAProxy 1.5:
Stunnel conf:
[frontend_ssl_offloading]
cert = /path/to/certificate
key = /path/to/key
accept = LISTENING_IP:443
connect = 127.0.0.1:80
protocol = proxy

[server1_ssl_connection]
cert = /path/to/certificate
key = /path/to/key
accept = 127.0.1.2:80
connect = SERVER_IP:443
protocol = proxy
client = yes


HAProxy configuration:
frontend web
mode http
option http-server-close
default_backend web

backend web
mode http
option http-server-close
balance roundrobin
cookie SERVERID insert indirect nocache
server srv1 127.0.1.2:80 check cookie srv1




The flow will be like that:
client -> stunnel frontend -> HAProxy frontend -> HAProxy backend ->
stunnel server -> server

Hope this helps

cheers







On Fri, Feb 10, 2012 at 9:42 PM, <[email protected]> wrote:
> Hi. Willy. I am interested in your haproxy and stunnel setting "about re-encrypt the traffic between haproxy and tomcat". could you post it, please?
>
> Thanks
>
> Jie
>
> ---
> posted at http://www.serverphorums.com
> http://www.serverphorums.com/read.php?10,230949,444281#msg-444281
>
Re: (haproxy) How-TO get HAPROXY to balanace 2 SSL encypted Webservers ?
February 13, 2012 07:31PM
does your configuration only work for Haproxy 1.5? I am using Haproxy 1.4.8

Thx

Jie
Re: (haproxy) How-TO get HAPROXY to balanace 2 SSL encypted Webservers ?
February 14, 2012 03:56PM
Hi.
I have a qeustion about traffic between HAProxy backend and Stunnel server. Per your configuration, HaProxy will connect 127.0.1.2:80 after the load balancing. Will HaProxy connect back to that port with SSL, even thougth the stunnel enables client=yes mode? My understanding is that stunnel is only listening https tranffic.

Thanks

Jie
Sorry, only registered users may post in this forum.

Click here to login