Welcome! Log In Create A New Profile

Advanced

haproxy without balancing

Posted by Johan Hendriks 
Johan Hendriks
haproxy without balancing
January 05, 2018 11:40AM
Hello.
First off all I wish everyone a really good 2018. And hopefully 2018
will serve a lot of good memory's.

BTW if this is the wrong list please excuse me.

We have an application running over multiple servers which all have
there own subdomain, there are about 12 of them.
We can live without loadbalancing, so there is no failover, each server
serves a couple of subdomains.
At this moment every server has its own ip, and so every subdomain has a
different DNS entry. What we want is a single point of entry and use
haproxy to route traffic to the right backend server.
Replacing an server is not easy at the moment. We have a lot of history
to deal with. We are working on it to leave that behind but till then we
need an solution.


I looked at this and i think i have two options.
Create for each server in the backend an ip on the haproxy machine and
connect a frontend for that IP to the desired backend server.
This way we still have multiple ipadresses, but they can stay the same
if servers come and go.

Secondly we could use a single ip and use ACL to route the traffic to
the right backend server.
The problem with the second option is that we have around 2000 different
subdomains and this number is still growing. So my haproxy config will
then consists over 4000 lines of acl rules.
and I do not know if haproxy can deal with that or if it will slowdown
request to much.

Maybe there are other options I did not think about?
For me the second config is the best option because of the single IP,
but i do not know if haproxy can handle 2000 acl rules.

Thank you for your time.

Regards
Johan
Jonathan Matthews
Re: haproxy without balancing
January 05, 2018 11:50AM
On 5 January 2018 at 10:28, Johan Hendriks <[email protected]> wrote:
> BTW if this is the wrong list please excuse me.

This looks to me like it might be the right list :-)

> We have an application running over multiple servers which all have
> there own subdomain, there are about 12 of them.
> We can live without loadbalancing, so there is no failover, each server
> serves a couple of subdomains.

What protocols are these servers serving?

- HTTP
- HTTPS
- if HTTPS, do you control the TLS certificates and their private keys?
- Something else?
- if something else, what?

> At this moment every server has its own ip, and so every subdomain has a
> different DNS entry. What we want is a single point of entry and use
> haproxy to route traffic to the right backend server.

Are the DNS entries for every subdomain under your control?
How painful would it be to change one of them?
How painful would it be to change all of them?

> Replacing an server is not easy at the moment. We have a lot of history
> to deal with. We are working on it to leave that behind but till then we
> need an solution.
>
> I looked at this and i think i have two options.
> Create for each server in the backend an ip on the haproxy machine and
> connect a frontend for that IP to the desired backend server.
> This way we still have multiple ipadresses, but they can stay the same
> if servers come and go.
>
> Secondly we could use a single ip and use ACL to route the traffic to
> the right backend server.
> The problem with the second option is that we have around 2000 different
> subdomains and this number is still growing. So my haproxy config will
> then consists over 4000 lines of acl rules.
> and I do not know if haproxy can deal with that or if it will slowdown
> request to much.

Haproxy will happily cope with that number of ACLs, but at first
glance I don't think you need to do it that way.

Assuming you're using HTTP/S, you would probably be able to use a map,
as describe in this blog post:
https://www.haproxy.com/blog/web-application-name-to-backend-mapping-in-haproxy/

Also, assuming you're using HTTP/S, if you can relatively easily
change DNS for all the subdomains to a single IP then I would
*definitely* do that.

If you're using HTTPS, then SNI client support
(https://en.wikipedia.org/wiki/Server_Name_Indication#Support) would
be something worth checking, but as a datapoint I've not bothered
supporting non-SNI clients for several years now.

All the best,
J
--
Jonathan Matthews
London, UK
http://www.jpluscplusm.com/contact.html
Angelo Hongens
Re: haproxy without balancing
January 05, 2018 12:00PM
On 05-01-2018 11:28, Johan Hendriks wrote:
> Secondly we could use a single ip and use ACL to route the traffic to
> the right backend server.
> The problem with the second option is that we have around 2000 different
> subdomains and this number is still growing. So my haproxy config will
> then consists over 4000 lines of acl rules.
> and I do not know if haproxy can deal with that or if it will slowdown
> request to much.
>
> Maybe there are other options I did not think about?
> For me the second config is the best option because of the single IP,
> but i do not know if haproxy can handle 2000 acl rules.

I would choose the second option. I don't think the 2000 acls is a
problem. I've been running with more than that without any problems.

A single point of entry is easiest.

We run a lot of balancers with varnish+hitch+haproxy+corosync for
high-available loadbalancing. Perhaps high-availability is not a
requirement, but it's also nice to be able to do maintenance during the
day and have your standby node take over..



--

met vriendelijke groet,

Angelo Höngens
Johan Hendriks
Re: haproxy without balancing
January 05, 2018 04:30PM
Op 05/01/2018 om 11:46 schreef Jonathan Matthews:
> On 5 January 2018 at 10:28, Johan Hendriks <[email protected]> wrote:
>> BTW if this is the wrong list please excuse me.
> This looks to me like it might be the right list :-)
>
>> We have an application running over multiple servers which all have
>> there own subdomain, there are about 12 of them.
>> We can live without loadbalancing, so there is no failover, each server
>> serves a couple of subdomains.
> What protocols are these servers serving?
>
> - HTTP
> - HTTPS
> - if HTTPS, do you control the TLS certificates and their private keys?
> - Something else?
> - if something else, what?
>
All protocols are HTTP and HTTPS
>
>> At this moment every server has its own ip, and so every subdomain has a
>> different DNS entry. What we want is a single point of entry and use
>> haproxy to route traffic to the right backend server.
> Are the DNS entries for every subdomain under your control?
> How painful would it be to change one of them?
> How painful would it be to change all of them?
If we go for the one ip, then a simple wildcard would suffice.
>
>> Replacing an server is not easy at the moment. We have a lot of history
>> to deal with. We are working on it to leave that behind but till then we
>> need an solution.
>>
>> I looked at this and i think i have two options.
>> Create for each server in the backend an ip on the haproxy machine and
>> connect a frontend for that IP to the desired backend server.
>> This way we still have multiple ipadresses, but they can stay the same
>> if servers come and go.
>>
>> Secondly we could use a single ip and use ACL to route the traffic to
>> the right backend server.
>> The problem with the second option is that we have around 2000 different
>> subdomains and this number is still growing. So my haproxy config will
>> then consists over 4000 lines of acl rules.
>> and I do not know if haproxy can deal with that or if it will slowdown
>> request to much.
> Haproxy will happily cope with that number of ACLs, but at first
> glance I don't think you need to do it that way.
>
> Assuming you're using HTTP/S, you would probably be able to use a map,
> as describe in this blog post:
> https://www.haproxy.com/blog/web-application-name-to-backend-mapping-in-haproxy/
That looks like a good option indeed.
>
> Also, assuming you're using HTTP/S, if you can relatively easily
> change DNS for all the subdomains to a single IP then I would
> *definitely* do that.
>
> If you're using HTTPS, then SNI client support
> (https://en.wikipedia.org/wiki/Server_Name_Indication#Support) would
> be something worth checking, but as a datapoint I've not bothered
> supporting non-SNI clients for several years now.
>
> All the best,
> J
Thank you Jonathan Matthews and Angelo Hongens for your prompt reply's.
I now know that ACL won't be an issue and then there is mapping.

Time to start testing.
Thanks again.

Regards,
Johan
Aleksandar Lazic
Re[2]: haproxy without balancing
January 05, 2018 10:10PM
Hi Angelo.

------ Originalnachricht ------
Von: "Angelo Hongens" <[email protected]>
An: haproxy@formilux.org
Gesendet: 05.01.2018 11:49:55
Betreff: Re: haproxy without balancing

>On 05-01-2018 11:28, Johan Hendriks wrote:
>>Secondly we could use a single ip and use ACL to route the traffic to
>>the right backend server.
>>The problem with the second option is that we have around 2000
>>different
>>subdomains and this number is still growing. So my haproxy config will
>>then consists over 4000 lines of acl rules.
>>and I do not know if haproxy can deal with that or if it will slowdown
>>request to much.
>>
>>Maybe there are other options I did not think about?
>>For me the second config is the best option because of the single IP,
>>but i do not know if haproxy can handle 2000 acl rules.
>
>I would choose the second option. I don't think the 2000 acls is a
>problem. I've been running with more than that without any problems.
>
>A single point of entry is easiest.
>
>We run a lot of balancers with varnish+hitch+haproxy+corosync for
>high-available loadbalancing. Perhaps high-availability is not a
>requirement, but it's also nice to be able to do maintenance during the
>day and have your standby node take over..
Just for my curiosity why hitch and not only haproxy for ssl
termination?

>--
>
>met vriendelijke groet,
>Angelo Höngens

Regards
Aleks
Angelo Hongens
Re: haproxy without balancing
January 06, 2018 06:30PM
Hey Aleksandar,

On 05-01-2018 22:05, Aleksandar Lazic wrote:
>> We run a lot of balancers with varnish+hitch+haproxy+corosync for
>> high-available loadbalancing. Perhaps high-availability is not a
>> requirement, but it's also nice to be able to do maintenance during
>> the day and have your standby node take over..
> Just for my curiosity why hitch and not only haproxy for ssl termination?

I use varnish as a single point of entry for requests and for caching. I
guess because it's a really good product, and we've been using it for a
long time. It has some custom business logic built in our vcl as well,
and allows for a lot of http magic. I got training on varnish tuning and
monitoring, and all of our scripts revolve around varnish and its logs.
And they have very cool real-time analysis tools like varnishlog,
varnishhist, varnishstat, etc.

Varnish passes all requests to a local haproxy instance, which passes
requests to the right backends based on hostname. So we use haproxy for
balancing to backends.

When the time came we needed ssl termination, I wanted a simple solution
that does that one thing well, and I still wanted varnish as entry
point. We played around with different products (squid, nginx), but then
the varnish team forked stud and called it hitch. And the nice thing is
almost all varnish users use hitch for ssl termination, and the varnish
team is willing to offer commercial support for both.

I've been thinking about different setups as well, such as running one
haproxy instance for ssl termination, passing requests to varnish and
then pass it to another instance of haproxy that sends requests to the
backends, but I think my current setup serves us best and we use the
best tool for the jobs at hand. I think hitch is a great ssl terminator,
varnish is a great cache/spoonfeeder, and haproxy is the best balancer.


--

met vriendelijke groet,

Angelo Höngens
Aleksandar Lazic
Re[2]: haproxy without balancing
January 07, 2018 12:40AM
Hi Angelo.

------ Originalnachricht ------
Von: "Angelo Hongens" <[email protected]>
An: "Aleksandar Lazic" <[email protected]>; haproxy@formilux.org
Gesendet: 06.01.2018 18:20:47
Betreff: Re: haproxy without balancing

>Hey Aleksandar,
>
>On 05-01-2018 22:05, Aleksandar Lazic wrote:
>>>We run a lot of balancers with varnish+hitch+haproxy+corosync for
>>>high-available loadbalancing. Perhaps high-availability is not a
>>>requirement, but it's also nice to be able to do maintenance during
>>>the day and have your standby node take over..
>>Just for my curiosity why hitch and not only haproxy for ssl
>>termination?
>
>I use varnish as a single point of entry for requests and for caching.
>I guess because it's a really good product, and we've been using it for
>a long time. It has some custom business logic built in our vcl as
>well, and allows for a lot of http magic. I got training on varnish
>tuning and monitoring, and all of our scripts revolve around varnish
>and its logs. And they have very cool real-time analysis tools like
>varnishlog, varnishhist, varnishstat, etc.
>
>Varnish passes all requests to a local haproxy instance, which passes
>requests to the right backends based on hostname. So we use haproxy for
>balancing to backends.
>
>When the time came we needed ssl termination, I wanted a simple
>solution that does that one thing well, and I still wanted varnish as
>entry point. We played around with different products (squid, nginx),
>but then the varnish team forked stud and called it hitch. And the nice
>thing is almost all varnish users use hitch for ssl termination, and
>the varnish team is willing to offer commercial support for both.
>
>I've been thinking about different setups as well, such as running one
>haproxy instance for ssl termination, passing requests to varnish and
>then pass it to another instance of haproxy that sends requests to the
>backends, but I think my current setup serves us best and we use the
>best tool for the jobs at hand. I think hitch is a great ssl
>terminator, varnish is a great cache/spoonfeeder, and haproxy is the
>best balancer.
>
>--
>met vriendelijke groet,
>Angelo Höngens
Thank you very much for your detailed answer.
I fully agree with you, a specially as you have a working and supported
set-up.

It would be interesting if hitch can be replaced with haproxy without
any issues.

I plan to use haproxy in front of varnish and I would be very
appreciative for any hints, maybe off-list so that we don't upset the
haproxy list members.

Best regards
Aleks
Sorry, only registered users may post in this forum.

Click here to login