Welcome! Log In Create A New Profile

Advanced

Switch from http mode to tcp mode at will

Posted by Hoggins! 
Hoggins!
Switch from http mode to tcp mode at will
April 30, 2018 04:20PM
Hello list,

I have a case where I would like HAProxy to "get out of the way" if a
certain pattern is detected in a HTTP request. In that particular case,
I would like to select a TCP backend over an HTTP one if the conditions
are met.
But I'm not sure if it's feasible or even if it's the correct practice
for my case.

Thanks !

    Hoggins!
Aleksandar Lazic
Re: Switch from http mode to tcp mode at will
April 30, 2018 04:40PM
Hi.

Am 30.04.2018 um 16:11 schrieb Hoggins!:
> Hello list,
>
> I have a case where I would like HAProxy to "get out of the way" if a
> certain pattern is detected in a HTTP request. In that particular case,
> I would like to select a TCP backend over an HTTP one if the conditions
> are met.
> But I'm not sure if it's feasible or even if it's the correct practice
> for my case.

Something similar like websockets?

https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/

> Thanks !
>
>     Hoggins!

Regards
Aleks
Hoggins!
Re: Switch from http mode to tcp mode at will
April 30, 2018 05:00PM
Hi,

Le 30/04/2018 à 16:35, Aleksandar Lazic a écrit :
> Hi.
>
> Am 30.04.2018 um 16:11 schrieb Hoggins!:
>> Hello list,
>>
>> I have a case where I would like HAProxy to "get out of the way" if a
>> certain pattern is detected in a HTTP request. In that particular case,
>> I would like to select a TCP backend over an HTTP one if the conditions
>> are met.
>> But I'm not sure if it's feasible or even if it's the correct practice
>> for my case.
> Something similar like websockets?
>
> https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/

I've tried that, actually, but HAProxy won't let me specify "mode tcp"
in the selected backend, as I apparently can't mix a "mode http" in the
frontend (or in globals) and a "mode tcp" in a backend selected by
"use_backend". That would have been my best shot, actually.

I think I'm missing something else in my whole setup, some things are
not clear, thanks for the help anyway !

>
>> Thanks !
>>
>>     Hoggins!
> Regards
> Aleks
>
>
>
Aleksandar Lazic
Re: Switch from http mode to tcp mode at will
April 30, 2018 05:30PM
Hi Hoggins.

It would help when you share some more informations.

haproxy -vv
Your config.

Best regards
Aleks


-------- Ursprüngliche Nachricht --------
Von: Hoggins! <[email protected]>
Gesendet: 30. April 2018 16:53:58 MESZ
An: haproxy@formilux.org
Betreff: Re: Switch from http mode to tcp mode at will

Hi,

Le 30/04/2018 à 16:35, Aleksandar Lazic a écrit :
> Hi.
>
> Am 30.04.2018 um 16:11 schrieb Hoggins!:
>> Hello list,
>>
>> I have a case where I would like HAProxy to "get out of the way" if a
>> certain pattern is detected in a HTTP request. In that particular case,
>> I would like to select a TCP backend over an HTTP one if the conditions
>> are met.
>> But I'm not sure if it's feasible or even if it's the correct practice
>> for my case.
> Something similar like websockets?
>
> https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/

I've tried that, actually, but HAProxy won't let me specify "mode tcp"
in the selected backend, as I apparently can't mix a "mode http" in the
frontend (or in globals) and a "mode tcp" in a backend selected by
"use_backend". That would have been my best shot, actually.

I think I'm missing something else in my whole setup, some things are
not clear, thanks for the help anyway !

>
>> Thanks !
>>
>>     Hoggins!
> Regards
> Aleks
>
>
>
Willy Tarreau
Re: Switch from http mode to tcp mode at will
April 30, 2018 08:20PM
Hi,

On Mon, Apr 30, 2018 at 04:53:58PM +0200, Hoggins! wrote:
> Hi,
>
> Le 30/04/2018 à 16:35, Aleksandar Lazic a écrit :
> > Hi.
> >
> > Am 30.04.2018 um 16:11 schrieb Hoggins!:
> >> Hello list,
> >>
> >> I have a case where I would like HAProxy to "get out of the way" if a
> >> certain pattern is detected in a HTTP request. In that particular case,
> >> I would like to select a TCP backend over an HTTP one if the conditions
> >> are met.
> >> But I'm not sure if it's feasible or even if it's the correct practice
> >> for my case.
> > Something similar like websockets?
> >
> > https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/
>
> I've tried that, actually, but HAProxy won't let me specify "mode tcp"
> in the selected backend, as I apparently can't mix a "mode http" in the
> frontend (or in globals) and a "mode tcp" in a backend selected by
> "use_backend". That would have been my best shot, actually.
>
> I think I'm missing something else in my whole setup, some things are
> not clear, thanks for the help anyway !

In fact you're probably mixing several things here. First, blindly
switching from HTTP to TCP doesn't make much sense as the conditions
would be based on an already prepared HTTP context that you can't
destroy without first processing it.

However, HTTP already offers provisions for this :
- establishing a tunnel via the CONNECT method : a client sends a CONNECT
request, the server responds "200 OK" and then the connection becomes a
fully bi-directional raw TCP connection.

- upgrading the current connection (same as the upgrade mechanism used
by TLS in RFC2817, WebSocket in RFC6455 and HTTP/2 in RFC7540) : the
client sends a request, adds Upgrade:<protocol_name> and Connection:
upgrade. If the server responds "101 switching protocols", then the
connection is upgraded to the protocol agreed between the client and
the server. In practice for most gateways (like haproxy), it simply
means switching to a plain TCP connection between the two ends so
nothing blocks whatever protocol the client and server want to talk.

These are the mechanisms offered by the standards, and they must definitely
be used whenever possible, for example if you're developing an application
which wants to make HTTP and your protocol coexist.

In additionto this haproxy can also detect HTTP protocol elements while
running in TCP mode. It requires to enable protocol inspection and it's
not always trivial, but it's sometimes quite convenient. And it allows
a TCP frontend to switch to an HTTP backend for HTTP requests, or to a
TCP backend for TCP requests. A typical use case is to distinguish HTTP
from HTTPS. You can for example proceed approximately like this (check
the doc to ensure I didn't mess up with some arguments but you should
get the overall idea) :

frontend public
bind :80
tcp-request inspect-delay 10s #10s max for an HTTP request to come
tcp-request content-accept if HTTP
use_backend http if HTTP
default_backend tcp

backend http
mode http
...
server srv1 1.1.1.1:80

backend tcp
mode tcp
...
server srv1 1.1.1.1:443

There are a few limitations which are often acceptable in this context,
such as the fact that you cannot log HTTP elements in the frontend. But
such mechanisms can be used to distinguish multiple protocols over a
single port. Only banner protocols (those where the server talks first)
cannot be distinguished but can sometimes be deduced after a timeout.

Hoping this helps,
Willy
Hoggins!
Re: Switch from http mode to tcp mode at will
May 09, 2018 03:50PM
Thanks !

That makes a lot of sense.

I was trying to find some more info because I was not sure what was
actually happening.
The idea was to use HAProxy as a frontend for some Icecast HTTP
streaming. I'm already using it as an SSL offloader, and it works like a
charm : streaming clients connect through HAProxy on the streaming
servers seamlessly.

I wanted to go another level, and make the streaming source(s) to also
go through HAProxy, because it was more convenient for me to have a
single (even if it is replicated) point of entry for that.
But when I made some tests, the streaming source would connect, send a
few bytes, get an HTTP 200 OK response, and stop streaming with an error.

My thought was that HAProxy, while in HTTP mode, was sending an HTTP 200
OK too early and make the source stop from streaming, so that's why I
was looking into having HAProxy "detecting" the HTTP method, and act in
consequence : if the request contained a PUT or a SOURCE method (as per
the Icecast specs), then it would passthough the traffic without
interfering, switching to TCP mode.

I "managed" somehow to have this behavior, but without the switching,
simply by listening on another port, having the frontend and backend
both in TCP mode.

So thank you for this helpful explanation on the guts of the system, I
believe I'll stick to what I did so far !

    Hoggins!

Le 30/04/2018 à 20:10, Willy Tarreau a écrit :
> Hi,
>
> On Mon, Apr 30, 2018 at 04:53:58PM +0200, Hoggins! wrote:
>> Hi,
>>
>> Le 30/04/2018 à 16:35, Aleksandar Lazic a écrit :
>>> Hi.
>>>
>>> Am 30.04.2018 um 16:11 schrieb Hoggins!:
>>>> Hello list,
>>>>
>>>> I have a case where I would like HAProxy to "get out of the way" if a
>>>> certain pattern is detected in a HTTP request. In that particular case,
>>>> I would like to select a TCP backend over an HTTP one if the conditions
>>>> are met.
>>>> But I'm not sure if it's feasible or even if it's the correct practice
>>>> for my case.
>>> Something similar like websockets?
>>>
>>> https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/
>> I've tried that, actually, but HAProxy won't let me specify "mode tcp"
>> in the selected backend, as I apparently can't mix a "mode http" in the
>> frontend (or in globals) and a "mode tcp" in a backend selected by
>> "use_backend". That would have been my best shot, actually.
>>
>> I think I'm missing something else in my whole setup, some things are
>> not clear, thanks for the help anyway !
> In fact you're probably mixing several things here. First, blindly
> switching from HTTP to TCP doesn't make much sense as the conditions
> would be based on an already prepared HTTP context that you can't
> destroy without first processing it.
>
> However, HTTP already offers provisions for this :
> - establishing a tunnel via the CONNECT method : a client sends a CONNECT
> request, the server responds "200 OK" and then the connection becomes a
> fully bi-directional raw TCP connection.
>
> - upgrading the current connection (same as the upgrade mechanism used
> by TLS in RFC2817, WebSocket in RFC6455 and HTTP/2 in RFC7540) : the
> client sends a request, adds Upgrade:<protocol_name> and Connection:
> upgrade. If the server responds "101 switching protocols", then the
> connection is upgraded to the protocol agreed between the client and
> the server. In practice for most gateways (like haproxy), it simply
> means switching to a plain TCP connection between the two ends so
> nothing blocks whatever protocol the client and server want to talk.
>
> These are the mechanisms offered by the standards, and they must definitely
> be used whenever possible, for example if you're developing an application
> which wants to make HTTP and your protocol coexist.
>
> In additionto this haproxy can also detect HTTP protocol elements while
> running in TCP mode. It requires to enable protocol inspection and it's
> not always trivial, but it's sometimes quite convenient. And it allows
> a TCP frontend to switch to an HTTP backend for HTTP requests, or to a
> TCP backend for TCP requests. A typical use case is to distinguish HTTP
> from HTTPS. You can for example proceed approximately like this (check
> the doc to ensure I didn't mess up with some arguments but you should
> get the overall idea) :
>
> frontend public
> bind :80
> tcp-request inspect-delay 10s #10s max for an HTTP request to come
> tcp-request content-accept if HTTP
> use_backend http if HTTP
> default_backend tcp
>
> backend http
> mode http
> ...
> server srv1 1.1.1.1:80
>
> backend tcp
> mode tcp
> ...
> server srv1 1.1.1.1:443
>
> There are a few limitations which are often acceptable in this context,
> such as the fact that you cannot log HTTP elements in the frontend. But
> such mechanisms can be used to distinguish multiple protocols over a
> single port. Only banner protocols (those where the server talks first)
> cannot be distinguished but can sometimes be deduced after a timeout.
>
> Hoping this helps,
> Willy
>
Sorry, only registered users may post in this forum.

Click here to login