Welcome! Log In Create A New Profile

Advanced

[Feature request] Call fan-out to all endpoints.

Posted by amotz 
I found myself needing the options to do "fantout" for a call. Meaning making
1 call to haproxy and have it pass that call to *all* of the endpoint
currently active.
I don't mind implementing this myself and push to code review Is this a
feature you would be interested in ?

Thanks,
Amotz
Jonathan Matthews
Re: [Feature request] Call fan-out to all endpoints.
June 10, 2018 12:40PM
On 10 June 2018 at 08:44, amotz <[email protected]> wrote:
> I found myself needing the options to do "fantout" for a call. Meaning
> making 1 call to haproxy and have it pass that call to all of the endpoint
> currently active.
> I don't mind implementing this myself and push to code review Is this a
> feature you would be interested in ?

Hey Amotz,

I'm merely an haproxy user (not a dev and nothing to do with the
project from a feature/code/merging point of view), but I'd be
interested in using this.

I feel like an important part of it would be how you'd handle the
merge of the different server responses. I.e. the fan-in part.

I can see various merge strategies which would be useful in different
situations.

e.g. "Reply with *this* backend's response but totally ignore this
other backend's response" could be useful for in a logging/audit
scenario.

"Merge the response bodies in this defined order" could be useful for
structured data/responses being assembled.

"Merge the response bodies in any order, so long as they gave an HTTP
response code in the range of X-Y" could be useful for unstructured or
self-contained data (e.g. a catalog API).

"Merge these N distinct JSON documents into one properly formed JSON
response" could be really handy, but would obviously move haproxy's
job up the stack somewhat, and might well be an anti-feature!

I could have used all the above strategies at various points in my career.

I think all but the first strategy might well be harder to implement,
as you'll have to cater for a situation where you've received a
response but the admin's configured merging strategy dictates that you
can't serve the response to the requestor yet. You'll have to find
somewhere to cache entire individual response bodies for an amount of
time. I don't have any insight into doing that - I can just see that
it might be ... interesting :-)

If Willy and the rest of the folks who'd have to support this in the
future feel like this feature is worth it, please take this as an
enthusiastic "yes please!" from a user!

Jonathan
On Sun, Jun 10, 2018 at 12:36 PM, Jonathan Matthews <[email protected]
> wrote:

> On 10 June 2018 at 08:44, amotz <[email protected]> wrote:
> > I found myself needing the options to do "fantout" for a call. Meaning
> > making 1 call to haproxy and have it pass that call to all of the
> endpoint
> > currently active.
> > I don't mind implementing this myself and push to code review Is this a
> > feature you would be interested in ?
>
> Hey Amotz,
>
> I'm merely an haproxy user (not a dev and nothing to do with the
> project from a feature/code/merging point of view), but I'd be
> interested in using this.
>
> I feel like an important part of it would be how you'd handle the
> merge of the different server responses. I.e. the fan-in part.
>
> I can see various merge strategies which would be useful in different
> situations.
>
> e.g. "Reply with *this* backend's response but totally ignore this
> other backend's response" could be useful for in a logging/audit
> scenario.
>
> "Merge the response bodies in this defined order" could be useful for
> structured data/responses being assembled.
>
> "Merge the response bodies in any order, so long as they gave an HTTP
> response code in the range of X-Y" could be useful for unstructured or
> self-contained data (e.g. a catalog API).
>
> "Merge these N distinct JSON documents into one properly formed JSON
> response" could be really handy, but would obviously move haproxy's
> job up the stack somewhat, and might well be an anti-feature!
>
> I could have used all the above strategies at various points in my career.
>
> I think all but the first strategy might well be harder to implement,
> as you'll have to cater for a situation where you've received a
> response but the admin's configured merging strategy dictates that you
> can't serve the response to the requestor yet. You'll have to find
> somewhere to cache entire individual response bodies for an amount of
> time. I don't have any insight into doing that - I can just see that
> it might be ... interesting :-)
>
> If Willy and the rest of the folks who'd have to support this in the
> future feel like this feature is worth it, please take this as an
> enthusiastic "yes please!" from a user!
>
> Jonathan
>
>

Hi,

what's the use case?
Is this API gateway kind of thing?

Baptiste
From my experience this is mostly needed for operations/management API.
Some examples:
getStaus (i.e get the status/health from all endpoint)
flashCache (make all endpoint flash their cache)
setConfig (you get the point ...)
more...

with regard to the fan-in question by Jonathan.
Maybe return 207 (multi-status) https://httpstatuses.com/207 ?
IMO, the most intuitive response would be a json array of all the endpoints
responses, but I'm open for suggestions.

Thanks,
Amotz

‫בתאריך יום א׳, 10 ביוני 2018 ב-14:23 מאת ‪Baptiste‬‏ <‪[email protected]
‬‏>:‬

>
>
> On Sun, Jun 10, 2018 at 12:36 PM, Jonathan Matthews <
> [email protected]> wrote:
>
>> On 10 June 2018 at 08:44, amotz <[email protected]> wrote:
>> > I found myself needing the options to do "fantout" for a call. Meaning
>> > making 1 call to haproxy and have it pass that call to all of the
>> endpoint
>> > currently active.
>> > I don't mind implementing this myself and push to code review Is this a
>> > feature you would be interested in ?
>>
>> Hey Amotz,
>>
>> I'm merely an haproxy user (not a dev and nothing to do with the
>> project from a feature/code/merging point of view), but I'd be
>> interested in using this.
>>
>> I feel like an important part of it would be how you'd handle the
>> merge of the different server responses. I.e. the fan-in part.
>>
>> I can see various merge strategies which would be useful in different
>> situations.
>>
>> e.g. "Reply with *this* backend's response but totally ignore this
>> other backend's response" could be useful for in a logging/audit
>> scenario.
>>
>> "Merge the response bodies in this defined order" could be useful for
>> structured data/responses being assembled.
>>
>> "Merge the response bodies in any order, so long as they gave an HTTP
>> response code in the range of X-Y" could be useful for unstructured or
>> self-contained data (e.g. a catalog API).
>>
>> "Merge these N distinct JSON documents into one properly formed JSON
>> response" could be really handy, but would obviously move haproxy's
>> job up the stack somewhat, and might well be an anti-feature!
>>
>> I could have used all the above strategies at various points in my career.
>>
>> I think all but the first strategy might well be harder to implement,
>> as you'll have to cater for a situation where you've received a
>> response but the admin's configured merging strategy dictates that you
>> can't serve the response to the requestor yet. You'll have to find
>> somewhere to cache entire individual response bodies for an amount of
>> time. I don't have any insight into doing that - I can just see that
>> it might be ... interesting :-)
>>
>> If Willy and the rest of the folks who'd have to support this in the
>> future feel like this feature is worth it, please take this as an
>> enthusiastic "yes please!" from a user!
>>
>> Jonathan
>>
>>
>
> Hi,
>
> what's the use case?
> Is this API gateway kind of thing?
>
> Baptiste
>
Aleksandar Lazic
Re: [Feature request] Call fan-out to all endpoints.
June 10, 2018 07:40PM
Hi.

On 10/06/2018 17:56, amotz wrote:
> Baptiste wrote:
>> Hi,
>>
>> what's the use case?
>> Is this API gateway kind of thing?
>>
>> Baptiste
>
>From my experience this is mostly needed for operations/management API.
>
>Some examples:
>getStaus (i.e get the status/health from all endpoint)
>flashCache (make all endpoint flash their cache)
>setConfig (you get the point ...)
>more...
>
>with regard to the fan-in question by Jonathan.
>Maybe return 207 (multi-status) https://httpstatuses.com/207 ?
>IMO, the most intuitive response would be a json array of all the endpoints
>responses, but I'm open for suggestions.

Let's say you have a `option allendpoints /_fan-out`.

When you now call `curl -sS https://haproxy:8080/_fan-out/health` then
you will receive a json from *all active* endpoints (pods,
app-server,...) with the result of there `/health`, something like this?

That sounds interesting. Maybe it's possible with a
`external-check command <command>` or some lua code?

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#external-check%20%28Alphabetically%20sorted%20keywords%20reference%29

>Thanks,
>Amotz

Best regards
Aleks

>z
>
>‫בתאריך יום א׳, 10 ביוני 2018 ב-14:23 מאת ‪Baptiste‬‏ <‪[email protected]
>‬‏>:‬
>
>>
>>
>> On Sun, Jun 10, 2018 at 12:36 PM, Jonathan Matthews <
>> [email protected]> wrote:
>>
>>> On 10 June 2018 at 08:44, amotz <[email protected]> wrote:
>>> > I found myself needing the options to do "fantout" for a call. Meaning
>>> > making 1 call to haproxy and have it pass that call to all of the
>>> endpoint
>>> > currently active.
>>> > I don't mind implementing this myself and push to code review Is this a
>>> > feature you would be interested in ?
>>>
>>> Hey Amotz,
>>>
>>> I'm merely an haproxy user (not a dev and nothing to do with the
>>> project from a feature/code/merging point of view), but I'd be
>>> interested in using this.
>>>
>>> I feel like an important part of it would be how you'd handle the
>>> merge of the different server responses. I.e. the fan-in part.
>>>
>>> I can see various merge strategies which would be useful in different
>>> situations.
>>>
>>> e.g. "Reply with *this* backend's response but totally ignore this
>>> other backend's response" could be useful for in a logging/audit
>>> scenario.
>>>
>>> "Merge the response bodies in this defined order" could be useful for
>>> structured data/responses being assembled.
>>>
>>> "Merge the response bodies in any order, so long as they gave an HTTP
>>> response code in the range of X-Y" could be useful for unstructured or
>>> self-contained data (e.g. a catalog API).
>>>
>>> "Merge these N distinct JSON documents into one properly formed JSON
>>> response" could be really handy, but would obviously move haproxy's
>>> job up the stack somewhat, and might well be an anti-feature!
>>>
>>> I could have used all the above strategies at various points in my career.
>>>
>>> I think all but the first strategy might well be harder to implement,
>>> as you'll have to cater for a situation where you've received a
>>> response but the admin's configured merging strategy dictates that you
>>> can't serve the response to the requestor yet. You'll have to find
>>> somewhere to cache entire individual response bodies for an amount of
>>> time. I don't have any insight into doing that - I can just see that
>>> it might be ... interesting :-)
>>>
>>> If Willy and the rest of the folks who'd have to support this in the
>>> future feel like this feature is worth it, please take this as an
>>> enthusiastic "yes please!" from a user!
>>>
>>> Jonathan
>>>
>>>
>>
>>
Patrick Hemmer
Re: [Feature request] Call fan-out to all endpoints.
June 11, 2018 02:10AM
On 2018/6/10 13:27, Aleksandar Lazic wrote:
> Hi.
>
> On 10/06/2018 17:56, amotz wrote:
>> Baptiste wrote:
>>> Hi,
>>>
>>> what's the use case?
>>> Is this API gateway kind of thing?
>>>
>>> Baptiste
>>
>> From my experience this is mostly needed for operations/management API.
>>
>> Some examples:
>> getStaus (i.e get the status/health from all endpoint)
>> flashCache (make all endpoint flash their cache)
>> setConfig (you get the point ...)
>> more...
>>
>> with regard to the fan-in question by Jonathan.
>> Maybe return 207 (multi-status) https://httpstatuses.com/207 ?
>> IMO, the most intuitive response would be a json array of all the
>> endpoints
>> responses, but I'm open for suggestions.
>
> Let's say you have a `option allendpoints /_fan-out`.
>
> When you now call `curl -sS https://haproxy:8080/_fan-out/health` then
> you will receive a json from *all active* endpoints (pods,
> app-server,...) with the result of there `/health`, something like this?
>
> That sounds interesting. Maybe it's possible with a
> `external-check command <command>` or some lua code?
>
> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#external-check%20%28Alphabetically%20sorted%20keywords%20reference%29
>

Just throwing out my own $0.02, I don't think this is a good
responsibility for haproxy to support. This is very specific application
level logic.
Haproxy don't care about content types (json). What if I want to use
this feature, but with some other encoding?
How should haproxy respond if a server sends a 1GB response? It can't
buffer the whole thing in memory so it can encode it and add it to the
response message.
What about the non-happy-path cases? What if one of the servers times
out, what should haproxy put in the response? What if a server sends a
partial response?
How should the headers from a server response be encoded?

This is basically the invention of a new protocol.

Don't get me wrong, the underlying goal, having a client send a single
request and that request getting duplicated amongst the servers, is a
good one. In fact we do this at my work. But we use a custom application
that is specifically designed to handle the protocol we are wrapping.

I think this might be reasonable to do in LUA, and maybe even possible
already, but there's still going to be lots of the fore-mentioned
difficulties.
However to put some measure of positive spin on things, I think HTTP/2
would fit very well with this use case. HTTP/2 supports server push
messages. Meaning it's built in to the protocol that the client can send
a single request, and receive multiple responses. Haproxy doesn't
support fully H2 passthrough right now, but that may not be necessary. I
think LUA really only needs a few things to be able to support this: The
ability to receive H2 requests & generate responses (LUA already has
http/1.1 response capabilities, but I have no idea if they work with H2
requests), and then the ability to trigger a request to a server, and
have that sent back to the client as a server-push message.

-Patrick
Sorry, only registered users may post in this forum.

Click here to login