Welcome! Log In Create A New Profile

Advanced

haproxy 1.9 status update

Posted by Willy Tarreau 
Willy Tarreau
haproxy 1.9 status update
May 25, 2018 06:20PM
Hi all,

I've long wanted to send a status update on where haproxy 1.9 is going,
and seeing some recent threads speculating about what will be available
reminded me that it's really time to send this update. Be careful, this
e-mail is long.

So first of all, while we didn't have *that* many bugs on 1.8, the ones
we received lately were really painful to work on. As was expected, most
of them came from the new core stuff, namely H2, threads, cache, master-
worker, as well as a few existing parts having moved a lot like SSL and
Lua.

Fixing them kept a number of developers quite busy scratching their heads
much longer than initially anticipated, to the point that the core parts
that needed to be worked on had little opportunities to make any acceptable
progress.

Fortunately, thanks to the amazing help that was provided here on the list
and on Discourse sorting out PEBKAC from bugs, we're progressively recovering
and getting back to development. The income of new contributions is also a
positive sign of this progressive restart.

So where are we now ? The current features were already merged :
- the fd_cache was made lockless, we've seen a ~40% performance gain on a
12-core machine when using threads [Olivier] ;

- the CLI now supports a payload, which will be used to update certs on
the fly [Aurélien] ;

- SPOE was significantly updated to provide a smoother load balancing with
pipeline-aware agents, allowing them to scale even better [Christopher]


A number of features have already been submitted at least once, are being
reviewed and/or discussed and are expected to get merged once everything is
considered OK :
- certificate update on the CLI [Aurélien] : from what I understand, some
more discussion is needed to figure how to reliably identify a certificate.

- make a "resolvers" section able to use /etc/resolv.conf [Ben]. Baptiste
is going to take a look at this very soon, I think we should be good
soon on this one.

- queue priorities manipulation [Patrick] : Patrick already sent his patch
set, he's just waiting for me to take a look. A quick look at it makes
me think we're about to be good as well, there will likely be very few
round trips.

- alert if some header addition cannot be performed [Tim] : it's not exactly
a feature but I suspect that we'll want to iterate over it to further
refine the reported information in case of errors, and it's really needed.
I need to take a look at it, still couldn't find time for this.

- scheduler update [Olivier] : the scheduler has been reworked to become
more thread-aware, to improve the inter-thread communication, and to
improve fairness between tasks. The first step consisted in changing
the internal API a bit. I've just received the patch set for review, it
should be merged soon. The second step removes a lot of locking and
will make the scheduler scale much better with many threads. It's also
part of the patch set I have to review.

- peers over SSL [Fred] : the purpose is to allow all bind options with
peers so that peers can exchange information securely. I think I've
seen it posted somewhere, I'll have to dig through the archives.


A number of other things were started but not submitted yet :
- applet/connection scheduler [Olivier] : the next step will consist in
using regular tasks to schedule applets and prevent them from clogging
the CPU. The last step will consist in using miniature sub-tasks called
"tasklets" to also schedule I/Os. This is needed to sort out the trouble
we have with the MUX architecture right now where it's impossible to just
wake a mux up without pretending to have received a fake I/O.

- analyzer timing [me] : in order to constantly improve haproxy's
observability, I want to add the ability to measure the CPU time taken
in each task, the latency added by other tasks, and the time spent in I/O.
It becomes really important with potentially heavy Lua scripts or huge
certificates which cause large delays affecting all other tasks. Now it
will be possible to see the culprits and the victims. A next step will
consist in adding processing timeouts to alert about, or kill offenders.

- buffer rework [me] : we've reached the point where the current buffer
internal API is really annoying and forces us to deploy more efforts to
work around it than to update it. The details of the changes have been
put into a new doc (buffers-rework.txt). It should lead to quite some
code removal. The changes will affect a lot of places, will be 90%
mechanical and 10% tricky. Over the long term it will allow to merge
chunks and buffers and remove buffer allocations at many places, as
well as permit to send error files that are larger than a buffer.

- native HTTP representation allowing end-to-end HTTP/2 [Christopher] :
this will allow HTTP/2 to be used end-to-end. For now Chris is working
on the "easy" part which is the HTTP/1 parsing/conversion. I'm putting
quotes around "easy" because while HTTP/1 appears easy to deal with,
that's where the highest number of obstacles are present due to the fact
that HTTP/1 is currently used as the native internal representation (H2
is currently translated to H1). This will heavily rely on the mux changes.
We know that some parts ("tcp-request content" L7-based rules, option
http-send-name-header, and filters/compression) will cause some head
scratching. The expected gains from this part are tremendous. But while
I thought that H2 and threads used to be the most complex changes over
the last decade, I think this one's complexity has already surpassed
them both. I'd say the chances of success for 1.9 are around 70%, which
is not much but which still gives us the motivation for pursuing the
efforts. Once completed, Christopher will deserve a barrel of beer to
forget about this painful task ;-)

- master/worker improvements [William] : William is currently working on
making the master process much smarter and able to really manage the
workers. It will be possible to have a CLI on the master to consult the
list of workers for example, and maybe later even bounce between CLIs
from the master. I'm unclear about all the details but when he explains
to me it sounds really cool.

- http-request "do-resolve" action [Baptiste] : the purpose is to be able
to perform DNS resolutions on the fly (ie read a name from a variable,
set the IP into another one). Since converters cannot yield, Baptiste
implemented this as an action. I don't remember if there's something
equivalent planned nor possible in Lua.


Not started yet but already planned :
- split certificates [William] : some users prefer to have separate .key
and .crt files (eg for permission reasons). William has already started
to look and it seems doable, there are "just" a few shortcomings to take
care of (I don't remember which ones).

- certificate merge on startup [William] : the purpose is to save a lot of
RAM usage by trying to load certificates only once even when they appear
on multiple bind lines. Some more discussion is required on this.

- mux rework [Chris/Olivier/me?] : the current mux design is painful as we
have to cross various levels multiple times and duplicate some application
level code. A rework attempt was made consisting in chaining layers in a
more natural way which relies on breaking a very old assumption, which is
that send() is only performed upon solliciation from I/O callbacks instead
of from the upper layers. We do have some experimental code for this and
the end-to-end HTTP code will heavily rely on this. Currently a big part
of the difficulties lie in the connection scheduling and buffers API
limitations, addressed in the steps above.

- mux upgrade [Chris/Olivier/me?] : Christopher has identified that we'll
likely need that a mux can be upgraded on the fly (eg: switch from a TCP
frontend to an HTTP backend). This is another necessary pain inflicted by
the HTTP mux design which will offer many new perspectives in the future.

- small improvements to the cache [William] : I don't remember exactly what,
I have some memories about removing the single-buffer size limitation and
possibly starting to take Vary into account in some cases.

- SSL layer splitting between FD level and buffer level [Olivier] : this will
be required for QUIC so that we can feed buffers containing SSL traffic to
OpenSSL. For now the SSL layer relies on OpenSSL's native read()/write()
calls (yes, the ones that force you to disable SIGPIPE in gdb).


Some ideas to study later, nothing really assigned :
- certificates : study if it makes sense to parse them on the fly on first
use to also save on startup time.

- logs: some improvements to the logging system have been discussed about
a few times, which can be summarized like this :
- some people would like a set of log "profiles" that can be reused
everywhere instead of copy-pasting the same log-format lines.
- some people really need a per-server log-format because they send,
say, native logs locally and JSON logs to another server.
- some people want to load-balance between log servers
- other people want to sample logs without having to trick the config
=> all of this has to be addressed together.

- the "return" directive, regularly talked about, may possible be done
for 1.9.


It's possible I accidently missed someone's tasks, if so, please yell now.
I'd like that any development outside the scope of what is enumerated above
and not started yet is not submitted for now, because discussing designs and
reviewing code takes a huge amount of time and concentration that inevitably
postpones code progress on planned stuff.

If some people want to start to work on significant changes, I'd prefer they
do it in their own branches. We can possibly create a -next branch to merge
such pending stuff once reviewed and accepted. If some are interested in
participating to the "not started but planned" part because they already
have things available, or want to take care of stuff not listed there,
please discuss this here so that we can all decide together. Just be patient
about the response :-)

I think I'll soon issue a -dev1 after we merge most of the pending stuff so
that at least those who contributed it can use it in a tagged version, and
can ask for testers around them. If I forget, ping me!

For the next steps given that we've spent a lot of time dealing with bugs and
painful patches, I'll be a bit more demanding regarding changes. While the
vast majority of regular contributors have made an awesome progress in the
quality of their patches, we're still spending some time reviewing huge
patches that should be broken up into smaller pieces, or re-editing commit
messages to avoid ruining the life of the -stable team. Sometimes patches
submitted for review stay pending for a long time just because of irritating
things that will require more processing time than needed (usually it's from
first-time contributors). Thus please ensure you've *really* read CONTRIBUTING
at least once in your life before submitting patches, even if you're used to
submitting them. Feel free to disagree with it and to propose patches against
it. At least for those who've read it, patches have always been much easier
handled and were merged much faster.

If you don't get a response for one week after submitting a patch, complain
loudly! If you're waiting for a maintainer who doesn't respond, complain as
well! We're all humans and we all miss stuff, and you cannot expect that
someone will suddenly rediscover your submission in the middle of 20000
other messages 1 month later. For maintainers, if you cannot respond in time
because you're busy, please at least respond "ping me again in 5 days" or
so.

Last point, as some of you might have noticed, we finally recovered the
haproxy GitHub account (thanks to Jeff for this as well as Dan, Lukas and
Joe for helping coordinate the operations). A new haproxy repository was
placed there only with mainline for now. By the way if you're having a
fork from more than one month ago, chances are that you accidently forked
the wrong one ;-)

We do have a few plans with this account now, some of which seem appealing,
and others who don't sound really compatible with some annoying GitHub
limitations, making me think that maybe in the end we should switch to
Gitlab (William already created haproxy.org there) :

- re-enable issues : we still don't have a public bug tracker/todo list
and it's a pain for everyone. It would be nice to show what's pending
or being worked on, and to sometimes add extra info regarding a given
task. The problem we've faced in the past with GitHub's issue tracker
is that it's impossible to restrict the creation of new issues to
participants. Given that haproxy is a complex component, the boundary
between human error, misunderstanding and bug is extremely thin. It
resulted in the issue tracker being filled with wrong bugs, 100% of
which were in fact requests for help. It makes the utility totally
useless for development and bug fixing as it requires more time to
maintain in a clean state than it takes to put issues in a mailbox.
Some people suggested using issue templates, but I hardly see how this
will help, at best it will annoy regular participants, at worst it will
not stop polluters from filling crap there. What I'd ideally like would
be that about any participant here on the list (i.e. those who help
others) can create an issue, and that anyone could see open issues and
add info to existing issues.

The closest we've found to this is Gitlab, but when issue creation is
limited to contributors, only these ones can watch them. I tend to
consider that once many participants here are contributors, the tracker
becomes almost public thus this limitation is not a big deal, but it's
still annoying.

A limitation that isn't addressed by any of them is that an issue has
a single status and not one per maintenance branch. Some will say that
labels replace this but I'd say that this isn't true, it just vaguely
emulates this. Anyway if we don't have better we can go with this. I
often dream of the day someone pissed of by using prehistoric issue
trackers writes a modern one, just like when Linus created Git...

- wiki : we all know that the architecture guide is obsolete, everyone
wants to refresh it and nobody can because it's a tedious task that
no single person can address, and nobody anymore knows all haproxy
pieces. In addition a lot of participants have useful tips to share
and would be much more at ease with putting this into a wiki than by
editing architecture.txt. Thus I'd like to reuse either Github or
Gitlab's wiki to place real contents edited and maintained by the
community, and to kill this obsolete file. This way the obsolete stuff
on the main haproxy.org page could go as well.

- automated builds and CI : gitlab and github have these possibilities
(either natively or relying on third party products). It would help
us detect the occasional stupid build breakage (like building with/
without threads, with/without SSL).

- we could imagine reusing self-hosted pages to put Cyril's generated
docs, or to link there if it's easier. Similar for Thierry's Lua docs,
it will first depend on what these ones prefer (I don't want to steal
your work guys, don't worry, just suggesting that it could be done if
you find it desirable).

- maybe placing the releases there will help (I don't know the upload
process for any of these, will have to check closer). It may even be
used for Aleks' docker images if that makes sense at all (I don't
know).


Feel free to discuss these points here on the list as usual. Please don't
PM me, I don't have time to rehash the same points with multiple persons.
If I missed anything important that you care about, say it now.

thanks,
Willy
Tim Düsterhus
Re: haproxy 1.9 status update
May 25, 2018 07:50PM
Willy,

Am 25.05.2018 um 18:10 schrieb Willy Tarreau:
> Not started yet but already planned :
> - split certificates [William] : some users prefer to have separate .key
> and .crt files (eg for permission reasons). William has already started
> to look and it seems doable, there are "just" a few shortcomings to take
> care of (I don't remember which ones).

This one would be great, as it would save me a bash script to
concatenate the output of certbot into something haproxy understands.

> It's possible I accidently missed someone's tasks, if so, please yell now.
> I'd like that any development outside the scope of what is enumerated above
> and not started yet is not submitted for now, because discussing designs and
> reviewing code takes a huge amount of time and concentration that inevitably
> postpones code progress on planned stuff.

One pain point I am missing from the list is "(add|set)-header" for
redirect responses (or is this part of something else?). We discussed
this briefly in March:

Subject: Re: add header into http-request redirect
Message-ID: <[email protected]>
https://www.mail-archive.com/[email protected]/msg29294.html

IMO this is a great example of why an issue tracker is superior to a
mailing list (I'll further get to that below): Things do not get lost in
the stream of other discussion; people can easily re-use the existing
discussion, without fishing for Message-IDs in the mail archive. You
acknowledged that in:

https://www.mail-archive.com/[email protected]/msg29358.html
Message-ID: <[email protected]>

> For the next steps given that we've spent a lot of time dealing with bugs and
> painful patches, I'll be a bit more demanding regarding changes. While the
> vast majority of regular contributors have made an awesome progress in the
> quality of their patches, we're still spending some time reviewing huge
> patches that should be broken up into smaller pieces, or re-editing commit
> messages to avoid ruining the life of the -stable team. Sometimes patches

Personally I really want to add my 2ct to some of the patches, but
what's preventing me from doing so is:

1. My email client makes it hard to comment on specific lines in
attachments. That's why I personally send my patches inline (!) using
git send-email / git format-patch.
2. I don't feel like bombarding all the list subscribers with a mail
complaining about typos in commit messages or code comments.
3. The annoying autoresponders of misconfigured Exchange mail servers,
informing me that some random employee of random company is currently
out of office. Every other time I send an email to the list I get
several of those messages back.

I grew up with GitHub and systemd, you don't like those, I very much
like both of them. And for me the barrier to participation would be much
lower using GitHub's Pull Requests.

Yes, Pull Requests encourage drive by committers more than now, but OTOH
you don't know what you might miss out on, both on good patches and on
good review. I know that CONTRIBUTING explicitly mentions pull requests
and merge commits as "bad", but GitHub nowadays supports rebased merges
directly from the web interface, thus avoiding merge commits.

I regularly contribute to various Open Source projects I use using
GitHub pull requests, it's much easier for me to fork the project,
skimming the CONTRIBUTING file and sending a (IMO) high-quality PR that
usually gets merged without much discussion. Often the whole turnaround
time is less than half a day.
For haproxy I would need to subscribe to the list, configure my email
client properly so that my patches don't get mangled, set up filters for
my mail so I don't miss out the review etc.

> We do have a few plans with this account now, some of which seem appealing,
> and others who don't sound really compatible with some annoying GitHub
> limitations, making me think that maybe in the end we should switch to
> Gitlab (William already created haproxy.org there) :

Personal opinion: Use GitHub for the ecosystem. For me GitLab would be
an even greater pain than the mailing list.

> - re-enable issues : we still don't have a public bug tracker/todo list
> and it's a pain for everyone. It would be nice to show what's pending
> or being worked on, and to sometimes add extra info regarding a given

Very much yes (see above).

> task. The problem we've faced in the past with GitHub's issue tracker
> is that it's impossible to restrict the creation of new issues to
> participants. Given that haproxy is a complex component, the boundary
> between human error, misunderstanding and bug is extremely thin. It
> resulted in the issue tracker being filled with wrong bugs, 100% of
> which were in fact requests for help. It makes the utility totally
> useless for development and bug fixing as it requires more time to
> maintain in a clean state than it takes to put issues in a mailbox.

I believe that an open issue tracker results it a net-positive overall
for the following reasons:

1. Issues can easily be triaged by volunteers (such as Lukas already
does on Discourse), be properly labeled and closed if invalid. This does
not take away time from the developers I believe. In fact it might be
even easier, because the developer can see at a glance that the issue is
closed and he must not read through a long mailing list thread.
2. Again: Barrier to entry. I outlined it above for pull requests, but
it does apply to issues as well. Projects miss out on valuable bug
reports for edge cases, because people don't bother subscribing to a
mailing list.

> Some people suggested using issue templates, but I hardly see how this
> will help, at best it will annoy regular participants, at worst it will

Templates are not mandatory on GitHub, experienced reporters can opt to
simply ignore them.

> not stop polluters from filling crap there.

See above why I believe an open issue tracker is a good thing.

> A limitation that isn't addressed by any of them is that an issue has
> a single status and not one per maintenance branch. Some will say that
> labels replace this but I'd say that this isn't true, it just vaguely
> emulates this. Anyway if we don't have better we can go with this. I
> often dream of the day someone pissed of by using prehistoric issue
> trackers writes a modern one, just like when Linus created Git...

As bugs will be fixed in the development branch first the status would
refer to that. The bugfix will naturally appear in the maintenance
branches in the future.

> - wiki : we all know that the architecture guide is obsolete, everyone
> wants to refresh it and nobody can because it's a tedious task that
> no single person can address, and nobody anymore knows all haproxy
> pieces. In addition a lot of participants have useful tips to share
> and would be much more at ease with putting this into a wiki than by
> editing architecture.txt. Thus I'd like to reuse either Github or
> Gitlab's wiki to place real contents edited and maintained by the
> community, and to kill this obsolete file. This way the obsolete stuff
> on the main haproxy.org page could go as well.

In my experience: GitHub / GitLab pages are far superior to the Wiki and
can be handled with Pull Requests as well. Regular contributors could be
given write access to the "guide repositories".

> - automated builds and CI : gitlab and github have these possibilities
> (either natively or relying on third party products). It would help
> us detect the occasional stupid build breakage (like building with/
> without threads, with/without SSL).

I agree this is a good thing.

> - maybe placing the releases there will help (I don't know the upload
> process for any of these, will have to check closer). It may even be
> used for Aleks' docker images if that makes sense at all (I don't
> know).

GitHub: You are able to attach downloads to git tags. This probably
would imply having all the maintenance repositories as separate branches
in a single repository (i.e. not master in haproxy-1.8.git, but 1.8.x in
haproxy.git).

Best regards
Tim Düsterhus
Frederic Lecaille
Re: haproxy 1.9 status update
May 28, 2018 09:20AM
On 05/25/2018 06:10 PM, Willy Tarreau wrote:
> Hi all,

[sniped]

> - peers over SSL [Fred] : the purpose is to allow all bind options with
> peers so that peers can exchange information securely. I think I've
> seen it posted somewhere, I'll have to dig through the archives.

Here are the rebased patches for this feature.

Fred.
Aleksandar Lazic
Re: haproxy 1.9 status update
June 02, 2018 02:10AM
Hi Willy.

Finally I found the time to write a answer.

On 25/05/2018 18:10, Willy Tarreau wrote:
> Hi all,
>
> I've long wanted to send a status update on where haproxy 1.9 is
> going, and seeing some recent threads speculating about what will be
> available reminded me that it's really time to send this update. Be
> careful, this e-mail is long.

8-O yes but very informative. Thanks for writing.

[snipp]

> - make a "resolvers" section able to use /etc/resolv.conf [Ben]. Baptiste
> is going to take a look at this very soon, I think we should be good
> soon on this one.

Are there any plans for DoH (= https://github.com/curl/curl/wiki/DNS-over-HTTPS)

[snipp]

> - native HTTP representation allowing end-to-end HTTP/2
> [Christopher] : this will allow HTTP/2 to be used end-to-end. For
> now Chris is working on the "easy" part which is the HTTP/1
> parsing/conversion. I'm putting quotes around "easy" because while
> HTTP/1 appears easy to deal with, that's where the highest number
> of obstacles are present due to the fact that HTTP/1 is currently
> used as the native internal representation (H2 is currently
> translated to H1). This will heavily rely on the mux changes. We
> know that some parts ("tcp-request content" L7-based rules, option
> http-send-name-header, and filters/compression) will cause some
> head scratching. The expected gains from this part are
> tremendous. But while I thought that H2 and threads used to be the
> most complex changes over the last decade, I think this one's
> complexity has already surpassed them both. I'd say the chances of
> success for 1.9 are around 70%, which is not much but which still
> gives us the motivation for pursuing the efforts. Once completed,
> Christopher will deserve a barrel of beer to forget about this
> painful task ;-)

Full ack.

That's one part for the gRPC, as Daniel mentioned in a previous mail ;-)

Will this be a 'plugin'?

The idea behind this question is to add another protocol plugin like MQTT
or other protocols

[snipp]

> - SSL layer splitting between FD level and buffer level [Olivier] :
> this will be required for QUIC so that we can feed buffers
> containing SSL traffic to OpenSSL. For now the SSL layer relies on
> OpenSSL's native read()/write()
> calls (yes, the ones that force you to disable SIGPIPE in gdb).

This is one question I wanted to ask 'what's the plan for QIUC is.'
Sounds like it's on the roadmap.

> Some ideas to study later, nothing really assigned :
> - certificates : study if it makes sense to parse them on the fly on
> first use to also save on startup time.
>
> - logs: some improvements to the logging system have been discussed
> about a few times, which can be summarized like this :
> - some people would like a set of log "profiles" that can be
> reused everywhere instead of copy-pasting the same log-format
> lines.
> - some people really need a per-server log-format because they
> send, say, native logs locally and JSON logs to another
> server.
> - some people want to load-balance between log servers
> - other people want to sample logs without having to trick the
> config
> => all of this has to be addressed together.

* How about to let rsyslog handle this. There are queues and pools and
ratelimits also are there some recipes how to create json logs.

or

* How about to build a dedicated worker for this like SPOE which handles
the specific logging requirement?

Jm2C

> - the "return" directive, regularly talked about, may possible be done
> for 1.9.

8-O What's this?

[snipp]

> Last point, as some of you might have noticed, we finally recovered
> the haproxy GitHub account (thanks to Jeff for this as well as Dan,
> Lukas and Joe for helping coordinate the operations). A new haproxy
> repository was placed there only with mainline for now. By the way if
> you're having a fork from more than one month ago, chances are that
> you accidently forked the wrong one ;-)
>
> We do have a few plans with this account now, some of which seem
> appealing, and others who don't sound really compatible with some
> annoying GitHub limitations, making me think that maybe in the end we
> should switch to Gitlab (William already created haproxy.org there) :

OH I like gitlab much more then github ;-)

> - re-enable issues : we still don't have a public bug tracker/todo
> list and it's a pain for everyone. It would be nice to show what's
> pending or being worked on, and to sometimes add extra info
> regarding a given task. The problem we've faced in the past with
> GitHub's issue tracker is that it's impossible to restrict the
> creation of new issues to participants. Given that haproxy is a
> complex component, the boundary between human error,
> misunderstanding and bug is extremely thin. It resulted in the
> issue tracker being filled with wrong bugs, 100% of which were in
> fact requests for help. It makes the utility totally useless for
> development and bug fixing as it requires more time to maintain in
> a clean state than it takes to put issues in a mailbox. Some
> people suggested using issue templates, but I hardly see how this
> will help, at best it will annoy regular participants, at worst it
> will not stop polluters from filling crap there. What I'd ideally
> like would be that about any participant here on the list
> (i.e. those who help others) can create an issue, and that anyone
> could see open issues and add info to existing issues.
>
> The closest we've found to this is Gitlab, but when issue creation
> is limited to contributors, only these ones can watch them. I tend
> to consider that once many participants here are contributors, the
> tracker becomes almost public thus this limitation is not a big
> deal, but it's still annoying.
>
> A limitation that isn't addressed by any of them is that an issue
> has a single status and not one per maintenance branch. Some will
> say that labels replace this but I'd say that this isn't true, it
> just vaguely emulates this. Anyway if we don't have better we can
> go with this. I often dream of the day someone pissed of by using
> prehistoric issue trackers writes a modern one, just like when
> Linus created Git...

In addition to Tims statement I suggest that we can use some labels to
separate the issue type like, help, lua, core, and so on. I think we
can use similar labels as we already use in the patches/changelog
[DOC/LUA/SSL...].

With labels can everyone just limit the view which the person want.

> - wiki : we all know that the architecture guide is obsolete,
> everyone wants to refresh it and nobody can because it's a tedious
> task that no single person can address, and nobody anymore knows
> all haproxy pieces. In addition a lot of participants have useful
> tips to share and would be much more at ease with putting this
> into a wiki than by editing architecture.txt. Thus I'd like to
> reuse either Github or Gitlab's wiki to place real contents edited
> and maintained by the community, and to kill this obsolete
> file. This way the obsolete stuff on the main haproxy.org page
> could go as well.

I agree here with Tim that the *pages solution looks much more usable
then the wiki way.

As I build my homepage www.me2digital.com with gohugo it would be nice
to have such a solution here also in place.

https://gohugo.io/hosting-and-deployment/hosting-on-github/

I can lend you a hand on this, at least for the beginning.

> - automated builds and CI : gitlab and github have these
> possibilities (either natively or relying on third party
> products). It would help us detect the occasional stupid build
> breakage (like building with/ without threads, with/without SSL).

Oh yes this would be nice but it's a beast on there own.

> - we could imagine reusing self-hosted pages to put Cyril's
> generated docs, or to link there if it's easier. Similar for
> Thierry's Lua docs, it will first depend on what these ones prefer
> (I don't want to steal your work guys, don't worry, just
> suggesting that it could be done if you find it desirable).

As Cyril's docs are already on "github pages" should the migration be a
easy task, IMHO.

> - maybe placing the releases there will help (I don't know the
> upload process for any of these, will have to check closer). It
> may even be used for Aleks' docker images if that makes sense at
> all (I don't know).

I don't think that the docker images are good releases on github, But
I'm open for discussion if it makes sense.

The doc suggest that the releases can be "easily build" but I haven't
done this before so maybe some one on the list with more experience could
share the experience with us.

https://help.github.com/articles/about-releases/

> Feel free to discuss these points here on the list as usual. Please don't
> PM me, I don't have time to rehash the same points with multiple persons.
> If I missed anything important that you care about, say it now.

Thank you for writing this mail too keep us up2date.

> thanks,
> Willy

Very best regars
Aleks
Baptiste
Re: haproxy 1.9 status update
June 05, 2018 12:10AM
Hi,

Thanks all for the amazing work :)

I just like to focus on a particular point:

- wiki : we all know that the architecture guide is obsolete, everyone
> wants to refresh it and nobody can because it's a tedious task that
> no single person can address, and nobody anymore knows all haproxy
> pieces. In addition a lot of participants have useful tips to share
> and would be much more at ease with putting this into a wiki than by
> editing architecture.txt. Thus I'd like to reuse either Github or
> Gitlab's wiki to place real contents edited and maintained by the
> community, and to kill this obsolete file. This way the obsolete stuff
> on the main haproxy.org page could go as well.
>
>
Here is my wish list for such architecture guide:
- gitfied
- "generic" formatting language (markdown, restructured text, etc...)
- ability to update pages using command line on a local PC or a browser (I
know github allows editing content online)

I have no idea what tools may be best suited for this purpose, but I'd like
we decide to use one quickly, so I can start contributing to it.

Cheers

PS: I'm testing github pages at the moment.
Baptiste
Re: haproxy 1.9 status update
June 10, 2018 09:40AM
Hi Willy,

I don't see anywhere DNS over TCP mentioned.
From my point of view (and integration with consul / kubernetes), it's an
important topic and I'd like to get it done by 1.9, ideally.
I have not checked yet how this could be implemented in HAProxy, but I
don't really feel comfortable to do it, I'm afraid it's a huge task and I
won't have enough time to do it.

Baptiste
Willy Tarreau
Re: haproxy 1.9 status update
June 10, 2018 10:00AM
Hi Baptiste,

On Sun, Jun 10, 2018 at 09:27:09AM +0200, Baptiste wrote:
> Hi Willy,
>
> I don't see anywhere DNS over TCP mentioned.

I have reported what I'm aware people are currently working on, as you
know I don't want to speculate anymore about what would be nice to have
if someone was willing to work on it, since the roadmap file is already
full of this.

> From my point of view (and integration with consul / kubernetes), it's an
> important topic and I'd like to get it done by 1.9, ideally.
> I have not checked yet how this could be implemented in HAProxy, but I
> don't really feel comfortable to do it, I'm afraid it's a huge task and I
> won't have enough time to do it.

I certainly can understand, but first it means that someone will have
to step up saying "I'm willing to work on this" and second you'll have
to find at least enough time to help this person and go back and forth
with the review, and later for the debugging. As you know this is not
a negligible part of the job at all!

Cheers,
Willy
Baptiste
Re: haproxy 1.9 status update
July 04, 2018 02:50PM
Sorry to wake up an old thread, but I'm very concerned by the lack of
"architecture guide" documentation with HAProxy.
Did we make any progress on this topic?

Baptiste
Sorry, only registered users may post in this forum.

Click here to login