Welcome! Log In Create A New Profile

Advanced

Haproxy 1.8 with OpenSSL 1.1.1-pre4 stops working after 1 hour

Posted by Sander Hoentjen 
Hi all,

I built Haproxy (1.8.7) against openssl 1.1.1-pre4, and now after 1 hour
running haproxy stops accepting new SSL connections. When I restart it
works again for almost(?) exactly 1 hour, then stops.
Any idea what might be causing this, or where I should look

# haproxy -vv
HA-Proxy version 1.8.7 2018/04/07
Copyright 2000-2018 Willy Tarreau <[email protected]>

Build options :
  TARGET  = linux2628
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -fno-strict-overflow -Wno-unused-label
  OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1
USE_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with network namespace support.
Built with zlib version : 1.2.3
Running on zlib version : 1.2.3
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 7.8 2008-09-05
Running on PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with multi-threading support.
Encrypted password support via crypt(3): yes
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Built with OpenSSL version : OpenSSL 1.1.1-pre4 (beta) 3 Apr 2018
Running on OpenSSL version : OpenSSL 1.1.1-pre4 (beta) 3 Apr 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
    [TRACE] trace
    [COMP] compression
    [SPOE] spoe

Regards,
Sander Hoentjen
Reading my email again it looks like somehow I messed up part of it,
retrying:

Hi all,

I built Haproxy (1.8.7) against openssl 1.1.1-pre4, and now after 1 hour
running haproxy stops accepting new SSL connections. When I restart it
works again for almost(?) exactly 1 hour, then stops. Any idea what
might be causing this, or where I should look? Especially the part that
it works for one hour seems weird to me. Next to that, only SSL
connections stop working, the plain ones continue to work. The setup has
one frontend that accepts both http and https, using:
    tcp-request inspect-delay 500ms
    tcp-request content accept if HTTP
    tcp-request content accept if { req.ssl_hello_type 1 }
Maybe this has something to do with it?
Exactly the same config, with only difference being built agains openssl
1.1.0 works without any issues.

Any help appreciated.

Regards,
Sander


On 04/13/2018 10:27 AM, Sander Hoentjen wrote:
> Hi all,
>
> I built Haproxy (1.8.7) against openssl 1.1.1-pre4, and now after 1 hour
> running haproxy stops accepting new SSL connections. When I restart it
> works again for almost(?) exactly 1 hour, then stops.
> Any idea what might be causing this, or where I should look
>
> # haproxy -vv
> HA-Proxy version 1.8.7 2018/04/07
> Copyright 2000-2018 Willy Tarreau <[email protected]>
>
> Build options :
>   TARGET  = linux2628
>   CPU     = generic
>   CC      = gcc
>   CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
> -fwrapv -fno-strict-overflow -Wno-unused-label
>   OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1
> USE_PCRE=1
>
> Default settings :
>   maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
>
> Built with network namespace support.
> Built with zlib version : 1.2.3
> Running on zlib version : 1.2.3
> Compression algorithms supported : identity("identity"),
> deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
> Built with PCRE version : 7.8 2008-09-05
> Running on PCRE version : 7.8 2008-09-05
> PCRE library supports JIT : no (USE_PCRE_JIT not set)
> Built with multi-threading support.
> Encrypted password support via crypt(3): yes
> Built with transparent proxy support using: IP_TRANSPARENT
> IPV6_TRANSPARENT IP_FREEBIND
> Built with OpenSSL version : OpenSSL 1.1.1-pre4 (beta) 3 Apr 2018
> Running on OpenSSL version : OpenSSL 1.1.1-pre4 (beta) 3 Apr 2018
> OpenSSL library supports TLS extensions : yes
> OpenSSL library supports SNI : yes
> OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
>
> Available polling systems :
>       epoll : pref=300,  test result OK
>        poll : pref=200,  test result OK
>      select : pref=150,  test result OK
> Total: 3 (3 usable), will use epoll.
>
> Available filters :
>     [TRACE] trace
>     [COMP] compression
>     [SPOE] spoe
>
> Regards,
> Sander Hoentjen
>
Hello Sander,


On 16 April 2018 at 10:55, Sander Hoentjen <[email protected]> wrote:
> Reading my email again it looks like somehow I messed up part of it,
> retrying:
>
> Hi all,
>
> I built Haproxy (1.8.7) against openssl 1.1.1-pre4, and now after 1 hour
> running haproxy stops accepting new SSL connections.

I have seen something like that a few weeks ago as well, but I've
reverted to openssl stable, as I did not have time to get involved
with troubleshooting openssl master at this point.


That having said you may want to retry with latest 1.1.1-pre5, it may
have been already fixed (Prevent a possible recursion in ERR_get_state
....).



Lukas
Hi Lucas,

On 04/17/2018 04:27 PM, Lukas Tribus wrote:
> Hello Sander,
>
>
> On 16 April 2018 at 10:55, Sander Hoentjen <[email protected]> wrote:
>> Reading my email again it looks like somehow I messed up part of it,
>> retrying:
>>
>> Hi all,
>>
>> I built Haproxy (1.8.7) against openssl 1.1.1-pre4, and now after 1 hour
>> running haproxy stops accepting new SSL connections.
> I have seen something like that a few weeks ago as well, but I've
> reverted to openssl stable, as I did not have time to get involved
> with troubleshooting openssl master at this point.
>
>
> That having said you may want to retry with latest 1.1.1-pre5, it may
> have been already fixed (Prevent a possible recursion in ERR_get_state
> ...).
I just tried 1.1.1-pre5, and I still have the same issue. What do you
think, is it better to report to the openssl team, or here? I have no
idea where the issue is coming from, but I was hoping the "broken after
exactly 1 hour" would ring some bells.

Sander
Sorry, only registered users may post in this forum.

Click here to login