Welcome! Log In Create A New Profile

Advanced

ERR_SSL_BAD_RECORD_MAC_ALERT when trying to reuse SSL session

Posted by Abilio Marques 
Using NGINX 1.12.2 on MIPS (haven't tested on x86), if I set:

ssl_session_cache shared:SSL:1m; # it also fails with 10m


And the client reestablishes the connection, it
gets: net::ERR_SSL_BAD_RECORD_MAC_ALERT when trying to reuse SSL session.

Has anyone seen anything like this?


More detail:

This was tested on 1.12.2, on a MIPS CPU, using OpenSSL 1.0.2j, and built
by gcc 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 r47070).

Interesting portion of my configuration file:

server {
listen 443 ssl;

ssl_certificate /etc/ssl/certs/bridge.cert.pem;
ssl_certificate_key /etc/ssl/private/bridge.key.pem;

ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256;
ssl_ecdh_curve prime256v1;

ssl_session_timeout 24h;
ssl_session_tickets on;
ssl_session_cache shared:SSL:1m; # set to 10m, still fails, remove, the
problem seems to disappear

keepalive_timeout 1s; # reduced during troubleshooting to make it
trigger easily
keepalive_requests 1; # reduced during troubleshooting to make it
trigger easily

include apiv1.conf; # where all the location rules are
}
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Hello!

On Thu, Mar 08, 2018 at 12:16:50PM +0100, Abilio Marques wrote:

> Using NGINX 1.12.2 on MIPS (haven't tested on x86), if I set:
>
> ssl_session_cache shared:SSL:1m; # it also fails with 10m
>
>
> And the client reestablishes the connection, it
> gets: net::ERR_SSL_BAD_RECORD_MAC_ALERT when trying to reuse SSL session.
>
> Has anyone seen anything like this?
>
>
> More detail:
>
> This was tested on 1.12.2, on a MIPS CPU, using OpenSSL 1.0.2j, and built
> by gcc 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 r47070).

This certainly works on x86, so it must be something
MIPS-specific or something specific to your particular build.

Last time I saw OpenWrt/Linaro nginx builds, they were compiled
using buggy 3rd party crossbuild patches, and didn't work due to
this (see https://trac.nginx.org/nginx/ticket/899). You may want
to check your build before trying to do anything else.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Hi,

After working a bit more on the issue, I also found that:

- Using a new pair of key/certificate makes the problem not to show
anymore. So, some files will make it fail, some files make it work. The
files are of different length, so it seems to be correlated to that.
- Using LD_PRELOAD with an "empty" (as in no C code) so file makes the
problem disappear. I discover this while trying to hook the calls to
OpenSSL, just to discover that even if I removed all my code, the problem
will go away.


As there are at least 3 different ways to make it disappear, looks to me
that is not directly related to SSL session, but to something completely
different. I cannot run valgrind on the MIPS hardware (no enough RAM), and
I've been trying to reproduce it on QEMU, to no avail.

Any ideas on how to proceed? Do you think Valgrind will help at all? Any
other insights?

On Thu, Mar 8, 2018 at 12:16 PM, Abilio Marques <[email protected]> wrote:

> Using NGINX 1.12.2 on MIPS (haven't tested on x86), if I set:
>
> ssl_session_cache shared:SSL:1m; # it also fails with 10m
>
>
> And the client reestablishes the connection, it
> gets: net::ERR_SSL_BAD_RECORD_MAC_ALERT when trying to reuse SSL session.
>
> Has anyone seen anything like this?
>
>
> More detail:
>
> This was tested on 1.12.2, on a MIPS CPU, using OpenSSL 1.0.2j, and built
> by gcc 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 r47070).
>
> Interesting portion of my configuration file:
>
> server {
> listen 443 ssl;
>
> ssl_certificate /etc/ssl/certs/bridge.cert.pem;
> ssl_certificate_key /etc/ssl/private/bridge.key.pem;
>
> ssl_protocols TLSv1.2;
> ssl_prefer_server_ciphers on;
> ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256;
> ssl_ecdh_curve prime256v1;
>
> ssl_session_timeout 24h;
> ssl_session_tickets on;
> ssl_session_cache shared:SSL:1m; # set to 10m, still fails, remove,
> the problem seems to disappear
>
> keepalive_timeout 1s; # reduced during troubleshooting to make it
> trigger easily
> keepalive_requests 1; # reduced during troubleshooting to make it
> trigger easily
>
> include apiv1.conf; # where all the location rules are
> }
>
>
>
>
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Hello!

On Mon, Mar 19, 2018 at 03:04:14PM +0100, Abilio Marques wrote:

> After working a bit more on the issue, I also found that:
>
> - Using a new pair of key/certificate makes the problem not to show
> anymore. So, some files will make it fail, some files make it work. The
> files are of different length, so it seems to be correlated to that.
> - Using LD_PRELOAD with an "empty" (as in no C code) so file makes the
> problem disappear. I discover this while trying to hook the calls to
> OpenSSL, just to discover that even if I removed all my code, the problem
> will go away.
>
>
> As there are at least 3 different ways to make it disappear, looks to me
> that is not directly related to SSL session, but to something completely
> different. I cannot run valgrind on the MIPS hardware (no enough RAM), and
> I've been trying to reproduce it on QEMU, to no avail.
>
> Any ideas on how to proceed? Do you think Valgrind will help at all? Any
> other insights?

As previously suggested, first of all you may want to check your
build, see here:

http://mailman.nginx.org/pipermail/nginx/2018-March/055829.html

Check "nginx -V" output. If it contains something like
"crossbuild", then recompile nginx yourself, without any 3rd party
patches, ideally - on the host itself (a virtual machine with the
same OS will be ok too), and check if the problem persists.

Also, it might be a good idea to play with different OpenSSL
versions (including compiling them statically into nginx using the
"--with-openssl" configure option) and different compilers.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Sorry, only registered users may post in this forum.

Click here to login