Welcome! Log In Create A New Profile

Advanced

TLS handshake works with certificate name mismatch using "verify required" and "verifyhost"

Posted by Martin RADEL 
Hi all,

we have a strange situation with our HAProxy, running on Version 1.8.8 with OpenSSL.
(See the details in the setup listed below - some lines are missing by intention. It's a config snippet with just the interesting parts mentioned)

Initial situation:
We run a HAProxy instance which enforces mutual TLS on the frontend, allowing only clients to connect to it when they will present a specific certificate.
The HAPRoxy also does mutual TLS to the backend, presenting its frontend server certificate to the backend as a client certificate.
The backend only allows connections when the HAProxy's certificate is presented to it.
To have a proper TLS handshake to the backend, and to be able to identify a man-in-the-middle scenario, we use the "verify required" directive together with the "verifyhost" directive.

The HAProxy is not able to resolve the backend's real DNS-hostname, so it's using the IP of the server instead (10.1.1.1)
The backend is presenting a wildcard server certificate with a DNS-hostname looking like "*.foo.bar"


In this configuration, one could assume that there is always a certificate name mismatch with the TLS handshake:
Backend server will present its server certificate with a proper DNS hostname in it, and the HAProxy will find out that it doesn't match the initially used connection name "10.1.1.1".


Issue:
In fact the connection to the backend works all the time, even when there is a name mismatch and even if we use the "verify required" option together with "verifyhost".
Seems as if HAProxy completely ignores the mismatch, as if we would use the option "verify none".


According to HAProxy documentation, this is clearly a not-expected behavior:
http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verify


Can somebody please share some knowledge why this is working, or can confirm that this is a bug?


#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log /dev/log local2
pidfile /run/haproxy/haproxy.pid
maxconn 20000
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS:!RC4
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option http-server-close
option redispatch
retries 3
maxconn 20000
errorfile 503 /etc/haproxy/errorpage.html
default-server init-addr last,libc,none

# ====================================================
# HAPROXY CONFIG WITH WILDCARD CERTIFICATE ON BACKEND
# ====================================================
# --- FRONTEND1 (TLS with mutual authentication) ---
frontend FRONTEND1
option forwardfor except 127.0.0.0/8
acl authorizedClient ssl_c_s_dn(cn) -m str -f /etc/haproxy/authorized_clients.cfg
bind *:443 ssl crt /etc/haproxy/certs/frontend-server-certificate.pem ca-file /etc/haproxy/certs/frontend-ca-certificates.crt verify required
use_backend BACKEND1 if authorizedClient frontend

# --- BACKEND1
backend BACKEND1
option forwardfor except 127.0.0.0/8
server BACKEND1-server 10.1.1.1:443 check inter 30s verify required ssl verifyhost *.foo.barhttp://www.foo.bar ca-file /etc/haproxy/certs/backend-ca-certificates.crt crt /etc/haproxy/certs/frontend-server-certificate.pem







This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to ? 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien).
Hi Martin,

On Thu, Jul 12, 2018 at 6:55 PM, Martin RADEL <
[email protected]> wrote:

> Hi all,
>
>
>
> we have a strange situation with our HAProxy, running on Version 1.8.8
> with OpenSSL.
>
> (See the details in the setup listed below - some lines are missing by
> intention. It’s a config snippet with just the interesting parts mentioned)
>
>
>
> Initial situation:
>
> We run a HAProxy instance which enforces mutual TLS on the frontend,
> allowing only clients to connect to it when they will present a specific
> certificate.
>
> The HAPRoxy also does mutual TLS to the backend, presenting its frontend
> server certificate to the backend as a client certificate.
>
> The backend only allows connections when the HAProxy’s certificate is
> presented to it.
>
> To have a proper TLS handshake to the backend, and to be able to identify
> a man-in-the-middle scenario, we use the “verify required” directive
> together with the “verifyhost” directive.
>
>
>
> The HAProxy is not able to resolve the backend’s real DNS-hostname, so
> it’s using the IP of the server instead (10.1.1.1)
>
> The backend is presenting a wildcard server certificate with a
> DNS-hostname looking like “*.foo.bar”
>
>
>
>
>
> In this configuration, one could assume that there is always a certificate
> name mismatch with the TLS handshake:
>
> Backend server will present its server certificate with a proper DNS
> hostname in it, and the HAProxy will find out that it doesn’t match the
> initially used connection name “10.1.1.1”.
>
>
>

​Just checking if the IP hasn't been by any chance included in the
certificate subjectAlternateNames ?


>
>
> Issue:
>
> In fact the connection to the backend works all the time, even when there
> is a name mismatch and even if we use the “verify required” option together
> with “verifyhost”.
>
> Seems as if HAProxy completely ignores the mismatch, as if we would use
> the option “verify none”.
>
>
>
>
>
> According to HAProxy documentation, this is clearly a not-expected
> behavior:
>
> http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verify
>
>
>
>
>
> Can somebody please share some knowledge why this is working, or can
> confirm that this is a bug?
>
>
>
>
>
> #---------------------------------------------------------------------
>
> # Global settings
>
> #---------------------------------------------------------------------
>
> global
>
> log /dev/log local2
>
> pidfile /run/haproxy/haproxy.pid
>
> maxconn 20000
>
> ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+
> AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!
> aNULL:!MD5:!DSS:!RC4
>
> ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
>
> stats socket /var/lib/haproxy/stats
>
>
>
> #---------------------------------------------------------------------
>
> # common defaults that all the 'listen' and 'backend' sections will
>
> # use if not designated in their block
>
> #---------------------------------------------------------------------
>
> defaults
>
> mode http
>
> log global
>
> option http-server-close
>
> option redispatch
>
> retries 3
>
> maxconn 20000
>
> errorfile 503 /etc/haproxy/errorpage.html
>
> default-server init-addr last,libc,none
>
>
>
> # ====================================================
>
> # HAPROXY CONFIG WITH WILDCARD CERTIFICATE ON BACKEND
>
> # ====================================================
>
> # --- FRONTEND1 (TLS with mutual authentication) ---
>
> frontend FRONTEND1
>
> option forwardfor except 127.0.0.0/8
>
> acl authorizedClient ssl_c_s_dn(cn) -m str -f /etc/haproxy/authorized_
> clients.cfg
>
> bind *:443 ssl crt /etc/haproxy/certs/frontend-server-certificate.pem
> ca-file /etc/haproxy/certs/frontend-ca-certificates.crt verify required
>
> use_backend BACKEND1 if authorizedClient frontend
>
>
>
> # --- BACKEND1
>
> backend BACKEND1
>
> option forwardfor except 127.0.0.0/8
>
> server BACKEND1-server 10.1.1.1:443 check inter 30s verify required
> ssl verifyhost *.foo.bar http://www.foo.bar ca-file
> /etc/haproxy/certs/backend-ca-certificates.crt crt
> /etc/haproxy/certs/frontend-server-certificate.pem
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> This message and any attachment ("the Message") are confidential. If you
> have received the Message in error, please notify the sender immediately
> and delete the Message from your system, any use of the Message is
> forbidden. Correspondence via e-mail is primarily for information purposes.
> RBI neither makes nor accepts legally binding statements via e-mail unless
> explicitly agreed otherwise. Information pursuant to § 14 Austrian
> Companies Code: Raiffeisen Bank International AG; Registered Office: Am
> Stadtpark 9
> https://maps.google.com/?q=Am+Stadtpark+9&entry=gmail&source=g, 1030
> Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court
> of Vienna (Handelsgericht Wien).
>


​Regards,
Igor​
On Fri, Jul 13, 2018 at 11:08 AM, Igor Cicimov <
[email protected]> wrote:

> Hi Martin,
>
> On Thu, Jul 12, 2018 at 6:55 PM, Martin RADEL <
> [email protected]> wrote:
>
>> Hi all,
>>
>>
>>
>> we have a strange situation with our HAProxy, running on Version 1.8.8
>> with OpenSSL.
>>
>> (See the details in the setup listed below - some lines are missing by
>> intention. It’s a config snippet with just the interesting parts mentioned)
>>
>>
>>
>> Initial situation:
>>
>> We run a HAProxy instance which enforces mutual TLS on the frontend,
>> allowing only clients to connect to it when they will present a specific
>> certificate.
>>
>> The HAPRoxy also does mutual TLS to the backend, presenting its frontend
>> server certificate to the backend as a client certificate.
>>
>> The backend only allows connections when the HAProxy’s certificate is
>> presented to it.
>>
>> To have a proper TLS handshake to the backend, and to be able to identify
>> a man-in-the-middle scenario, we use the “verify required” directive
>> together with the “verifyhost” directive.
>>
>>
>>
>> The HAProxy is not able to resolve the backend’s real DNS-hostname, so
>> it’s using the IP of the server instead (10.1.1.1)
>>
>> The backend is presenting a wildcard server certificate with a
>> DNS-hostname looking like “*.foo.bar”
>>
>>
>>
>>
>>
>> In this configuration, one could assume that there is always a
>> certificate name mismatch with the TLS handshake:
>>
>> Backend server will present its server certificate with a proper DNS
>> hostname in it, and the HAProxy will find out that it doesn’t match the
>> initially used connection name “10.1.1.1”.
>>
>>
>>
>
> ​Just checking if the IP hasn't been by any chance included in the
> certificate subjectAlternateNames ?
> ​
>
>>
>>
>> Issue:
>>
>> In fact the connection to the backend works all the time, even when there
>> is a name mismatch and even if we use the “verify required” option together
>> with “verifyhost”.
>>
>> Seems as if HAProxy completely ignores the mismatch, as if we would use
>> the option “verify none”.
>>
>>
>>
>>
>>
>> According to HAProxy documentation, this is clearly a not-expected
>> behavior:
>>
>> http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verify
>>
>>
>>
>>
>>
>> Can somebody please share some knowledge why this is working, or can
>> confirm that this is a bug?
>>
>>
>>
>>
>>
>> #---------------------------------------------------------------------
>>
>> # Global settings
>>
>> #---------------------------------------------------------------------
>>
>> global
>>
>> log /dev/log local2
>>
>> pidfile /run/haproxy/haproxy.pid
>>
>> maxconn 20000
>>
>> ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES
>> 256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS:!RC4
>>
>> ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
>>
>> stats socket /var/lib/haproxy/stats
>>
>>
>>
>> #---------------------------------------------------------------------
>>
>> # common defaults that all the 'listen' and 'backend' sections will
>>
>> # use if not designated in their block
>>
>> #---------------------------------------------------------------------
>>
>> defaults
>>
>> mode http
>>
>> log global
>>
>> option http-server-close
>>
>> option redispatch
>>
>> retries 3
>>
>> maxconn 20000
>>
>> errorfile 503 /etc/haproxy/errorpage.html
>>
>> default-server init-addr last,libc,none
>>
>>
>>
>> # ====================================================
>>
>> # HAPROXY CONFIG WITH WILDCARD CERTIFICATE ON BACKEND
>>
>> # ====================================================
>>
>> # --- FRONTEND1 (TLS with mutual authentication) ---
>>
>> frontend FRONTEND1
>>
>> option forwardfor except 127.0.0.0/8
>>
>> acl authorizedClient ssl_c_s_dn(cn) -m str -f
>> /etc/haproxy/authorized_clients.cfg
>>
>> bind *:443 ssl crt /etc/haproxy/certs/frontend-server-certificate.pem
>> ca-file /etc/haproxy/certs/frontend-ca-certificates.crt verify required
>>
>> use_backend BACKEND1 if authorizedClient frontend
>>
>>
>>
>> # --- BACKEND1
>>
>> backend BACKEND1
>>
>> option forwardfor except 127.0.0.0/8
>>
>> server BACKEND1-server 10.1.1.1:443 check inter 30s verify required
>> ssl verifyhost *.foo.bar http://www.foo.bar ca-file
>> /etc/haproxy/certs/backend-ca-certificates.crt crt
>> /etc/haproxy/certs/frontend-server-certificate.pem
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> This message and any attachment ("the Message") are confidential. If you
>> have received the Message in error, please notify the sender immediately
>> and delete the Message from your system, any use of the Message is
>> forbidden. Correspondence via e-mail is primarily for information purposes.
>> RBI neither makes nor accepts legally binding statements via e-mail unless
>> explicitly agreed otherwise. Information pursuant to § 14 Austrian
>> Companies Code: Raiffeisen Bank International AG; Registered Office: Am
>> Stadtpark 9
>> https://maps.google.com/?q=Am+Stadtpark+9&entry=gmail&source=g, 1030
>> Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court
>> of Vienna (Handelsgericht Wien).
>>
>
>
> ​Regards,
> Igor​
>
> ​
Ah, another thing I forgot to ask and might be relevant, are you using SNI
enabled client for the testing or not?​
On Fri, Jul 13, 2018 at 11:26 AM, Igor Cicimov <
[email protected]> wrote:

> On Fri, Jul 13, 2018 at 11:08 AM, Igor Cicimov <
> [email protected]> wrote:
>
>> Hi Martin,
>>
>> On Thu, Jul 12, 2018 at 6:55 PM, Martin RADEL <
>> [email protected]> wrote:
>>
>>> Hi all,
>>>
>>>
>>>
>>> we have a strange situation with our HAProxy, running on Version 1.8.8
>>> with OpenSSL.
>>>
>>> (See the details in the setup listed below - some lines are missing by
>>> intention. It’s a config snippet with just the interesting parts mentioned)
>>>
>>>
>>>
>>> Initial situation:
>>>
>>> We run a HAProxy instance which enforces mutual TLS on the frontend,
>>> allowing only clients to connect to it when they will present a specific
>>> certificate.
>>>
>>> The HAPRoxy also does mutual TLS to the backend, presenting its frontend
>>> server certificate to the backend as a client certificate.
>>>
>>> The backend only allows connections when the HAProxy’s certificate is
>>> presented to it.
>>>
>>> To have a proper TLS handshake to the backend, and to be able to
>>> identify a man-in-the-middle scenario, we use the “verify required”
>>> directive together with the “verifyhost” directive.
>>>
>>>
>>>
>>> The HAProxy is not able to resolve the backend’s real DNS-hostname, so
>>> it’s using the IP of the server instead (10.1.1.1)
>>>
>>> The backend is presenting a wildcard server certificate with a
>>> DNS-hostname looking like “*.foo.bar”
>>>
>>>
>>>
>>>
>>>
>>> In this configuration, one could assume that there is always a
>>> certificate name mismatch with the TLS handshake:
>>>
>>> Backend server will present its server certificate with a proper DNS
>>> hostname in it, and the HAProxy will find out that it doesn’t match the
>>> initially used connection name “10.1.1.1”.
>>>
>>>
>>>
>>
>> ​Just checking if the IP hasn't been by any chance included in the
>> certificate subjectAlternateNames ?
>> ​
>>
>>>
>>>
>>> Issue:
>>>
>>> In fact the connection to the backend works all the time, even when
>>> there is a name mismatch and even if we use the “verify required” option
>>> together with “verifyhost”.
>>>
>>> Seems as if HAProxy completely ignores the mismatch, as if we would use
>>> the option “verify none”.
>>>
>>>
>>>
>>>
>>>
>>> According to HAProxy documentation, this is clearly a not-expected
>>> behavior:
>>>
>>> http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verify
>>>
>>>
>>>
>>>
>>>
>>> Can somebody please share some knowledge why this is working, or can
>>> confirm that this is a bug?
>>>
>>>
>>>
>>>
>>>
>>> #---------------------------------------------------------------------
>>>
>>> # Global settings
>>>
>>> #---------------------------------------------------------------------
>>>
>>> global
>>>
>>> log /dev/log local2
>>>
>>> pidfile /run/haproxy/haproxy.pid
>>>
>>> maxconn 20000
>>>
>>> ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES
>>> 256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!
>>> MD5:!DSS:!RC4
>>>
>>> ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
>>>
>>> stats socket /var/lib/haproxy/stats
>>>
>>>
>>>
>>> #---------------------------------------------------------------------
>>>
>>> # common defaults that all the 'listen' and 'backend' sections will
>>>
>>> # use if not designated in their block
>>>
>>> #---------------------------------------------------------------------
>>>
>>> defaults
>>>
>>> mode http
>>>
>>> log global
>>>
>>> option http-server-close
>>>
>>> option redispatch
>>>
>>> retries 3
>>>
>>> maxconn 20000
>>>
>>> errorfile 503 /etc/haproxy/errorpage.html
>>>
>>> default-server init-addr last,libc,none
>>>
>>>
>>>
>>> # ====================================================
>>>
>>> # HAPROXY CONFIG WITH WILDCARD CERTIFICATE ON BACKEND
>>>
>>> # ====================================================
>>>
>>> # --- FRONTEND1 (TLS with mutual authentication) ---
>>>
>>> frontend FRONTEND1
>>>
>>> option forwardfor except 127.0.0.0/8
>>>
>>> acl authorizedClient ssl_c_s_dn(cn) -m str -f
>>> /etc/haproxy/authorized_clients.cfg
>>>
>>> bind *:443 ssl crt /etc/haproxy/certs/frontend-server-certificate.pem
>>> ca-file /etc/haproxy/certs/frontend-ca-certificates.crt verify required
>>>
>>> use_backend BACKEND1 if authorizedClient frontend
>>>
>>>
>>>
>>> # --- BACKEND1
>>>
>>> backend BACKEND1
>>>
>>> option forwardfor except 127.0.0.0/8
>>>
>>> server BACKEND1-server 10.1.1.1:443 check inter 30s verify
>>> required ssl verifyhost *.foo.bar http://www.foo.bar ca-file
>>> /etc/haproxy/certs/backend-ca-certificates.crt crt
>>> /etc/haproxy/certs/frontend-server-certificate.pem
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> This message and any attachment ("the Message") are confidential. If you
>>> have received the Message in error, please notify the sender immediately
>>> and delete the Message from your system, any use of the Message is
>>> forbidden. Correspondence via e-mail is primarily for information purposes.
>>> RBI neither makes nor accepts legally binding statements via e-mail unless
>>> explicitly agreed otherwise. Information pursuant to § 14 Austrian
>>> Companies Code: Raiffeisen Bank International AG; Registered Office: Am
>>> Stadtpark 9
>>> https://maps.google.com/?q=Am+Stadtpark+9&entry=gmail&source=g, 1030
>>> Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court
>>> of Vienna (Handelsgericht Wien).
>>>
>>
>>
>> ​Regards,
>> Igor​
>>
>> ​
> Ah, another thing I forgot to ask and might be relevant, are you using SNI
> enabled client for the testing or not?​
>
>
​Sorry to reply again but just noticed the "verifyhost *.foo.bar
http://www.foo.bar"; in your server line which might explain it all​, see
http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verifyhost

This setting is only available when support for OpenSSL was built in, and
only takes effect if 'verify required' is also specified. *This directive sets
a default static hostname to check the server's certificate against when no
SNI was used to connect to the server*. If SNI is not used, this is the only
way to enable hostname verification. This static hostname, when set, will
also be used for health checks (which cannot provide an SNI value). *If none
of the hostnames in the certificate match the specified hostname, the
handshake is aborted*. The hostnames in the server-provided certificate may
include wildcards. See also "verify
<http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#>";, "sni
<http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#sni>";
and "no-verifyhost
<http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#no-verifyhost>";
options.


I guess, in combination with absence of SNI, this setting would cause the
hostname to always match your cert.
Hello Martin,


> we have a strange situation with our HAProxy, running on Version 1.8.8 with OpenSSL.

Please share the output of haproxy -vv. Did you build openssl yourself
or is this a distribution provided openssl lib? I am asking because
build issues can lead to very strange behavior.



> server BACKEND1-server 10.1.1.1:443 check inter 30s verify required ssl verifyhost *.foo.bar

*.foo.bar is not a valid hostname. It is a valid wildcard
representation in a cert's SAN, yes, but not a hostname. Use real
hostname for verifyhost instead, like www.foo.bar



Also, lets confirm the backend is really configured as per
expectations, by running requests via curl from the haproxy box:

This should work:
curl -v --cacert /etc/haproxy/certs/backend-ca-certificates.crt
--resolve www.foo.bar:443:10.1.1.1 https://www.foo.bar/

This should fail:
curl -v --cacert /etc/haproxy/certs/backend-ca-certificates.crt
--resolve www.foo.fail:443:10.1.1.1 https://www.foo.fail/




cheers,
lukas
Hi,

The certificate subject and subject alternate name are set to “*.foo.bar” (I’m replacing real DNS name here with foo.bar here because of security reasons).
There is no IP address included in the server’s certificate.

We are not using SNI on our clients.

BR
Martin


From: [email protected]<mailto:[email protected]> [mailto:[email protected]]
Sent: Freitag, 13. Juli 2018 03:27
To: Martin RADEL <[email protected]<mailto:[email protected]>>
Cc: [email protected]<mailto:[email protected]>
Subject: Re: TLS handshake works with certificate name mismatch using "verify required" and "verifyhost"

On Fri, Jul 13, 2018 at 11:08 AM, Igor Cicimov <[email protected]<mailto:[email protected]>> wrote:
Hi Martin,

On Thu, Jul 12, 2018 at 6:55 PM, Martin RADEL <[email protected]<mailto:[email protected]>> wrote:
Hi all,

we have a strange situation with our HAProxy, running on Version 1.8.8 with OpenSSL.
(See the details in the setup listed below - some lines are missing by intention. It’s a config snippet with just the interesting parts mentioned)

Initial situation:
We run a HAProxy instance which enforces mutual TLS on the frontend, allowing only clients to connect to it when they will present a specific certificate.
The HAPRoxy also does mutual TLS to the backend, presenting its frontend server certificate to the backend as a client certificate.
The backend only allows connections when the HAProxy’s certificate is presented to it.
To have a proper TLS handshake to the backend, and to be able to identify a man-in-the-middle scenario, we use the “verify required” directive together with the “verifyhost” directive.

The HAProxy is not able to resolve the backend’s real DNS-hostname, so it’s using the IP of the server instead (10.1.1.1)
The backend is presenting a wildcard server certificate with a DNS-hostname looking like “*.foo.bar”


In this configuration, one could assume that there is always a certificate name mismatch with the TLS handshake:
Backend server will present its server certificate with a proper DNS hostname in it, and the HAProxy will find out that it doesn’t match the initially used connection name “10.1.1.1”.


​Just checking if the IP hasn't been by any chance included in the certificate subjectAlternateNames ?


Issue:
In fact the connection to the backend works all the time, even when there is a name mismatch and even if we use the “verify required” option together with “verifyhost”.
Seems as if HAProxy completely ignores the mismatch, as if we would use the option “verify none”.


According to HAProxy documentation, this is clearly a not-expected behavior:
http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verifyhttps://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcbonte.github.io%2Fhaproxy-dconv%2F1.8%2Fconfiguration.html%235.2-verify&data=02%7C01%7Cmartin.radel%40rbinternational.com%7C7e7f5dccd90b4745bc0408d5e8601877%7C9b511fdaf0b143a5b06e1e720f64520a%7C1%7C1%7C636670421783654330&sdata=jl8aPpmP%2BtWoT2D4pjEqtU710EQxZg7LLNr3gTh4nEM%3D&reserved=0


Can somebody please share some knowledge why this is working, or can confirm that this is a bug?


#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log /dev/log local2
pidfile /run/haproxy/haproxy.pid
maxconn 20000
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS:!RC4
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option http-server-close
option redispatch
retries 3
maxconn 20000
errorfile 503 /etc/haproxy/errorpage.html
default-server init-addr last,libc,none

# ====================================================
# HAPROXY CONFIG WITH WILDCARD CERTIFICATE ON BACKEND
# ====================================================
# --- FRONTEND1 (TLS with mutual authentication) ---
frontend FRONTEND1
option forwardfor except 127.0.0.0/8https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.0%2F8&data=02%7C01%7Cmartin.radel%40rbinternational.com%7C7e7f5dccd90b4745bc0408d5e8601877%7C9b511fdaf0b143a5b06e1e720f64520a%7C1%7C1%7C636670421783664338&sdata=%2FUqCo%2BIaPk3YnRAJYvRZAKykLc0yfzj42pufo2BrJ20%3D&reserved=0
acl authorizedClient ssl_c_s_dn(cn) -m str -f /etc/haproxy/authorized_clients.cfg
bind *:443 ssl crt /etc/haproxy/certs/frontend-server-certificate.pem ca-file /etc/haproxy/certs/frontend-ca-certificates.crt verify required
use_backend BACKEND1 if authorizedClient frontend

# --- BACKEND1
backend BACKEND1
option forwardfor except 127.0.0.0/8https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.0%2F8&data=02%7C01%7Cmartin.radel%40rbinternational.com%7C7e7f5dccd90b4745bc0408d5e8601877%7C9b511fdaf0b143a5b06e1e720f64520a%7C1%7C1%7C636670421783674347&sdata=wobKyu3xkU37ZUyXsF0gXQv9Q28ARYEJZhPD8VDjrkg%3D&reserved=0
server BACKEND1-server 10.1.1.1:443https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F10.1.1.1%3A443&data=02%7C01%7Cmartin.radel%40rbinternational.com%7C7e7f5dccd90b4745bc0408d5e8601877%7C9b511fdaf0b143a5b06e1e720f64520a%7C1%7C1%7C636670421783674347&sdata=gCEQKcTBQ%2B%2Fc2MKr%2BLfkTQz7%2BIJjI7y1tz8jnZoIfYQ%3D&reserved=0 check inter 30s verify required ssl verifyhost *.foo.barhttps://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.foo.bar&data=02%7C01%7Cmartin.radel%40rbinternational.com%7C7e7f5dccd90b4745bc0408d5e8601877%7C9b511fdaf0b143a5b06e1e720f64520a%7C1%7C1%7C636670421783684355&sdata=ezWoVH71QZNaXLwrqDe8PD%2FcWjWD3AxS3VgtSMPHKrE%3D&reserved=0 ca-file /etc/haproxy/certs/backend-ca-certificates.crt crt /etc/haproxy/certs/frontend-server-certificate.pem







This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to § 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaps.google.com%2F%3Fq%3DAm%2BStadtpark%2B9%26entry%3Dgmail%26source%3Dg&data=02%7C01%7Cmartin.radel%40rbinternational.com%7C7e7f5dccd90b4745bc0408d5e8601877%7C9b511fdaf0b143a5b06e1e720f64520a%7C1%7C0%7C636670421783694363&sdata=hrkVo4P3Z84EemtrrMr5qeZLzAjkTn2%2Fxt9kW8ye974%3D&reserved=0, 1030 Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien).


​Regards,
Igor​


Ah, another thing I forgot to ask and might be relevant, are you using SNI enabled client for the testing or not?​

This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to § 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien).
Hi,

I think we found the issue:
Seems that there was a misunderstanding from us regarding the haproxy documentation with the "verifyhost" option.

If I get it right, the documentation says that if we have a haproxy config that
- Has "verify required"
- Does not use SNI
- Has no "verifyhost"
Then HAProxy will simply ignore whatever hostname the server sends back in its certificate and the handshake will be OK.

If the "verifyhost" option is set and it does match the pattern, SSL handshake will also be OK.
If the "verifyhost" option is set and it does not match the pattern, SSL handshake will fail.


We tested this with two different HAProxy configs now and I can confirm that it's exactly like that.
(Server is always presenting the same certificate with "*.foo.bar" in it's common name / subject)

TESTBACKEND1 config (WORKING) looks like this:
# --- TESTBACKEND1
backend TESTBACKEND1
option forwardfor except 127.0.0.0/8
server TESTBACKEND1-server 10.1.1.1:443 check inter 30s verify required ssl verifyhost www.foo.bar ca-file /etc/haproxy/certs/backend-ca-certificates.crt crt /etc/haproxy/certs/frontend-server-certificate.pem


TESTBACKEND2 config (NOT WORKING) looks like this:
# --- TESTBACKEND2
backend TESTBACKEND2
option forwardfor except 127.0.0.0/8
server TESTBACKEND2-server 10.1.1.1:443 check inter 30s verify required ssl verifyhost www.ham.eggs ca-file /etc/haproxy/certs/backend-ca-certificates.crt crt /etc/haproxy/certs/frontend-server-certificate.pem


Please can you confirm that our understanding of HAProxy documentation is correct?
If so, then we could mark this topic as "solved" :-)


BR
Martin


-----Original Message-----
From: lukas@ltri.eu [mailto:[email protected]]
Sent: Samstag, 14. Juli 2018 11:35
To: Martin RADEL <[email protected]>
Cc: haproxy@formilux.org
Subject: Re: TLS handshake works with certificate name mismatch using "verify required" and "verifyhost"

Hello Martin,


> we have a strange situation with our HAProxy, running on Version 1.8.8 with OpenSSL.

Please share the output of haproxy -vv. Did you build openssl yourself or is this a distribution provided openssl lib? I am asking because build issues can lead to very strange behavior.



> server BACKEND1-server 10.1.1.1:443 check inter 30s verify required
> ssl verifyhost *.foo.bar

*.foo.bar is not a valid hostname. It is a valid wildcard representation in a cert's SAN, yes, but not a hostname. Use real hostname for verifyhost instead, like www.foo.bar

Also, lets confirm the backend is really configured as per expectations, by running requests via curl from the haproxy box:

This should work:
curl -v --cacert /etc/haproxy/certs/backend-ca-certificates.crt
--resolve www.foo.bar:443:10.1.1.1 https://www.foo.bar/

This should fail:
curl -v --cacert /etc/haproxy/certs/backend-ca-certificates.crt
--resolve www.foo.fail:443:10.1.1.1 https://www.foo.bar/



cheers,
lukas
This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to § 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien).
On Mon, 16 Jul 2018 at 11:57, Martin RADEL
<[email protected]> wrote:
>
> Hi,
>
> I think we found the issue:
> Seems that there was a misunderstanding from us regarding the haproxy documentation with the "verifyhost" option.
>
> If I get it right, the documentation says that if we have a haproxy config that
> - Has "verify required"
> - Does not use SNI
> - Has no "verifyhost"
> Then HAProxy will simply ignore whatever hostname the server sends back in its certificate and the handshake will be OK.

Yes, that is correct, also see the verify docs:
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verify

Not sure how we ended up in this situation though. I remember there
was a vivid discussion about whether "verify" should default to none
or required. We opted for "required", to be "secure by default", but
this is totally useless given that it requires verifyhost or sni, and
will silently disable cert verification when those option are not
given. That's probably the worst thing we can do in this case; this
configuration should be rejected, imho. People that don't care about
cert verification should simply set "verify none". But here we are
now, and this is documented behavior :(

I think this was introduced in 2ab88675, maybe we can change this in 1.9.



> Please can you confirm that our understanding of HAProxy documentation is correct?
> If so, then we could mark this topic as "solved" :-)

Yes, but I don't understand, you reported that verification is not
happening *with* verifyhost:

> the connection to the backend works all the time, even when there is a name mismatch and even if we use the “verify required” option together with “verifyhost”.


"verify required ssl verifyhost www.ham.eggs" fails as expected for
you now, correct?



Thanks,
Lukas
Hi Lukas,

Right, "verify required ssl verifyhost www.ham.eggs" fails now as expected.

My initial report that it doesn't work with "verifyhost" option was not completely right,
because in fact we never tried what would happen if we set a non-matching pattern in the "verifyhost" directive.

We tested without the verifyhost because of lack of knowledge that this would be mandatory. Then it always worked (even with the name mismatch),
And we tested with verifyhost directive, but only with a matching pattern, also always worked.

Now with the new information, it's clear why this always worked and what we have to do to achieve a correct haproxy config.
-> always use "verify required" *together* with "verifyhost"

I would also vote to change the HAProxy default behavior to more security-oriented when there are some directives not passed.
This would on the one hand generate more questions "why is it not working?", but on the other hand would have a stronger security out of the box.

Thanks for your help!

BR
Martin

-----Original Message-----
From: lukas@ltri.eu [mailto:[email protected]]
Sent: Montag, 16. Juli 2018 14:11
To: Martin RADEL <[email protected]>
Cc: haproxy@formilux.org; [email protected]; manu@gandi.net
Subject: Re: TLS handshake works with certificate name mismatch using "verify required" and "verifyhost"

On Mon, 16 Jul 2018 at 11:57, Martin RADEL <mailto:[email protected]> wrote:
>
> Hi,
>
> I think we found the issue:
> Seems that there was a misunderstanding from us regarding the haproxy documentation with the "verifyhost" option.
>
> If I get it right, the documentation says that if we have a haproxy
> config that
> - Has "verify required"
> - Does not use SNI
> - Has no "verifyhost"
> Then HAProxy will simply ignore whatever hostname the server sends back in its certificate and the handshake will be OK.

Yes, that is correct, also see the verify docs:
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-verify

Not sure how we ended up in this situation though. I remember there was a vivid discussion about whether "verify" should default to none or required. We opted for "required", to be "secure by default", but this is totally useless given that it requires verifyhost or sni, and will silently disable cert verification when those option are not given. That's probably the worst thing we can do in this case; this configuration should be rejected, imho. People that don't care about cert verification should simply set "verify none". But here we are now, and this is documented behavior :(

I think this was introduced in 2ab88675, maybe we can change this in 1.9.



> Please can you confirm that our understanding of HAProxy documentation is correct?
> If so, then we could mark this topic as "solved" :-)

Yes, but I don't understand, you reported that verification is not happening *with* verifyhost:

> the connection to the backend works all the time, even when there is a name mismatch and even if we use the “verify required” option together with “verifyhost”.


"verify required ssl verifyhost www.ham.eggs" fails as expected for you now, correct?



Thanks,
Lukas
This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to § 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 Vienna,Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien).
Sorry, only registered users may post in this forum.

Click here to login