Welcome! Log In Create A New Profile

Advanced

Certificate order

Posted by Sander Hoentjen 
Sander Hoentjen
Certificate order
April 06, 2017 10:50AM
Hi guys,

We have a setup where we sometimes have multiple certificates for a
domain. We use multiple directories for that and would like the
following behavior:
- Look in dir A for any match, use it if found
- Look in dir B for any match, use it if found
- Look in dir .. etc

This works great, except for wildcards. Right now a domain match in dir
B takes precedence over a wildcard match in dir A.

Is there a way to get haproxy to behave the way I describe?

Regards,
Sander
Sander Klein
Re: Certificate order
April 06, 2017 02:20PM
Hi Sander,

On 2017-04-06 10:45, Sander Hoentjen wrote:
> Hi guys,
>
> We have a setup where we sometimes have multiple certificates for a
> domain. We use multiple directories for that and would like the
> following behavior:
> - Look in dir A for any match, use it if found
> - Look in dir B for any match, use it if found
> - Look in dir .. etc
>
> This works great, except for wildcards. Right now a domain match in dir
> B takes precedence over a wildcard match in dir A.
>
> Is there a way to get haproxy to behave the way I describe?

I have been playing with this some time ago and my solution was to just
think about the order of certificate loading. I then found out that the
last certificate was preferred if it matched. Not sure if this has
changed over time.

Greets,

Sander
Sander Hoentjen
Re: Certificate order
April 07, 2017 03:50PM
Hi Sander,

On 04/06/2017 02:06 PM, Sander Klein wrote:
> Hi Sander,
>
> On 2017-04-06 10:45, Sander Hoentjen wrote:
>> Hi guys,
>>
>> We have a setup where we sometimes have multiple certificates for a
>> domain. We use multiple directories for that and would like the
>> following behavior:
>> - Look in dir A for any match, use it if found
>> - Look in dir B for any match, use it if found
>> - Look in dir .. etc
>>
>> This works great, except for wildcards. Right now a domain match in dir
>> B takes precedence over a wildcard match in dir A.
>>
>> Is there a way to get haproxy to behave the way I describe?
>
> I have been playing with this some time ago and my solution was to
> just think about the order of certificate loading. I then found out
> that the last certificate was preferred if it matched. Not sure if
> this has changed over time.
This does not work for wildcard certs, it seems they are always tried last.

Regards,
Sander
Sander Hoentjen
Re: Certificate order
April 10, 2017 05:10PM
The attached patch against haproxy 1.7.5 honours crt order also for
wildcards.

On 04/07/2017 03:42 PM, Sander Hoentjen wrote:
> Hi Sander,
>
> On 04/06/2017 02:06 PM, Sander Klein wrote:
>> Hi Sander,
>>
>> On 2017-04-06 10:45, Sander Hoentjen wrote:
>>> Hi guys,
>>>
>>> We have a setup where we sometimes have multiple certificates for a
>>> domain. We use multiple directories for that and would like the
>>> following behavior:
>>> - Look in dir A for any match, use it if found
>>> - Look in dir B for any match, use it if found
>>> - Look in dir .. etc
>>>
>>> This works great, except for wildcards. Right now a domain match in dir
>>> B takes precedence over a wildcard match in dir A.
>>>
>>> Is there a way to get haproxy to behave the way I describe?
>> I have been playing with this some time ago and my solution was to
>> just think about the order of certificate loading. I then found out
>> that the last certificate was preferred if it matched. Not sure if
>> this has changed over time.
> This does not work for wildcard certs, it seems they are always tried last.
>
> Regards,
> Sander
>
Sander Hoentjen
Re: Certificate order
April 10, 2017 05:10PM
No scratch that, this is wrong.

On 04/10/2017 04:57 PM, Sander Hoentjen wrote:
> The attached patch against haproxy 1.7.5 honours crt order also for
> wildcards.
>
> On 04/07/2017 03:42 PM, Sander Hoentjen wrote:
>> Hi Sander,
>>
>> On 04/06/2017 02:06 PM, Sander Klein wrote:
>>> Hi Sander,
>>>
>>> On 2017-04-06 10:45, Sander Hoentjen wrote:
>>>> Hi guys,
>>>>
>>>> We have a setup where we sometimes have multiple certificates for a
>>>> domain. We use multiple directories for that and would like the
>>>> following behavior:
>>>> - Look in dir A for any match, use it if found
>>>> - Look in dir B for any match, use it if found
>>>> - Look in dir .. etc
>>>>
>>>> This works great, except for wildcards. Right now a domain match in dir
>>>> B takes precedence over a wildcard match in dir A.
>>>>
>>>> Is there a way to get haproxy to behave the way I describe?
>>> I have been playing with this some time ago and my solution was to
>>> just think about the order of certificate loading. I then found out
>>> that the last certificate was preferred if it matched. Not sure if
>>> this has changed over time.
>> This does not work for wildcard certs, it seems they are always tried last.
>>
>> Regards,
>> Sander
>>
Sander Hoentjen
Re: Certificate order
April 10, 2017 08:10PM
This is a corrected patch against 1.7.5.

On 04/10/2017 05:00 PM, Sander Hoentjen wrote:
> No scratch that, this is wrong.
>
> On 04/10/2017 04:57 PM, Sander Hoentjen wrote:
>> The attached patch against haproxy 1.7.5 honours crt order also for
>> wildcards.
>>
>> On 04/07/2017 03:42 PM, Sander Hoentjen wrote:
>>> Hi Sander,
>>>
>>> On 04/06/2017 02:06 PM, Sander Klein wrote:
>>>> Hi Sander,
>>>>
>>>> On 2017-04-06 10:45, Sander Hoentjen wrote:
>>>>> Hi guys,
>>>>>
>>>>> We have a setup where we sometimes have multiple certificates for a
>>>>> domain. We use multiple directories for that and would like the
>>>>> following behavior:
>>>>> - Look in dir A for any match, use it if found
>>>>> - Look in dir B for any match, use it if found
>>>>> - Look in dir .. etc
>>>>>
>>>>> This works great, except for wildcards. Right now a domain match in dir
>>>>> B takes precedence over a wildcard match in dir A.
>>>>>
>>>>> Is there a way to get haproxy to behave the way I describe?
>>>> I have been playing with this some time ago and my solution was to
>>>> just think about the order of certificate loading. I then found out
>>>> that the last certificate was preferred if it matched. Not sure if
>>>> this has changed over time.
>>> This does not work for wildcard certs, it seems they are always tried last.
>>>
>>> Regards,
>>> Sander
>>>
>
Daniel Schneller
Re: Certificate order
April 18, 2017 11:40AM
Hi!

Not being very familiar with the code, so I thought I’d ask before something changes unexpectedly :)
I asked about certificate ordering a while ago, too, and I seem to remember (and we currently rely on this) that exact domain matches are “weighted higher” than wildcard matches on purpose, so that if I just dump the certificates in a directory, it will pick a more specific one over a wildcard that is also there as a “catchall”.

Not saying one or the other is right or wrong, but if this should be merged, it must be made very clear that people might have to change their setups.

Daniel



--
Daniel Schneller
Principal Cloud Engineer

CenterDevice GmbH | Hochstraße 11
| 42697 Solingen
tel: +49 1754155711 | Deutschland
daniel.schneller@centerdevice.de | www.centerdevice.de

Geschäftsführung: Dr. Patrick Peschlow, Dr. Lukas Pustina,
Michael Rosbach, Handelsregister-Nr.: HRB 18655,
HR-Gericht: Bonn, USt-IdNr.: DE-815299431


> On 10. Apr. 2017, at 20:02, Sander Hoentjen <[email protected]> wrote:
>
> This is a corrected patch against 1.7.5.
>
> On 04/10/2017 05:00 PM, Sander Hoentjen wrote:
>> No scratch that, this is wrong.
>>
>> On 04/10/2017 04:57 PM, Sander Hoentjen wrote:
>>> The attached patch against haproxy 1.7.5 honours crt order also for
>>> wildcards.
>>>
>>> On 04/07/2017 03:42 PM, Sander Hoentjen wrote:
>>>> Hi Sander,
>>>>
>>>> On 04/06/2017 02:06 PM, Sander Klein wrote:
>>>>> Hi Sander,
>>>>>
>>>>> On 2017-04-06 10:45, Sander Hoentjen wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> We have a setup where we sometimes have multiple certificates for a
>>>>>> domain. We use multiple directories for that and would like the
>>>>>> following behavior:
>>>>>> - Look in dir A for any match, use it if found
>>>>>> - Look in dir B for any match, use it if found
>>>>>> - Look in dir .. etc
>>>>>>
>>>>>> This works great, except for wildcards. Right now a domain match in dir
>>>>>> B takes precedence over a wildcard match in dir A.
>>>>>>
>>>>>> Is there a way to get haproxy to behave the way I describe?
>>>>> I have been playing with this some time ago and my solution was to
>>>>> just think about the order of certificate loading. I then found out
>>>>> that the last certificate was preferred if it matched. Not sure if
>>>>> this has changed over time.
>>>> This does not work for wildcard certs, it seems they are always tried last.
>>>>
>>>> Regards,
>>>> Sander
>>>>
>>
>
> <haproxy-wildcards.patch>
Sander Hoentjen
Re: Certificate order
April 18, 2017 12:00PM
Hi Daniel,

Yes, I understand your concern. I don't know if haproxy developers are
willing to accept this change. Personally I think it is a good idea,
because as it is now a sysadmin cannot ensure ordering of a specific
wildcard before some domain specific one, whereas with my patch you are
in full control (if you want it to have less preference, just make sure
it is loaded later).

Regards,
Sander

On 04/18/2017 11:25 AM, Daniel Schneller wrote:
> Hi!
>
> Not being very familiar with the code, so I thought I’d ask before
> something changes unexpectedly :)
> I asked about certificate ordering a while ago, too, and I seem to
> remember (and we currently rely on this) that exact domain matches are
> “weighted higher” than wildcard matches on purpose, so that if I just
> dump the certificates in a directory, it will pick a more specific one
> over a wildcard that is also there as a “catchall”.
>
> Not saying one or the other is right or wrong, but if this should be
> merged, it must be made very clear that people might have to change
> their setups.
>
> Daniel
>
>
>
> --
> Daniel Schneller
> Principal Cloud Engineer
>
> CenterDevice GmbH | Hochstraße 11
> | 42697 Solingen
> tel: +49 1754155711 | Deutschland
> daniel.schneller@centerdevice.de
> <mailto:[email protected]> | www.centerdevice.de
> http://www.centerdevice.de
>
> Geschäftsführung: Dr. Patrick Peschlow, Dr. Lukas Pustina,
> Michael Rosbach, Handelsregister-Nr.: HRB 18655,
> HR-Gericht: Bonn, USt-IdNr.: DE-815299431
>
>
>> On 10. Apr. 2017, at 20:02, Sander Hoentjen <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> This is a corrected patch against 1.7.5.
>>
>> On 04/10/2017 05:00 PM, Sander Hoentjen wrote:
>>> No scratch that, this is wrong.
>>>
>>> On 04/10/2017 04:57 PM, Sander Hoentjen wrote:
>>>> The attached patch against haproxy 1.7.5 honours crt order also for
>>>> wildcards.
>>>>
>>>> On 04/07/2017 03:42 PM, Sander Hoentjen wrote:
>>>>> Hi Sander,
>>>>>
>>>>> On 04/06/2017 02:06 PM, Sander Klein wrote:
>>>>>> Hi Sander,
>>>>>>
>>>>>> On 2017-04-06 10:45, Sander Hoentjen wrote:
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> We have a setup where we sometimes have multiple certificates for a
>>>>>>> domain. We use multiple directories for that and would like the
>>>>>>> following behavior:
>>>>>>> - Look in dir A for any match, use it if found
>>>>>>> - Look in dir B for any match, use it if found
>>>>>>> - Look in dir .. etc
>>>>>>>
>>>>>>> This works great, except for wildcards. Right now a domain match
>>>>>>> in dir
>>>>>>> B takes precedence over a wildcard match in dir A.
>>>>>>>
>>>>>>> Is there a way to get haproxy to behave the way I describe?
>>>>>> I have been playing with this some time ago and my solution was to
>>>>>> just think about the order of certificate loading. I then found out
>>>>>> that the last certificate was preferred if it matched. Not sure if
>>>>>> this has changed over time.
>>>>> This does not work for wildcard certs, it seems they are always
>>>>> tried last.
>>>>>
>>>>> Regards,
>>>>> Sander
>>>>>
>>>
>>
>> <haproxy-wildcards.patch>
>
Willy Tarreau
Re: Certificate order
April 18, 2017 12:00PM
Hi Daniel,

On Tue, Apr 18, 2017 at 11:25:43AM +0200, Daniel Schneller wrote:
> Hi!
>
> Not being very familiar with the code, so I thought I'd ask before something
> changes unexpectedly :)
> I asked about certificate ordering a while ago, too, and I seem to remember
> (and we currently rely on this) that exact domain matches are "weighted
> higher" than wildcard matches on purpose, so that if I just dump the
> certificates in a directory, it will pick a more specific one over a wildcard
> that is also there as a "catchall".
>
> Not saying one or the other is right or wrong, but if this should be merged,
> it must be made very clear that people might have to change their setups.

FQDN matches always have precedence over wildcards (fortunately). Sander,
I'm a bit surprized by your motivation for this change. You always want
foo.example.com to have precedence over *.example.com and this is not a
matter of directory. By changing this you'd silently break some certs by
presenting the wrong one (the wildcard one) instead of the fully qualified
name. If you have any reason for wanting to do that anyway, I think this
is the wrong approach and you should instead refuse to load domain certs
when they conflict with a wildcard, or at least emit a warning indicating
that there's overlapping. But that still seems very strange to me :-/

Willy
Sander Hoentjen
Re: Certificate order
April 18, 2017 12:20PM
On 04/18/2017 11:52 AM, Willy Tarreau wrote:
> Hi Daniel,
>
> On Tue, Apr 18, 2017 at 11:25:43AM +0200, Daniel Schneller wrote:
>> Hi!
>>
>> Not being very familiar with the code, so I thought I'd ask before something
>> changes unexpectedly :)
>> I asked about certificate ordering a while ago, too, and I seem to remember
>> (and we currently rely on this) that exact domain matches are "weighted
>> higher" than wildcard matches on purpose, so that if I just dump the
>> certificates in a directory, it will pick a more specific one over a wildcard
>> that is also there as a "catchall".
>>
>> Not saying one or the other is right or wrong, but if this should be merged,
>> it must be made very clear that people might have to change their setups.
> FQDN matches always have precedence over wildcards (fortunately). Sander,
> I'm a bit surprized by your motivation for this change. You always want
> foo.example.com to have precedence over *.example.com and this is not a
> matter of directory. By changing this you'd silently break some certs by
> presenting the wrong one (the wildcard one) instead of the fully qualified
> name. If you have any reason for wanting to do that anyway, I think this
> is the wrong approach and you should instead refuse to load domain certs
> when they conflict with a wildcard, or at least emit a warning indicating
> that there's overlapping. But that still seems very strange to me :-/
Hi Willy,

In our case we always request Let's Encrypt certificates for all our
customers. We put those in a directory that is loaded last. When a
customer buys a certificate himself this certificate is put in a
directory that is loaded before the Let's Encrypt ones. If a customer
has bought a certificate he always wants that certificate to be used,
even if it is a wildcard. If and when a customer removes his own cert,
he will always still have working SSL.

Regards,
Sander
Sander Hoentjen
Re: Certificate order
April 20, 2017 03:10PM
A new patch, that puts the order like this:
config:
crt A crt B

if A contains wildcard, but not exact match, then wildcard is used.
if A contains exact match, exact match is used.
(this also means that if A contains both wildcard and exact match, exact
match is used.)
If A contains wildcard, and B contains exact match, then wildcard is used.

This last one is different behavior from what is implemented now.





On 04/18/2017 12:09 PM, Sander Hoentjen wrote:
>
> On 04/18/2017 11:52 AM, Willy Tarreau wrote:
>> Hi Daniel,
>>
>> On Tue, Apr 18, 2017 at 11:25:43AM +0200, Daniel Schneller wrote:
>>> Hi!
>>>
>>> Not being very familiar with the code, so I thought I'd ask before something
>>> changes unexpectedly :)
>>> I asked about certificate ordering a while ago, too, and I seem to remember
>>> (and we currently rely on this) that exact domain matches are "weighted
>>> higher" than wildcard matches on purpose, so that if I just dump the
>>> certificates in a directory, it will pick a more specific one over a wildcard
>>> that is also there as a "catchall".
>>>
>>> Not saying one or the other is right or wrong, but if this should be merged,
>>> it must be made very clear that people might have to change their setups.
>> FQDN matches always have precedence over wildcards (fortunately). Sander,
>> I'm a bit surprized by your motivation for this change. You always want
>> foo.example.com to have precedence over *.example.com and this is not a
>> matter of directory. By changing this you'd silently break some certs by
>> presenting the wrong one (the wildcard one) instead of the fully qualified
>> name. If you have any reason for wanting to do that anyway, I think this
>> is the wrong approach and you should instead refuse to load domain certs
>> when they conflict with a wildcard, or at least emit a warning indicating
>> that there's overlapping. But that still seems very strange to me :-/
> Hi Willy,
>
> In our case we always request Let's Encrypt certificates for all our
> customers. We put those in a directory that is loaded last. When a
> customer buys a certificate himself this certificate is put in a
> directory that is loaded before the Let's Encrypt ones. If a customer
> has bought a certificate he always wants that certificate to be used,
> even if it is a wildcard. If and when a customer removes his own cert,
> he will always still have working SSL.
>
> Regards,
> Sander
>
Sander Hoentjen
Re: Certificate order
April 20, 2017 06:30PM
Now with patch attached, thanks Fred :)

On 04/20/2017 03:05 PM, Sander Hoentjen wrote:
> A new patch, that puts the order like this:
> config:
> crt A crt B
>
> if A contains wildcard, but not exact match, then wildcard is used.
> if A contains exact match, exact match is used.
> (this also means that if A contains both wildcard and exact match, exact
> match is used.)
> If A contains wildcard, and B contains exact match, then wildcard is used.
>
> This last one is different behavior from what is implemented now.
>
>
>
>
>
> On 04/18/2017 12:09 PM, Sander Hoentjen wrote:
>> On 04/18/2017 11:52 AM, Willy Tarreau wrote:
>>> Hi Daniel,
>>>
>>> On Tue, Apr 18, 2017 at 11:25:43AM +0200, Daniel Schneller wrote:
>>>> Hi!
>>>>
>>>> Not being very familiar with the code, so I thought I'd ask before something
>>>> changes unexpectedly :)
>>>> I asked about certificate ordering a while ago, too, and I seem to remember
>>>> (and we currently rely on this) that exact domain matches are "weighted
>>>> higher" than wildcard matches on purpose, so that if I just dump the
>>>> certificates in a directory, it will pick a more specific one over a wildcard
>>>> that is also there as a "catchall".
>>>>
>>>> Not saying one or the other is right or wrong, but if this should be merged,
>>>> it must be made very clear that people might have to change their setups.
>>> FQDN matches always have precedence over wildcards (fortunately). Sander,
>>> I'm a bit surprized by your motivation for this change. You always want
>>> foo.example.com to have precedence over *.example.com and this is not a
>>> matter of directory. By changing this you'd silently break some certs by
>>> presenting the wrong one (the wildcard one) instead of the fully qualified
>>> name. If you have any reason for wanting to do that anyway, I think this
>>> is the wrong approach and you should instead refuse to load domain certs
>>> when they conflict with a wildcard, or at least emit a warning indicating
>>> that there's overlapping. But that still seems very strange to me :-/
>> Hi Willy,
>>
>> In our case we always request Let's Encrypt certificates for all our
>> customers. We put those in a directory that is loaded last. When a
>> customer buys a certificate himself this certificate is put in a
>> directory that is loaded before the Let's Encrypt ones. If a customer
>> has bought a certificate he always wants that certificate to be used,
>> even if it is a wildcard. If and when a customer removes his own cert,
>> he will always still have working SSL.
>>
>> Regards,
>> Sander
>>
>
Lukas Tribus
Re: Certificate order
April 20, 2017 11:40PM
Hello,


Am 20.04.2017 um 15:05 schrieb Sander Hoentjen:
> A new patch, that puts the order like this:
> config:
> crt A crt B
>
> [...]
> If A contains wildcard, and B contains exact match, then wildcard is used.
>
> This last one is different behavior from what is implemented now.

People rely on the specific current behavior you are changing already.
We made it clear that exact matches always take precedence; inverting
the behavior fixes your single use-case but breaks many others.


My opinion about your use-case is that your provisioning layer has to
handle this and remove, if necessary, any more specific and unwanted
lets-encrypt certificates.
Maybe the crt-list [1] feature can be an alternative, otherwise, what
you'd need from haproxy would be a multi-layered approach, with
certificate "weights", which I don't believe makes sense to implement
(but should be abstracted into provisioning tools).


Regards,
Lukas

[1]
https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.1-crt-list
Willy Tarreau
Re: Certificate order
April 21, 2017 07:40AM
On Thu, Apr 20, 2017 at 11:33:17PM +0200, Lukas Tribus wrote:
> Hello,
>
>
> Am 20.04.2017 um 15:05 schrieb Sander Hoentjen:
> > A new patch, that puts the order like this:
> > config:
> > crt A crt B
> >
> > [...]
> > If A contains wildcard, and B contains exact match, then wildcard is used.
> >
> > This last one is different behavior from what is implemented now.
>
> People rely on the specific current behavior you are changing already.
> We made it clear that exact matches always take precedence; inverting the
> behavior fixes your single use-case but breaks many others.
>
>
> My opinion about your use-case is that your provisioning layer has to handle
> this and remove, if necessary, any more specific and unwanted lets-encrypt
> certificates.
> Maybe the crt-list [1] feature can be an alternative, otherwise, what you'd
> need from haproxy would be a multi-layered approach, with certificate
> "weights", which I don't believe makes sense to implement (but should be
> abstracted into provisioning tools).

+1

Willy
Sander Hoentjen
Re: Certificate order
April 21, 2017 11:10AM
On 04/21/2017 07:27 AM, Willy Tarreau wrote:
> On Thu, Apr 20, 2017 at 11:33:17PM +0200, Lukas Tribus wrote:
>> Hello,
>>
>>
>> Am 20.04.2017 um 15:05 schrieb Sander Hoentjen:
>>> A new patch, that puts the order like this:
>>> config:
>>> crt A crt B
>>>
>>> [...]
>>> If A contains wildcard, and B contains exact match, then wildcard is used.
>>>
>>> This last one is different behavior from what is implemented now.
>> People rely on the specific current behavior you are changing already.
>> We made it clear that exact matches always take precedence; inverting the
>> behavior fixes your single use-case but breaks many others.
>>
>>
>> My opinion about your use-case is that your provisioning layer has to handle
>> this and remove, if necessary, any more specific and unwanted lets-encrypt
>> certificates.
>> Maybe the crt-list [1] feature can be an alternative, otherwise, what you'd
>> need from haproxy would be a multi-layered approach, with certificate
>> "weights", which I don't believe makes sense to implement (but should be
>> abstracted into provisioning tools).
> +1
Well, in essence certificate "weights" is what I implemented in the
patch, with the config order determining the weight. While unfortunate
for me, I can understand your reasoning. I do still think my approach
offers more flexibility. Anyway, I attached a new patch which I hope you
will consider :)
As for my own use case, I'll just carry the previous patch myself.
In my case I can't just simply remove a (multi-domain) certificate,
since they may contain names that are not covered by the wildcard. This
means I need to use a neg filter, but I can't use a directory in
crt-list, so I need to specify all certificates in the list. While not a
very large issue it adds unneeded complexity, with a chance for things
to go out of sync.
In the end I would like to say that while I hoped my patch would be
accepted, I appreciate the responses on this mailing list. I prefer
being told no to being ignored. Thanks for this awesome project with a
healthy community. Maybe in the future I can contribute something else :)

Regards,
Sander
Aleksandar Lazic
Re: Certificate order
April 23, 2017 11:30AM
Hi.

Am 21-04-2017 11:02, schrieb Sander Hoentjen:

[snipp]

> Well, in essence certificate "weights" is what I implemented in the
> patch, with the config order determining the weight. While unfortunate
> for me, I can understand your reasoning. I do still think my approach
> offers more flexibility. Anyway, I attached a new patch which I hope
> you
> will consider :)

[snipp]

I was able to patch successfully the 1.8 & 1.7 with your patch ;-)

Regards
Aleks
Sorry, only registered users may post in this forum.

Click here to login