Welcome! Log In Create A New Profile

Advanced

[PHP-DEV] hash_hkdf() signature and return value

Posted by Yasuo Ohgaki 
Yasuo Ohgaki
[PHP-DEV] hash_hkdf() signature and return value
September 06, 2017 03:20AM
Hi all,

This is the last recommendation for hash_hkdf[1]. In fact,
this would be the last chance to fix because we'll have 7.2 soon.
The issue is secure usage and API consistency.

Currently hash_hkdf() has following signature:

<binary string> hash_hkdf(string $algo , string $ikm [, int $length = 0 [,
string $info = '' [, string $salt = '' ]]] )

These are rationals behind recommendation. There are many, but
please read on.

=== Parameter Order ===

HKDF[2] algorithm is:
1. General purpose key derivation function as per RFC 5869
2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned
RFC 5869
3. "$salt" (or preshared key) is very strongly recommended for security
season as per RFC 5869
4. Supplying salt that the same length of input key does not affect
performance as per RFC 5969
5. "$info" is what makes HKDF useful, but it's less important as described
in RFC 5869
6. "$length" is truly an optional parameter for very specific encryption
algorithm or usage.

Rationale for change:
1. Key derivations without secondary key ($salt) does not make sense when
secondary key could be available. HKDF is designed for best possible
key
security with the key. Not using secondary key ($salt) simply
downgrades
key security without no reason. i.e. HKDF performance is the same
when $salt has the same as hash is set.
2. HKDF is based on HMAC. When $info has no use, HMAC would be the best
choice for it. i.e. $newkey = hash_hmac($ikm, $key);
3. It does not make sense violating RFC recommendations for a RFC
implementation.

From these facts and reasons, $salt, $info and $length parameter order and
requirement should be changed from

string $algo , string $ikm [, int $length = 0 [, string $info = '' [,
string $salt = '' ]]]

to

string $algo , string $ikm , string $salt, string $info = '' [, int $length
= 0 ]
Note: Users can set empty string if they really don't need $salt and/or
$info.

Conclusion:
This way, users would have better chances to use hash_hkdf() more securely
and
properly.

[1] http://php.net/hash_hkdf
[2] http://www.faqs.org/rfcs/rfc5869.html

=== Return Value and Output Option ===

The most common HKDF usage with PHP would be:
1. CSRF token generation that is specific to a request with expiration
time.
(HEX return value would be appropriate, not BINARY)
2. API access token generation that does not transmit "The API Key", but
derived key by HKDF. It also should have expiration time.
(HEX return value would be appropriate, not BINARY)

Consistency with other hash_*() functions:
1. All of other hash_*() returns HEX string hash value.
2. hash_hkdf() is the only one returns BINARY hash value.

Conclusion:
hash_hkdf() should return HEX by default, not BINARY.
Optional [, bool $raw_output = false ] should be added just like other
hash_*().


=== Compatibility ===

IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0
in the first place. The mess could be resolved by 7.2.

Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are
using it yet. If we change API with 7.2 release, there would be least
possible confusions. (We may remove it from 7.1 to avoid confusions, too)

Our choices:
- Keep the current insecure/inconsistent API forever.
- Change the API to have secure/consistent API forever.

Conclusion:
No conclusion for this. There would be conflicting options.


I strongly think it is not worth to keep this insecure/inconsistent API
forever.
I prefer to change the API to what it should be.

What should we do for this?
Comments?



P.S.

Nikita, you've said following during HKDF discussion:
- HKDF for CSRF tokens/etc does not make sense at all.
- Current parameter order and return value makes perfect sense
and has valid/common usages.
- Everyone one this list, shouldn't listen to me because I'm insane and
totally misunderstood what the HKDF is.

Phrases are not the exact, but it should be correct enough. Some part
is my fault, w/o reading mail and/or poor English. I blindly assumed
you've understand RFC 5869 and possible common usages with PHP. I
apologized for the confusion and tried to explain why w/o success.

If you still believe what you've said is correct, I don't request you to
take these back, show us at least one common/reasonable hash_hkdf()
usage example for current API, and point out what's wrong my recommendations
and rationales above.

If not, I request you to take it back. I respect your contributions much,
but
the last one you've said is out of tolerance.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Dan Ackroyd
Re: [PHP-DEV] hash_hkdf() signature and return value
September 06, 2017 01:10PM
On 6 September 2017 at 02:15, Yasuo Ohgaki <[email protected]> wrote:
> What should we do for this?

Not us, you.

You should start listening to other people's feedback.

You continually refuse to accept any feedback that doesn't agree with
your world-view, not only on the subject of hkdf, but on validation
and other things you think are "MANDATORY"

You should respect RFC votes and stop bringing up the same discussions
over and over again. This is incredibly tedious.

In particular your suggestion about hash_hkdf was rejected
unanimously, apart from your own vote
https://wiki.php.net/rfc/improve_hash_hkdf_parameter and so probably
shouldn't be brought up for discussion, except if you can bring new
facts for discussion.

"Not liking the result of the vote" is not a new fact to discuss.

Additionally though, your ideas about adding validation/filter
functions as a C extension, rather than implementing them in PHP have
also been resoundingly rejected,
https://wiki.php.net/rfc/add_validate_functions_to_filter and yet you
continue to promote the idea. This is also tedious.

This pattern of behaviour, continually refusing to accept that other
people have opinions that don't align with yours, and continually
bringing up the same topics for discussion over, and over, and over
again is not productive. It does not make people want to engage you in
discussion, as it is a waste of their time. This is not something
other people can fix for you.

cheers
Dan
Ack

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] hash_hkdf() signature and return value
September 07, 2017 07:50AM
Hi Dan,

I appreciate your feedback, regardless of your opinion towards this issue.

On Wed, Sep 6, 2017 at 8:04 PM, Dan Ackroyd <[email protected]> wrote:

> On 6 September 2017 at 02:15, Yasuo Ohgaki <[email protected]> wrote:
> > What should we do for this?
>
> Not us, you.
>

OK. It is recorded that you think current API is totally valid.
I'm not sure who is "us". Please reply and record your opinion.


>
> You should start listening to other people's feedback.
>
> You continually refuse to accept any feedback that doesn't agree with
> your world-view, not only on the subject of hkdf, but on validation
> and other things you think are "MANDATORY"
>

Well, my thoughts are not totally my inventions.

For HKDF, it came from the RFC 5689 basically.

For input validations, it is originated in Defensive Programming. Defensive
Programming is referred in early 90's AFAIK, it is called secure programming
or secure coding now. I believe the basic idea was developed 60's computer
scientists who researched program execution correctness verification
methods.

You should respect RFC votes and stop bringing up the same discussions
> over and over again. This is incredibly tedious.
>

Did you really read the RFC 5689?
Please mention what is wrong with my statements if you think my statements
are totally wrong. It should be easy to point it out, since I provided
many points.
This is technical list, not political list nor religious list after all.

I should note that no valid code example was presented that justifies
current API.

Not a single valid code example, yet.

This fact infers what you say "us" have concrete reason(s) that supports
current API validity.

Please provide undisclosed valid code example that would be more common
than CSRF token and API token derivations. There are even more URL singing,
etc in
my PHP RFC.


> In particular your suggestion about hash_hkdf was rejected
> unanimously, apart from your own vote
> https://wiki.php.net/rfc/improve_hash_hkdf_parameter and so probably
> shouldn't be brought up for discussion, except if you can bring new
> facts for discussion.
>

If any one of you provided example usages that would be common, valid
(and optimal, it should be optimal since HKDF is designed for the best
possible derived key with HMAC), I would not raise this issue again.


>
> "Not liking the result of the vote" is not a new fact to discuss.
>

Sorry but, I'm not liking the vote result.
I'm frustrating the fact there is no code example(s) justifies current API
design
that is insecure and inconsistent.


Additionally though, your ideas about adding validation/filter
> functions as a C extension, rather than implementing them in PHP have
> also been resoundingly rejected,
> https://wiki.php.net/rfc/add_validate_functions_to_filter and yet you
> continue to promote the idea. This is also tedious.
>

It's totally new module.
I already replied to your comments in other thread.
Apparently, you are ignoring single responsibility principle.
Please take into the principle into your thoughts.

This pattern of behaviour, continually refusing to accept that other
> people have opinions that don't align with yours, and continually
> bringing up the same topics for discussion over, and over, and over
> again is not productive. It does not make people want to engage you in
> discussion, as it is a waste of their time. This is not something
> other people can fix for you.
>

As I wrote _MANY_ times in past mails.
I would have stopped discussion, if there are example codes that
justify the API.

Sorry for repeating requests, but this is technical list and no evidence
is provided.

If you really think my statements are wrong. Please comment
each line by replying "wrong" so that your idea becomes more clear.
I shouldn't difficult.

I provided more than handful valid use cases in my PHP RFC.
The technical evidence should not be difficult.
It's just example(s).

I'm waiting.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] hash_hkdf() signature and return value
September 07, 2017 08:00AM
Hi Dan,

Sorry for keep posting broken English.

I shouldn't difficult.
>

It shouldn't be difficult.

Looking forward more than handful, useful and common hash_hkdf()
application
examples for PHP that justify the API. If you would not like to spend time
for working
code, just ideas are OK also.

Regards,

Yasuo
Yasuo Ohgaki
[PHP-DEV] Re: hash_hkdf() signature and return value
September 07, 2017 11:40PM
Hi RMs,

I suppose nobody can give example(s) that justify current API.
I'll leave this issue to RMs decision, since I think this result in no
conclusion.
Please consider carefully if the current API should be kept or not.
I wrote summary for discussion for you.

Regards,

Misunderstandings summary from previous discussion:

Nikita misunderstood what HKDF is made for.
He said key derivation that generates AES 128 bit key from AES 256 bit key
for
compatibility reason as follows is "textbook example" and
good(recommended?)
usecase.

$sha128key = hash_hkdf("sha256", $sha256key, 16);

Although this could be acceptable useage when salt cannot be used, but not
recommended. HKDF is specifically designed to discourage this kind of key
derivation because this is weak. The RFC strongly encourage salt use for
key security. Appropriate key derivation is

$sha128key = hash_hkdf("sha256", $sha256key, 32, "",
$csprng_generated_random_key);
// Store $cspring_generated_ random_key to somewhere.


The reason why latter is a lot more secure is related to Andrey's
misunderstanding.
He said "when ikm is cryptographically strong, salt wouldn't add no more
entropy.
so salt does not matter". (not exact sentence)
What he said partially true, but wrong in a sense of key security.

When ikm has enough entropy, HKDF does not add more entropy.
However, even when ikm is strong, additional salt adds significant
complexities
for key analysis. To make things simpler, suppose we have very simple KDF
as follows.

okm = VSKDF(ikm, salt) where
- okm, ikm, salt is 2 digit int
- when salt is NULL, default to 2
- VSKDF is simple function that multiply ikm and salt (ikm * salt) returns
last 2 digits
as derived key.

When ikm=64, salt=NULL

28 = VSKDF(64, NULL);

Since algorithm is known, attackers are able to guess ikm very easily.
Now, suppose salt=16 is provided

24 = VSKDF(64, 16);

It is still easy to guess due to very simple KDF, but guessing ikm is a lot
harder
than previous example while it does not add any entropy.
HKDF has a lot more complex algorithm, therefore salt gives significantly
stronger
key protection even when salt is disclosed as noted in the RFC.

Other misunderstanding should be noted is what HKDF for. It's for general
purpose KDF as the RFC described in HKDF application section. Andrey said
"I'm cherry picking sentence", but the section should be what the HKDF for.
The section even describes obscure usage, HKDF for CSPRNG. This usage
is not even key derivation.

This one I'm not sure misunderstanding or not, but probably it is.
HKDF is designed for any ikm and works with appropriate usage. Very
weak ikm like user entered password can be handled relatively safely.

$safe_key = hash_hkdf("sha256", 'mypassword', 0, '',
$csprng_generated_random_key);
// $csprng_generated_random_key should be kept secret because ikm is too
weak

Without salt, it's disaster. Please note that salt is the last optional
parameter currently.

$dangerous_key = hash_hkdf("sha256", 'mypassword'); // Disaster!

With this usage, attackers can build pre hashed dictionary. Even when they
don't have
dictionary, usual brute force attack is very effective. One may think
additional hashing
would mitigate risk. i.e.

$dangerous_key = hash_hkdf("sha256", hash("sha256", 'mypassword')); //
Disaster!

This does not help much when algorithm is known to attackers. Brute force
attack is
effective still. Adding secret salt(key) helps with this usage also.

Please remember crypt() abuse history. We should imagine similar thing
could happen
with hash_hkdf() API. Current API encourages misuse/abuse too much by making
salt the last optional parameter.

Better documentation may help, "Users must use random $salt always whenever
it is possible even if it is the last optional parameter.".
However, do we really keep this mess forever? for a new function?


(BTW, Nikita, are you going to take your statement back or not. It's too
impolite)


--
Yasuo Ohgaki
yohgaki@ohgaki.net

On Wed, Sep 6, 2017 at 10:15 AM, Yasuo Ohgaki <[email protected]> wrote:

> Hi all,
>
> This is the last recommendation for hash_hkdf[1]. In fact,
> this would be the last chance to fix because we'll have 7.2 soon.
> The issue is secure usage and API consistency.
>
> Currently hash_hkdf() has following signature:
>
> <binary string> hash_hkdf(string $algo , string $ikm [, int $length = 0 [,
> string $info = '' [, string $salt = '' ]]] )
>
> These are rationals behind recommendation. There are many, but
> please read on.
>
> === Parameter Order ===
>
> HKDF[2] algorithm is:
> 1. General purpose key derivation function as per RFC 5869
> 2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned
> RFC 5869
> 3. "$salt" (or preshared key) is very strongly recommended for security
> season as per RFC 5869
> 4. Supplying salt that the same length of input key does not affect
> performance as per RFC 5969
> 5. "$info" is what makes HKDF useful, but it's less important as
> described in RFC 5869
> 6. "$length" is truly an optional parameter for very specific encryption
> algorithm or usage.
>
> Rationale for change:
> 1. Key derivations without secondary key ($salt) does not make sense when
> secondary key could be available. HKDF is designed for best possible
> key
> security with the key. Not using secondary key ($salt) simply
> downgrades
> key security without no reason. i.e. HKDF performance is the same
> when $salt has the same as hash is set.
> 2. HKDF is based on HMAC. When $info has no use, HMAC would be the best
> choice for it. i.e. $newkey = hash_hmac($ikm, $key);
> 3. It does not make sense violating RFC recommendations for a RFC
> implementation.
>
> From these facts and reasons, $salt, $info and $length parameter order and
> requirement should be changed from
>
> string $algo , string $ikm [, int $length = 0 [, string $info = '' [,
> string $salt = '' ]]]
>
> to
>
> string $algo , string $ikm , string $salt, string $info = '' [,
> int $length = 0 ]
> Note: Users can set empty string if they really don't need $salt and/or
> $info.
>
> Conclusion:
> This way, users would have better chances to use hash_hkdf() more securely
> and
> properly.
>
> [1] http://php.net/hash_hkdf
> [2] http://www.faqs.org/rfcs/rfc5869.html
>
> === Return Value and Output Option ===
>
> The most common HKDF usage with PHP would be:
> 1. CSRF token generation that is specific to a request with expiration
> time.
> (HEX return value would be appropriate, not BINARY)
> 2. API access token generation that does not transmit "The API Key", but
> derived key by HKDF. It also should have expiration time.
> (HEX return value would be appropriate, not BINARY)
>
> Consistency with other hash_*() functions:
> 1. All of other hash_*() returns HEX string hash value.
> 2. hash_hkdf() is the only one returns BINARY hash value.
>
> Conclusion:
> hash_hkdf() should return HEX by default, not BINARY.
> Optional [, bool $raw_output = false ] should be added just like other
> hash_*().
>
>
> === Compatibility ===
>
> IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0
> in the first place. The mess could be resolved by 7.2.
>
> Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are
> using it yet. If we change API with 7.2 release, there would be least
> possible confusions. (We may remove it from 7.1 to avoid confusions, too)
>
> Our choices:
> - Keep the current insecure/inconsistent API forever.
> - Change the API to have secure/consistent API forever.
>
> Conclusion:
> No conclusion for this. There would be conflicting options.
>
>
> I strongly think it is not worth to keep this insecure/inconsistent API
> forever.
> I prefer to change the API to what it should be.
>
> What should we do for this?
> Comments?
>
>
>
> P.S.
>
> Nikita, you've said following during HKDF discussion:
> - HKDF for CSRF tokens/etc does not make sense at all.
> - Current parameter order and return value makes perfect sense
> and has valid/common usages.
> - Everyone one this list, shouldn't listen to me because I'm insane and
> totally misunderstood what the HKDF is.
>
> Phrases are not the exact, but it should be correct enough. Some part
> is my fault, w/o reading mail and/or poor English. I blindly assumed
> you've understand RFC 5869 and possible common usages with PHP. I
> apologized for the confusion and tried to explain why w/o success.
>
> If you still believe what you've said is correct, I don't request you to
> take these back, show us at least one common/reasonable hash_hkdf()
> usage example for current API, and point out what's wrong my
> recommendations
> and rationales above.
>
> If not, I request you to take it back. I respect your contributions much,
> but
> the last one you've said is out of tolerance.
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
>
Andrey Andreev
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 01:20AM
Hi,

On Fri, Sep 8, 2017 at 12:32 AM, Yasuo Ohgaki <[email protected]> wrote:
>
> The reason why latter is a lot more secure is related to Andrey's
> misunderstanding.
> He said "when ikm is cryptographically strong, salt wouldn't add no more
> entropy.
> so salt does not matter". (not exact sentence)
> What he said partially true, but wrong in a sense of key security.
>

I have never said that. You are aware enough of your own English
fluency, and should know not to perephrase other people's words, as
you are completely twisting their meaning.

>
> Other misunderstanding should be noted is what HKDF for. It's for general
> purpose KDF as the RFC described in HKDF application section. Andrey said
> "I'm cherry picking sentence", but the section should be what the HKDF for.
> The section even describes obscure usage, HKDF for CSPRNG. This usage
> is not even key derivation.
>

You ARE cherry-picking, and ignoring all evidence that contradicts you:

> This one I'm not sure misunderstanding or not, but probably it is.
> HKDF is designed for any ikm and works with appropriate usage. Very
> weak ikm like user entered password can be handled relatively safely.
>
> $safe_key = hash_hkdf("sha256", 'mypassword', 0, '',
> $csprng_generated_random_key);
> // $csprng_generated_random_key should be kept secret because ikm is too
> weak
>
> Without salt, it's disaster. Please note that salt is the last optional
> parameter currently.
>
> $dangerous_key = hash_hkdf("sha256", 'mypassword'); // Disaster!
>
> With this usage, attackers can build pre hashed dictionary. Even when they
> don't have
> dictionary, usual brute force attack is very effective. One may think
> additional hashing
> would mitigate risk. i.e.
>
> $dangerous_key = hash_hkdf("sha256", hash("sha256", 'mypassword')); //
> Disaster!
>
> This does not help much when algorithm is known to attackers. Brute force
> attack is
> effective still. Adding secret salt(key) helps with this usage also.
>

IKM must always be strong; this is explicitly stated in the RFC, as I
already pointed out here: https://externals.io/message/98639#98874
And the reasons why were already explained in very simple terms here:
https://externals.io/message/98250#98272

Enough already.

Cheers,
Andrey.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 09:00AM
Hi Andrey,

On Fri, Sep 8, 2017 at 8:14 AM, Andrey Andreev <[email protected]> wrote:

> Hi,
>
> On Fri, Sep 8, 2017 at 12:32 AM, Yasuo Ohgaki <[email protected]> wrote:
> >
> > The reason why latter is a lot more secure is related to Andrey's
> > misunderstanding.
> > He said "when ikm is cryptographically strong, salt wouldn't add no more
> > entropy.
> > so salt does not matter". (not exact sentence)
> > What he said partially true, but wrong in a sense of key security.
> >
>
> I have never said that. You are aware enough of your own English
> fluency, and should know not to perephrase other people's words, as
> you are completely twisting their meaning.
>


My words:
-------
He said "when ikm is cryptographically strong, salt wouldn't add no more
entropy. so salt does not matter". (not exact sentence)
-------

This is your exact words.
------
And this doesn't stop at passwords. Please note that this paragraph
explicitly states this:

The extract step in HKDF can concentrate existing entropy but
cannot amplify entropy.

Which means that it is NOT designed to do key stretching, or in other
words it should NOT be relied upon to produce strong outputs from weak
inputs - the exact scenario for which you wanted to make salts
non-optional.
-------

Although you are referring other sentences from somewhere, it is obvious
this is your thought as well.

Note for others: "The extract step in HKDF can concentrate existing entropy
but cannot amplify entropy." is not came from the RFC. If a RFC states
this I would be stunned. Please read on you'll see the evidence.



> >
> > Other misunderstanding should be noted is what HKDF for. It's for general
> > purpose KDF as the RFC described in HKDF application section. Andrey said
> > "I'm cherry picking sentence", but the section should be what the HKDF
> for.
> > The section even describes obscure usage, HKDF for CSPRNG. This usage
> > is not even key derivation.
> >
>
> You ARE cherry-picking, and ignoring all evidence that contradicts you:
>

HKDF is general purpose KDF by its design, obviously.
The RFC has "Application of HKDF" section that states

RFC 5680 - https://tools.ietf.org/html/rfc5869#section-4
---------
HKDF is intended for use in a wide variety of KDF applications.
---------

"Wide variety of KDF applications" even includes non KDF operation like
CSPRNG. Your statement makes no sense.



> > This one I'm not sure misunderstanding or not, but probably it is.
> > HKDF is designed for any ikm and works with appropriate usage. Very
> > weak ikm like user entered password can be handled relatively safely.
> >
> > $safe_key = hash_hkdf("sha256", 'mypassword', 0, '',
> > $csprng_generated_random_key);
> > // $csprng_generated_random_key should be kept secret because ikm is too
> > weak
> >
> > Without salt, it's disaster. Please note that salt is the last optional
> > parameter currently.
> >
> > $dangerous_key = hash_hkdf("sha256", 'mypassword'); // Disaster!
> >
> > With this usage, attackers can build pre hashed dictionary. Even when
> they
> > don't have
> > dictionary, usual brute force attack is very effective. One may think
> > additional hashing
> > would mitigate risk. i.e.
> >
> > $dangerous_key = hash_hkdf("sha256", hash("sha256", 'mypassword')); //
> > Disaster!
> >
> > This does not help much when algorithm is known to attackers. Brute force
> > attack is
> > effective still. Adding secret salt(key) helps with this usage also.
> >
>
> IKM must always be strong; this is explicitly stated in the RFC, as I
> already pointed out here: https://externals.io/message/98639#98874
> And the reasons why were already explained in very simple terms here:
> https://externals.io/message/98250#98272
>
> Enough already.
>


Weak key is allowed by the RFC.
These are sentences from the RFC.

RFC 5689 - https://tools.ietf.org/html/rfc5869#section-3.3
--------
In some applications, the input key material IKM may already be
present as a cryptographically strong key (for example, the premaster
secret in TLS RSA cipher suites would be a pseudorandom string,
except for the first two octets). In this case, one can skip the
extract part and use IKM directly to key HMAC in the expand step.
---------
Note: expand step is HKDF-Expand(PRK, info, L) -> OKM, which is supposed
to expand length of resulting hash.

Obviously, your statement is wrong.

If I'm reading it correctly, salt may be omitted only when input key is
already strong.
i.e. This means weak key and strong salt results in strong PRK where PRK =
HMAC-Hash(salt, IKM)

However, "may be omitted" does not mean users should omit salt even with
strong IKM, because the RFC strongly recommends salt for significantly
better key security with other sentence.

Sorry Andrey, but I have to say you don't read the RFC at all or don't
understand it at all. My apology for strong words, but I was polite enough
until now even with your nonsense and offensive replies.

Regards,

P.S. I thought, HMAC produces strong hash with strong key even for
extremely weak message, is common sense, but apparently not. Note for
others: HKDF is made by HMAC.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Niklas Keller
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 10:00AM
>
> Note for others: "The extract step in HKDF can concentrate existing
> entropy
> but cannot amplify entropy." is not came from the RFC. If a RFC states
> this I would be stunned. Please read on you'll see the evidence.
>

This is ridiculous. Be stunned. It's right in the section about
applications of HKDF: https://tools.ietf.org/html/rfc5869#section-4, in the
middle of the second paragraph.

Please stop, it's enough.

Regards, Niklas
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 10:50AM
Hi Niklas,

On Fri, Sep 8, 2017 at 4:57 PM, Niklas Keller <[email protected]> wrote:

> Note for others: "The extract step in HKDF can concentrate existing
>> entropy
>> but cannot amplify entropy." is not came from the RFC. If a RFC states
>> this I would be stunned. Please read on you'll see the evidence.
>>
>
> This is ridiculous. Be stunned. It's right in the section about
> applications of HKDF: https://tools.ietf.org/html/rfc5869#section-4, in
> the middle of the second paragraph.
>
> Please stop, it's enough.
>

This is my bad I didn't understand "amplify" well.
I mixed up "add" and "amplify" I admit this.
However, the reason why I mixed up is Andrey's misunderstanding about
HMAC nature.

Andrey's words.
------
The extract step in HKDF can concentrate existing entropy but
cannot amplify entropy.

Which means that it is NOT designed to do key stretching, or in other
words it should NOT be relied upon to produce strong outputs from weak
inputs - the exact scenario for which you wanted to make salts
non-optional.
-------

Following is wrong by HMAC nature.
-------
it should NOT be relied upon to produce strong outputs from weak
inputs
-------

Who would think resulting $hash of following is insecure?
$hash = hash_hmac('sha256', $weak_msg, $strong_key);



RFC-5689 - https://tools.ietf.org/html/rfc5869#section-4
---------
One significant example is the derivation of cryptographic
eys from a source of low entropy, such as a user's password. The
extract step in HKDF can concentrate existing entropy but cannot
amplify entropy. In the case of password-based KDFs, a main goal is
to slow down dictionary attacks using two ingredients: a salt value,
and the intentional slowing of the key derivation computation. HKDF
naturally accommodates the use of salt; however, a slowing down
mechanism is not part of this specification. Applications interested
in a password-based KDF should consider whether, for example, [PKCS5]
meets their needs better than HKDF.
---------

This whole sentence means HKDF cannot be used like PBKDF2. That's all.
This is does not support Andrey's statement
-------
it should NOT be relied upon to produce strong outputs from weak
inputs
-------

Instead, he should read this section.

RFC 5689 - https://tools.ietf.org/html/rfc5869#section-3.3
--------
In some applications, the input key material IKM may already be
present as a cryptographically strong key. In this case, one can skip the
extract part and use IKM directly to key HMAC in the expand step.
---------

"one can skip the extract part" is "one can omit salt".

Obviously, the RFC is prepared for weak keys and it is perfectly OK to
use weak keys unlike Andrey statement.

If IKM should be strong always, HKDF would NOT have salt at all.
However, HKDF has salt.
Weak IKM is explicitly allowed by HKDF definition.

Andrey, sorry for my confusion, but your claim is wrong by HKDF
(HMAC) definition still.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 11:10AM
Hi all,

I finally find out what's wrong.

Andrey and Nikita, you missed

RFC 5689 - https://tools.ietf.org/html/rfc5869#section-3.3
--------
In some applications, the input key material IKM may already be
present as a cryptographically strong key. In this case, one can skip the
extract part and use IKM directly to key HMAC in the expand step.
---------

Therefore, you are debating "IKM should be strong always" and
"salt is pure optional parameter".

The section 3.3 is clearly expecting and allowing "weak keys", isn't it?
Existence of "salt" parameter is the evidence.

No wonder why you couldn't present reasonable example codes w/o salt.
Salt should/must be used for both strong and weak keys for better and
mandatory security.

Regards,

P.S. Thank you Niklas! Now we will have conclusion.


--
Yasuo Ohgaki
yohgaki@ohgaki.net

On Fri, Sep 8, 2017 at 5:44 PM, Yasuo Ohgaki <[email protected]> wrote:

> Hi Niklas,
>
> On Fri, Sep 8, 2017 at 4:57 PM, Niklas Keller <[email protected]> wrote:
>
>> Note for others: "The extract step in HKDF can concentrate existing
>>> entropy
>>> but cannot amplify entropy." is not came from the RFC. If a RFC states
>>> this I would be stunned. Please read on you'll see the evidence.
>>>
>>
>> This is ridiculous. Be stunned. It's right in the section about
>> applications of HKDF: https://tools.ietf.org/html/rfc5869#section-4, in
>> the middle of the second paragraph.
>>
>> Please stop, it's enough.
>>
>
> This is my bad I didn't understand "amplify" well.
> I mixed up "add" and "amplify" I admit this.
> However, the reason why I mixed up is Andrey's misunderstanding about
> HMAC nature.
>
> Andrey's words.
> ------
> The extract step in HKDF can concentrate existing entropy but
> cannot amplify entropy.
>
> Which means that it is NOT designed to do key stretching, or in other
> words it should NOT be relied upon to produce strong outputs from weak
> inputs - the exact scenario for which you wanted to make salts
> non-optional.
> -------
>
> Following is wrong by HMAC nature.
> -------
> it should NOT be relied upon to produce strong outputs from weak
> inputs
> -------
>
> Who would think resulting $hash of following is insecure?
> $hash = hash_hmac('sha256', $weak_msg, $strong_key);
>
>
>
> RFC-5689 - https://tools.ietf.org/html/rfc5869#section-4
> ---------
> One significant example is the derivation of cryptographic
> eys from a source of low entropy, such as a user's password. The
> extract step in HKDF can concentrate existing entropy but cannot
> amplify entropy. In the case of password-based KDFs, a main goal is
> to slow down dictionary attacks using two ingredients: a salt value,
> and the intentional slowing of the key derivation computation. HKDF
> naturally accommodates the use of salt; however, a slowing down
> mechanism is not part of this specification. Applications interested
> in a password-based KDF should consider whether, for example, [PKCS5]
> meets their needs better than HKDF.
> ---------
>
> This whole sentence means HKDF cannot be used like PBKDF2. That's all.
> This is does not support Andrey's statement
> -------
> it should NOT be relied upon to produce strong outputs from weak
> inputs
> -------
>
> Instead, he should read this section.
>
> RFC 5689 - https://tools.ietf.org/html/rfc5869#section-3.3
> --------
> In some applications, the input key material IKM may already be
> present as a cryptographically strong key. In this case, one can skip the
> extract part and use IKM directly to key HMAC in the expand step.
> ---------
>
> "one can skip the extract part" is "one can omit salt".
>
> Obviously, the RFC is prepared for weak keys and it is perfectly OK to
> use weak keys unlike Andrey statement.
>
> If IKM should be strong always, HKDF would NOT have salt at all.
> However, HKDF has salt.
> Weak IKM is explicitly allowed by HKDF definition.
>
> Andrey, sorry for my confusion, but your claim is wrong by HKDF
> (HMAC) definition still.
>
> Regards,
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
>
>
Niklas Keller
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 11:50AM
>
> I finally find out what's wrong.
>

No, you didn't. You still want to use user-supplied passwords as IKM. HKDF
is not suited for that purpose.


> RFC 5689 - https://tools.ietf.org/html/rfc5869#section-3.3
> --------
> In some applications, the input key material IKM may already be
> present as a cryptographically strong key. In this case, one can skip the
> extract part and use IKM directly to key HMAC in the expand step.
> ---------
>
> Therefore, you are debating "IKM should be strong always" and
> "salt is pure optional parameter".
>

Yes, HKDF might be used for lower-entropy IKM, but not for short inputs
like passwords. The extract part requires a large low-entropy input to
concentrate the entropy into a smaller output. HKDF doesn't add / amplify
entropy, but it can concentrate a larger low-entropy input to a
smaller output with entropy.

Further reading material: https://eprint.iacr.org/2010/264.pdf

Regards, Niklas
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 12:30PM
Hi Niklas,

On Fri, Sep 8, 2017 at 6:38 PM, Niklas Keller <[email protected]> wrote:

> I finally find out what's wrong.
>>
>
> No, you didn't. You still want to use user-supplied passwords as IKM. HKDF
> is not suited for that purpose.
>

Andrey and Nikita clearly misunderstood the RFC 5869.
This is what was wrong in previous discussions.

Weak key usage is smaller issue as I insisted HKDF is perfectly
good for CSRF token, API keys and URI signing. These 3 would be
the most common PHP HKDF applications.

What do you mean by "No, you didn't"?

I totally agree with that weak key has more risks.
The risk is too obvious for any algorithms.
Suppose we have "A" as the key, there is no protection at all.
Not even PBKDF2 or anything can protect such passwords.

I think you don't mean users shouldn't use key derivation with password.
Users may use HKDF and password with the security level of the password.


RFC 5689 - https://tools.ietf.org/html/rfc5869#section-3.3
>> --------
>> In some applications, the input key material IKM may already be
>> present as a cryptographically strong key. In this case, one can skip the
>> extract part and use IKM directly to key HMAC in the expand step.
>> ---------
>>
>> Therefore, you are debating "IKM should be strong always" and
>> "salt is pure optional parameter".
>>
>
> Yes, HKDF might be used for lower-entropy IKM, but not for short inputs
> like passwords. The extract part requires a large low-entropy input to
> concentrate the entropy into a smaller output. HKDF doesn't add / amplify
> entropy, but it can concentrate a larger low-entropy input to a
> smaller output with entropy.
>
> Further reading material: https://eprint.iacr.org/2010/264.pdf
>

Thank you nice reading. I've read briefly.
It sounded like SHA-2 and SHA-3 difference that could be ignored in
practice now.

Now issue is whether we'll fix the improper and inconsistent API or not.

Thank you Niklas,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 08, 2017 12:50PM
Hi Niklas,

On Fri, Sep 8, 2017 at 7:27 PM, Yasuo Ohgaki <[email protected]> wrote:

> Hi Niklas,
>
> On Fri, Sep 8, 2017 at 6:38 PM, Niklas Keller <[email protected]> wrote:
>
>> I finally find out what's wrong.
>>>
>>
>> No, you didn't. You still want to use user-supplied passwords as IKM.
>> HKDF is not suited for that purpose.
>>
>
> Andrey and Nikita clearly misunderstood the RFC 5869.
> This is what was wrong in previous discussions.
>
> Weak key usage is smaller issue as I insisted HKDF is perfectly
> good for CSRF token, API keys and URI signing. These 3 would be
> the most common PHP HKDF applications.
>
> What do you mean by "No, you didn't"?
>
> I totally agree with that weak key has more risks.
> The risk is too obvious for any algorithms.
> Suppose we have "A" as the key, there is no protection at all.
> Not even PBKDF2 or anything can protect such passwords.
>
> I think you don't mean users shouldn't use key derivation with password.
> Users may use HKDF and password with the security level of the password.
>

I was thinking in password hashing context, not key derivation. I was wrong.
(as well as you) I take it back last statement.

Even with super weak passwords, attackers cannot guess the derived keys
when "Salt" is used properly. i.e. Strong random "secret" salt makes
derived
key a perfect random.

Although, there is minor issues (i.e. misuse), users can safely use HKDF
with any passwords.

Now, please discuss the most important.
Are we going to fix the hash_hkdf() API mess or not?

Regards,

--
Yasuo Ohgaki
Yasuo Ohgaki
[PHP-DEV] Re: hash_hkdf() signature and return value
September 12, 2017 05:00AM
Hi all,

hash_hkdf() discussion was mess, but it came to conclusion finally a while
ago.
Apparently, there are confused readers still. I discovered it in other
thread.
I would like to make it clear again what's wrong in the discussion.
If you think I was wrong about HKDF, you should read this.


Nikita and Adnrey's Opinion:
1. Keys("IKM") must be strong always
(with very limited exception? Usage example is not disclosed)
2. From 1, "Salt" is pure optional parameter
3. HKDF is designed for specific purpose
(crypto keys only? I suppose)

Therefore, "Salt" is the last optional parameter and hash_hkdf() returns
BINARY.


I assume Andrey realized his misunderstanding finally and would not repeat
this ridiculous discussion. Andrey, you should realize when you couldn't
show
us any valid usage example. However, I appreciate discussion because I
couldn't
find out where you misunderstood exactly and why you insisted above
without
discussion.

Anyway, their statements make none of sense with HKDF specifications.
It is easy to accuse "You are wrong" or "You don't understand" regardless
of
validity. It requires a lot of work to make it clear what's really
correct/incorrect.
Those who are still do not understand what the HKDF RFC is about, please
read carefully what the RFC states.



What HKDF RFC says:

The algorithm: https://tools.ietf.org/html/rfc5869#section-2.2
----
HKDF-Extract(salt, IKM) -> PRK
PRK = HMAC-Hash(salt, IKM)
----
From this algorithm alone, HKDF is clearly expecting weak keys("IKM").
i.e. Remember how hash_hmac() works.


About "Salt": https://tools.ietf.org/html/rfc5869#section-3.1
----
HKDF is defined to operate with and without random salt. This is
done to accommodate applications where a salt value is not available.
We stress, however, that the use of salt adds significantly to the
strength of HKDF, ensuring independence between different uses of the
hash function, supporting "source-independent" extraction, and
strengthening the analytical results that back the HKDF design.
----
It is clear that salt is made optional only for apps that cannot use salt.
It is clear that HKDF recommends "Salt" regardless of key("IKM") strength.
"Salt" is required for "source-independent" extraction. i.e. Make IKM
entropy/strength irrelevant and derived key stronger.


Omitting "Salt": https://tools.ietf.org/html/rfc5869#section-3.3
----
In some applications, the input key material IKM may already be
present as a cryptographically strong key. In this case, one can skip
the
extract part and use IKM directly to key HMAC in the expand step.
----
It says "Salt" may be omitted __only when__ keys("IKM") are strong already.
In other words, "Salt" is mandatory for not cryptographically strong
keys("IKM").

e.g. $rsa256key = hash_hkdf("sha256", $rsa128key) is misuse, _NOT_ textbook
HKDF usage example as Nikita mentioned.
hash_hkdf("sha256", $rsa128key, 0, '', $random256salt) is correct/designed
usage.



Reason Why Nikita and Andrey misunderstood the RFC:

They ignored previously explained descriptions and cherry picked irrelevant
statements in the RFC.

Application of HKDF: https://tools.ietf.org/html/rfc5869#section-4
-----
On the other hand, it is anticipated that some applications will not
be able to use HKDF "as-is" due to specific operational requirements,
or will be able to use it but without the full benefits of the
scheme. One significant example is the derivation of cryptographic
keys from a source of low entropy, such as a user's password. The
extract step in HKDF can concentrate existing entropy but cannot
amplify entropy. In the case of password-based KDFs, a main goal is
to slow down dictionary attacks using two ingredients: a salt value,
and the intentional slowing of the key derivation computation. HKDF
naturally accommodates the use of salt; however, a slowing down
mechanism is not part of this specification.
-----

Especially this part

-----
the derivation of cryptographic
keys from a source of low entropy, such as a user's password. The
extract step in HKDF can concentrate existing entropy but cannot
amplify entropy.
-----

It says HKDF cannot be used with low entropy/weak keys("IKM"). This supports
their opinion.

However, these are explanations for __"Invalid/Wrong HKDF usage"__ as
"Password
hashing like PBKDF2". These are __irrelevant__ for "Valid/Correct HKDF
usage" at
all. i.e. This basically says "Hey, you shouldn't use HKDF for password
hashing "as-is".
HKDF is not password hashing that requires stretching.".

Key derivation and password hashing is fundamentally different operation
because "Salt" in password hashing is _non secret_ always by design.
Password hashing requires entropy amplification(stretching), but key
derivation does not require amplification by HMAC/cryptographic hash
characteristics.

Even with weakest password, password(IKM) is protected and OKM is secure
because HKDF's "Salt" is designed to be "secret" or "non secret" as
explicitly
described in the RFC. Secret salt makes keys are secured. This is obvious
from the algorithm, i.e. Remember how hash_hmac() works again.


Lastly,
Application of HKDF: https://tools.ietf.org/html/rfc5869#section-4
states in the first sentence.
-----
HKDF is intended for use in a wide variety of KDF applications.
-----
It is obvious that HKDF is general purpose KDF. The RFC even explains
non KDF usage as CSPRNG.



My Comments:
From the algorithm alone, it is very clear to me that HKDF is expecting
weak keys, and
salt is used for additional entropy/complexity for key security, and salt
should be used
always.

It was very difficult for me to understand what Nikita and Andrey were
insisting and
referring, because their discussion was very confusing and makes none of
sense
with HKDF RFC.(and HMAC, crypto hash characteristics)

Current API
- encourages insecure misuse, "Salt" is mandatory always with the
exception
when salt cannot be used.
- extremely insecure, weak keys are vulnerable without "secret Salt"
i.e. Who would use hash_hmac() without keys?

In addition, it has completely unnecessary API inconsistency with respect
to other
hash functions.

Nikita and Andrey, don't you have something to say about this mess, do you?

IMO, you should ask to the list by yourself so that your ridiculous
hash_hkdf() API
will not be kept forever because of your misunderstanding. If I were you, I
would
not leave behind such shameful API that would be laughed at by everyone.

RMs, if I were you, I will consider fixing the API seriously. Promoting
insecure
_key_ derivations forever is nonsense.

This would be the last post for this issue, unless there are people who
still don't
understand what HKDF is.

Regards,

P.S.
No one really read the internet RFC, this is another issue...
Session security and input data security discussion have similar pattern.
I insists based on standards and/or guidelines, others just ignore
standards/guidelines/security because they don't use/are not familiar with
them.
If it is simply a matter of preference, it would be acceptable. However,
in case of security related issues, this is not a healty technical
discussion
at all.


--
Yasuo Ohgaki
yohgaki@ohgaki.net

On Wed, Sep 6, 2017 at 10:15 AM, Yasuo Ohgaki <[email protected]> wrote:

> Hi all,
>
> This is the last recommendation for hash_hkdf[1]. In fact,
> this would be the last chance to fix because we'll have 7.2 soon.
> The issue is secure usage and API consistency.
>
> Currently hash_hkdf() has following signature:
>
> <binary string> hash_hkdf(string $algo , string $ikm [, int $length = 0 [,
> string $info = '' [, string $salt = '' ]]] )
>
> These are rationals behind recommendation. There are many, but
> please read on.
>
> === Parameter Order ===
>
> HKDF[2] algorithm is:
> 1. General purpose key derivation function as per RFC 5869
> 2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned
> RFC 5869
> 3. "$salt" (or preshared key) is very strongly recommended for security
> season as per RFC 5869
> 4. Supplying salt that the same length of input key does not affect
> performance as per RFC 5969
> 5. "$info" is what makes HKDF useful, but it's less important as
> described in RFC 5869
> 6. "$length" is truly an optional parameter for very specific encryption
> algorithm or usage.
>
> Rationale for change:
> 1. Key derivations without secondary key ($salt) does not make sense when
> secondary key could be available. HKDF is designed for best possible
> key
> security with the key. Not using secondary key ($salt) simply
> downgrades
> key security without no reason. i.e. HKDF performance is the same
> when $salt has the same as hash is set.
> 2. HKDF is based on HMAC. When $info has no use, HMAC would be the best
> choice for it. i.e. $newkey = hash_hmac($ikm, $key);
> 3. It does not make sense violating RFC recommendations for a RFC
> implementation.
>
> From these facts and reasons, $salt, $info and $length parameter order and
> requirement should be changed from
>
> string $algo , string $ikm [, int $length = 0 [, string $info = '' [,
> string $salt = '' ]]]
>
> to
>
> string $algo , string $ikm , string $salt, string $info = '' [,
> int $length = 0 ]
> Note: Users can set empty string if they really don't need $salt and/or
> $info.
>
> Conclusion:
> This way, users would have better chances to use hash_hkdf() more securely
> and
> properly.
>
> [1] http://php.net/hash_hkdf
> [2] http://www.faqs.org/rfcs/rfc5869.html
>
> === Return Value and Output Option ===
>
> The most common HKDF usage with PHP would be:
> 1. CSRF token generation that is specific to a request with expiration
> time.
> (HEX return value would be appropriate, not BINARY)
> 2. API access token generation that does not transmit "The API Key", but
> derived key by HKDF. It also should have expiration time.
> (HEX return value would be appropriate, not BINARY)
>
> Consistency with other hash_*() functions:
> 1. All of other hash_*() returns HEX string hash value.
> 2. hash_hkdf() is the only one returns BINARY hash value.
>
> Conclusion:
> hash_hkdf() should return HEX by default, not BINARY.
> Optional [, bool $raw_output = false ] should be added just like other
> hash_*().
>
>
> === Compatibility ===
>
> IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0
> in the first place. The mess could be resolved by 7.2.
>
> Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are
> using it yet. If we change API with 7.2 release, there would be least
> possible confusions. (We may remove it from 7.1 to avoid confusions, too)
>
> Our choices:
> - Keep the current insecure/inconsistent API forever.
> - Change the API to have secure/consistent API forever.
>
> Conclusion:
> No conclusion for this. There would be conflicting options.
>
>
> I strongly think it is not worth to keep this insecure/inconsistent API
> forever.
> I prefer to change the API to what it should be.
>
> What should we do for this?
> Comments?
>
>
>
> P.S.
>
> Nikita, you've said following during HKDF discussion:
> - HKDF for CSRF tokens/etc does not make sense at all.
> - Current parameter order and return value makes perfect sense
> and has valid/common usages.
> - Everyone one this list, shouldn't listen to me because I'm insane and
> totally misunderstood what the HKDF is.
>
> Phrases are not the exact, but it should be correct enough. Some part
> is my fault, w/o reading mail and/or poor English. I blindly assumed
> you've understand RFC 5869 and possible common usages with PHP. I
> apologized for the confusion and tried to explain why w/o success.
>
> If you still believe what you've said is correct, I don't request you to
> take these back, show us at least one common/reasonable hash_hkdf()
> usage example for current API, and point out what's wrong my
> recommendations
> and rationales above.
>
> If not, I request you to take it back. I respect your contributions much,
> but
> the last one you've said is out of tolerance.
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
>
Joe Watkins
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 13, 2017 08:30AM
This proposal was rejected.

6 months has not passed since it was rejected.

There will be no vote on these proposals in the near future.

Please stop.

Joe

On Tue, Sep 12, 2017 at 3:52 AM, Yasuo Ohgaki <[email protected]> wrote:

> Hi all,
>
> hash_hkdf() discussion was mess, but it came to conclusion finally a while
> ago.
> Apparently, there are confused readers still. I discovered it in other
> thread.
> I would like to make it clear again what's wrong in the discussion.
> If you think I was wrong about HKDF, you should read this.
>
>
> Nikita and Adnrey's Opinion:
> 1. Keys("IKM") must be strong always
> (with very limited exception? Usage example is not disclosed)
> 2. From 1, "Salt" is pure optional parameter
> 3. HKDF is designed for specific purpose
> (crypto keys only? I suppose)
>
> Therefore, "Salt" is the last optional parameter and hash_hkdf() returns
> BINARY.
>
>
> I assume Andrey realized his misunderstanding finally and would not repeat
> this ridiculous discussion. Andrey, you should realize when you couldn't
> show
> us any valid usage example. However, I appreciate discussion because I
> couldn't
> find out where you misunderstood exactly and why you insisted above
> without
> discussion.
>
> Anyway, their statements make none of sense with HKDF specifications.
> It is easy to accuse "You are wrong" or "You don't understand" regardless
> of
> validity. It requires a lot of work to make it clear what's really
> correct/incorrect.
> Those who are still do not understand what the HKDF RFC is about, please
> read carefully what the RFC states.
>
>
>
> What HKDF RFC says:
>
> The algorithm: https://tools.ietf.org/html/rfc5869#section-2.2
> ----
> HKDF-Extract(salt, IKM) -> PRK
> PRK = HMAC-Hash(salt, IKM)
> ----
> From this algorithm alone, HKDF is clearly expecting weak keys("IKM").
> i.e. Remember how hash_hmac() works.
>
>
> About "Salt": https://tools.ietf.org/html/rfc5869#section-3.1
> ----
> HKDF is defined to operate with and without random salt. This is
> done to accommodate applications where a salt value is not available.
> We stress, however, that the use of salt adds significantly to the
> strength of HKDF, ensuring independence between different uses of the
> hash function, supporting "source-independent" extraction, and
> strengthening the analytical results that back the HKDF design.
> ----
> It is clear that salt is made optional only for apps that cannot use salt.
> It is clear that HKDF recommends "Salt" regardless of key("IKM") strength.
> "Salt" is required for "source-independent" extraction. i.e. Make IKM
> entropy/strength irrelevant and derived key stronger.
>
>
> Omitting "Salt": https://tools.ietf.org/html/rfc5869#section-3.3
> ----
> In some applications, the input key material IKM may already be
> present as a cryptographically strong key. In this case, one can skip
> the
> extract part and use IKM directly to key HMAC in the expand step.
> ----
> It says "Salt" may be omitted __only when__ keys("IKM") are strong already.
> In other words, "Salt" is mandatory for not cryptographically strong
> keys("IKM").
>
> e.g. $rsa256key = hash_hkdf("sha256", $rsa128key) is misuse, _NOT_ textbook
> HKDF usage example as Nikita mentioned.
> hash_hkdf("sha256", $rsa128key, 0, '', $random256salt) is correct/designed
> usage.
>
>
>
> Reason Why Nikita and Andrey misunderstood the RFC:
>
> They ignored previously explained descriptions and cherry picked irrelevant
> statements in the RFC.
>
> Application of HKDF: https://tools.ietf.org/html/rfc5869#section-4
> -----
> On the other hand, it is anticipated that some applications will not
> be able to use HKDF "as-is" due to specific operational requirements,
> or will be able to use it but without the full benefits of the
> scheme. One significant example is the derivation of cryptographic
> keys from a source of low entropy, such as a user's password. The
> extract step in HKDF can concentrate existing entropy but cannot
> amplify entropy. In the case of password-based KDFs, a main goal is
> to slow down dictionary attacks using two ingredients: a salt value,
> and the intentional slowing of the key derivation computation. HKDF
> naturally accommodates the use of salt; however, a slowing down
> mechanism is not part of this specification.
> -----
>
> Especially this part
>
> -----
> the derivation of cryptographic
> keys from a source of low entropy, such as a user's password. The
> extract step in HKDF can concentrate existing entropy but cannot
> amplify entropy.
> -----
>
> It says HKDF cannot be used with low entropy/weak keys("IKM"). This
> supports
> their opinion.
>
> However, these are explanations for __"Invalid/Wrong HKDF usage"__ as
> "Password
> hashing like PBKDF2". These are __irrelevant__ for "Valid/Correct HKDF
> usage" at
> all. i.e. This basically says "Hey, you shouldn't use HKDF for password
> hashing "as-is".
> HKDF is not password hashing that requires stretching.".
>
> Key derivation and password hashing is fundamentally different operation
> because "Salt" in password hashing is _non secret_ always by design.
> Password hashing requires entropy amplification(stretching), but key
> derivation does not require amplification by HMAC/cryptographic hash
> characteristics.
>
> Even with weakest password, password(IKM) is protected and OKM is secure
> because HKDF's "Salt" is designed to be "secret" or "non secret" as
> explicitly
> described in the RFC. Secret salt makes keys are secured. This is obvious
> from the algorithm, i.e. Remember how hash_hmac() works again.
>
>
> Lastly,
> Application of HKDF: https://tools.ietf.org/html/rfc5869#section-4
> states in the first sentence.
> -----
> HKDF is intended for use in a wide variety of KDF applications.
> -----
> It is obvious that HKDF is general purpose KDF. The RFC even explains
> non KDF usage as CSPRNG.
>
>
>
> My Comments:
> From the algorithm alone, it is very clear to me that HKDF is expecting
> weak keys, and
> salt is used for additional entropy/complexity for key security, and salt
> should be used
> always.
>
> It was very difficult for me to understand what Nikita and Andrey were
> insisting and
> referring, because their discussion was very confusing and makes none of
> sense
> with HKDF RFC.(and HMAC, crypto hash characteristics)
>
> Current API
> - encourages insecure misuse, "Salt" is mandatory always with the
> exception
> when salt cannot be used.
> - extremely insecure, weak keys are vulnerable without "secret Salt"
> i.e. Who would use hash_hmac() without keys?
>
> In addition, it has completely unnecessary API inconsistency with respect
> to other
> hash functions.
>
> Nikita and Andrey, don't you have something to say about this mess, do you?
>
> IMO, you should ask to the list by yourself so that your ridiculous
> hash_hkdf() API
> will not be kept forever because of your misunderstanding. If I were you, I
> would
> not leave behind such shameful API that would be laughed at by everyone.
>
> RMs, if I were you, I will consider fixing the API seriously. Promoting
> insecure
> _key_ derivations forever is nonsense.
>
> This would be the last post for this issue, unless there are people who
> still don't
> understand what HKDF is.
>
> Regards,
>
> P.S.
> No one really read the internet RFC, this is another issue...
> Session security and input data security discussion have similar pattern.
> I insists based on standards and/or guidelines, others just ignore
> standards/guidelines/security because they don't use/are not familiar with
> them.
> If it is simply a matter of preference, it would be acceptable. However,
> in case of security related issues, this is not a healty technical
> discussion
> at all.
>
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
> On Wed, Sep 6, 2017 at 10:15 AM, Yasuo Ohgaki <[email protected]> wrote:
>
> > Hi all,
> >
> > This is the last recommendation for hash_hkdf[1]. In fact,
> > this would be the last chance to fix because we'll have 7.2 soon.
> > The issue is secure usage and API consistency.
> >
> > Currently hash_hkdf() has following signature:
> >
> > <binary string> hash_hkdf(string $algo , string $ikm [, int $length = 0
> [,
> > string $info = '' [, string $salt = '' ]]] )
> >
> > These are rationals behind recommendation. There are many, but
> > please read on.
> >
> > === Parameter Order ===
> >
> > HKDF[2] algorithm is:
> > 1. General purpose key derivation function as per RFC 5869
> > 2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned
> > RFC 5869
> > 3. "$salt" (or preshared key) is very strongly recommended for security
> > season as per RFC 5869
> > 4. Supplying salt that the same length of input key does not affect
> > performance as per RFC 5969
> > 5. "$info" is what makes HKDF useful, but it's less important as
> > described in RFC 5869
> > 6. "$length" is truly an optional parameter for very specific encryption
> > algorithm or usage.
> >
> > Rationale for change:
> > 1. Key derivations without secondary key ($salt) does not make sense
> when
> > secondary key could be available. HKDF is designed for best possible
> > key
> > security with the key. Not using secondary key ($salt) simply
> > downgrades
> > key security without no reason. i.e. HKDF performance is the same
> > when $salt has the same as hash is set.
> > 2. HKDF is based on HMAC. When $info has no use, HMAC would be the best
> > choice for it. i.e. $newkey = hash_hmac($ikm, $key);
> > 3. It does not make sense violating RFC recommendations for a RFC
> > implementation.
> >
> > From these facts and reasons, $salt, $info and $length parameter order
> and
> > requirement should be changed from
> >
> > string $algo , string $ikm [, int $length = 0 [, string $info = '' [,
> > string $salt = '' ]]]
> >
> > to
> >
> > string $algo , string $ikm , string $salt, string $info = '' [,
> > int $length = 0 ]
> > Note: Users can set empty string if they really don't need $salt and/or
> > $info.
> >
> > Conclusion:
> > This way, users would have better chances to use hash_hkdf() more
> securely
> > and
> > properly.
> >
> > [1] http://php.net/hash_hkdf
> > [2] http://www.faqs.org/rfcs/rfc5869.html
> >
> > === Return Value and Output Option ===
> >
> > The most common HKDF usage with PHP would be:
> > 1. CSRF token generation that is specific to a request with expiration
> > time.
> > (HEX return value would be appropriate, not BINARY)
> > 2. API access token generation that does not transmit "The API Key", but
> > derived key by HKDF. It also should have expiration time.
> > (HEX return value would be appropriate, not BINARY)
> >
> > Consistency with other hash_*() functions:
> > 1. All of other hash_*() returns HEX string hash value.
> > 2. hash_hkdf() is the only one returns BINARY hash value.
> >
> > Conclusion:
> > hash_hkdf() should return HEX by default, not BINARY.
> > Optional [, bool $raw_output = false ] should be added just like other
> > hash_*().
> >
> >
> > === Compatibility ===
> >
> > IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0
> > in the first place. The mess could be resolved by 7.2.
> >
> > Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are
> > using it yet. If we change API with 7.2 release, there would be least
> > possible confusions. (We may remove it from 7.1 to avoid confusions, too)
> >
> > Our choices:
> > - Keep the current insecure/inconsistent API forever.
> > - Change the API to have secure/consistent API forever.
> >
> > Conclusion:
> > No conclusion for this. There would be conflicting options.
> >
> >
> > I strongly think it is not worth to keep this insecure/inconsistent API
> > forever.
> > I prefer to change the API to what it should be.
> >
> > What should we do for this?
> > Comments?
> >
> >
> >
> > P.S.
> >
> > Nikita, you've said following during HKDF discussion:
> > - HKDF for CSRF tokens/etc does not make sense at all.
> > - Current parameter order and return value makes perfect sense
> > and has valid/common usages.
> > - Everyone one this list, shouldn't listen to me because I'm insane and
> > totally misunderstood what the HKDF is.
> >
> > Phrases are not the exact, but it should be correct enough. Some part
> > is my fault, w/o reading mail and/or poor English. I blindly assumed
> > you've understand RFC 5869 and possible common usages with PHP. I
> > apologized for the confusion and tried to explain why w/o success.
> >
> > If you still believe what you've said is correct, I don't request you to
> > take these back, show us at least one common/reasonable hash_hkdf()
> > usage example for current API, and point out what's wrong my
> > recommendations
> > and rationales above.
> >
> > If not, I request you to take it back. I respect your contributions much,
> > but
> > the last one you've said is out of tolerance.
> >
> > --
> > Yasuo Ohgaki
> > yohgaki@ohgaki.net
> >
> >
>
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 14, 2017 11:10PM
On Wed, Sep 13, 2017 at 3:24 PM, Joe Watkins <[email protected]> wrote:

> This proposal was rejected.
>
> 6 months has not passed since it was rejected.
>
> There will be no vote on these proposals in the near future.
>
> Please stop.
>
> Joe
>

Joe, you have great responsibility as RM for ridiculous hash_hkdf() API.
Current hash_hkdf() API is perfectly makes sense for you. I understand it.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] Re: hash_hkdf() signature and return value
September 14, 2017 11:30PM
On Fri, Sep 15, 2017 at 6:06 AM, Yasuo Ohgaki <[email protected]> wrote:

> On Wed, Sep 13, 2017 at 3:24 PM, Joe Watkins <[email protected]>
> wrote:
>
>> This proposal was rejected.
>>
>> 6 months has not passed since it was rejected.
>>
>> There will be no vote on these proposals in the near future.
>>
>> Please stop.
>>
>> Joe
>>
>
> Joe, you have great responsibility as RM for ridiculous hash_hkdf() API.
> Current hash_hkdf() API is perfectly makes sense for you. I understand it.
>

This means you think following key derivation is perfectly OK.
$new_key = hash_hmac('sha256', $my_secret_password, '');

Just my .02

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Sorry, only registered users may post in this forum.

Click here to login