Welcome! Log In Create A New Profile

Advanced

[PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter

Posted by Yasuo Ohgaki 
Yasuo Ohgaki
[PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
March 25, 2017 11:40PM
Hi all,

Since hash_hkdf() is in PHP 7.1.2, I restarted vote.
I posted previous announce in discussion thread by mistake.

https://wiki.php.net/rfc/improve_hash_hkdf_parameter

Vote start: 2017-03-26
Vote end: 2017-04-07 UTC 23:59:59

Current hash_hkdf() function signature does not make sense.

- HKDF is KEY derivation function, yet derivation KEY is the last option.
- hash_hkdf() is simple hash_hmac() extension, yet it has totally
different signature.
- Return value is binary unlike other hash functions.
- The signature is INSECURE.

Current signature is overly optimized very limited crypto operation
and cannot be optimal by above reasons.

Fortunately, almost all users are not using current hash_hkdf().
It's only in 7.1.2/7.1.3 now. We should avoid yet another new inconsistent
and insecure function. It would be better to be fixed ASAP, IMHO.

I suggest you to disclose the reason why against this change.
Otherwise, you may be considered you don't understand crypto basic.
i.e. HKDF(IKM) security depends on PRK being secure. To make PRK
secure or more secure, "salt" parameter is required. "length" is irrelevant
for security.

Thank you for voting.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
[PHP-DEV] Re: [RFC][VOTE] Improve hash_hkdf() parameter
March 27, 2017 01:20AM
Hi all,

On Sun, Mar 26, 2017 at 7:29 AM, Yasuo Ohgaki <[email protected]> wrote:

> I suggest you to disclose the reason why against this change.
> Otherwise, you may be considered you don't understand crypto basic.
> i.e. HKDF(IKM) security depends on PRK being secure. To make PRK
> secure or more secure, "salt" parameter is required. "length" is
> irrelevant
> for security.
>

I'll try to explain a bit more by examples.

HKDF is designed to obtain the best possible "cryptographically strong
hash value" (key) for various key derivation operations. Current signature
could lead to insecure/wrong usages. (We have similar experience with
our PHP functions. e.g. uniqid, crypt, etc)


Example #1 : Deriving strong 256 bit AES key from 128 bit AES key.

$new_key = hash_hkdf('sha256', $AES_128bit_key, 32); // Derive 256 bit AES
key from 128 bit key
// No additional entropy, thus $new_key is not strong 256 bit AES key.
// Far from the best possible.

Users must not do this with HKDF. The same $new_key quality can
be obtained by simple SHA-256 hashing which is faster. Without
"strong derivation key", HKDF is not useful at all. The optimal way is

$new_key = hash_hkdf('sha256', $AES_128bit_key, 0, '',
$strong_derivation_key);
// where $strong_derivation_key = random_bytes(32); or like.



Example #2 : Deriving strong key from week key such as user entered password

$new_key = hash_hkdf('sha256', [email protected]');
// Almost the same as hash('sha256', [email protected]'); All of us should know how
bad this is.
// Far from the best possible.

Users must not do this with HKDF. The same could be done with simple hash().
Users must provide cryptographically strong derivation key, otherwise HKDF
is
useless.

$new_key = hash_hkdf('sha256', [email protected]', 0, '', $strong_derivation_key);
// where $strong_derivation_key = random_bytes(32); or like.
// Since input key material is weak, $strong_derivation_key must be secret


Example #3 : Deriving CSRF token from secret seed

$new_key = hash_hkdf('sha256', $secret_seed, 0, $version);
// Almost the same as hash('sha256', $secret_seed . $version);
// Far from the best possible.

Users must not do this with HKDF. The same could be done with simple hash().
Users must provide cryptographically strong derivation key, otherwise HKDF
is
useless.

$new_key = hash_hkdf('sha256', $secret_seed, 0, $version,
$strong_derivation_key);
// where $strong_derivation_key = random_bytes(32); or like.



There are looong lists of this kind of insecure/wrong usage with current
signature.

If you understand how to derive "strong key" by HKDF, you should realize
current hash_hkdf() function signature is far from the best.

Detailed rationale is explained the PHP RFC, but it seems many of us does
not
understand this. HKDF is supposed to derive "strong key", why should we
encourage "weak key" derivations with non optimal signature?

Regards,

P.S. I strongly objected the current signature before 7.1 merge. Shouldn't
committer write RFC before commit in the first place? Especially for
released versions.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Stephen Reay
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
March 27, 2017 06:20AM
>
> I'll try to explain a bit more by examples.
>

Hi Yasuo,

It sounds to me like it is *possible* to currently use hash_hkdf() in a secure manner, but that you (and some others?) feel the arg order and default args are not conducive to safe/secure usage.

Given that the function is live in the wild, massively changing the order of things and defaults is an instant red flag for myself, and I believe a lot of other people.

To me this sounds more like an issue that could be relatively quickly improved by a documentation update that highlights how to securely use the function.

Yes, if there are more secure defaults that would be nice, but that ship has sailed, and the function was on it.


Just my 2 cents.


Cheers

Stephen
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 01, 2017 04:30AM
Hi Stephen,

On Mon, Mar 27, 2017 at 1:09 PM, Stephen Reay <[email protected]>
wrote:

>
> It sounds to me like it is *possible* to currently use hash_hkdf() in a
> secure manner, but that you (and some others?) feel the arg order and
> default args are not conducive to safe/secure usage.
>

It's _possible_, of course. Problem is _new_ function has
- insecure signature (it ignores strong RFC 5689 recommendation)
- inconsistent signature and return value (hash() and hash_hmac())
- no major use(application) for PHP apps (Length has almost no use with
PHP apps)

If users would like to generate arbitrary length hash from existing hash
value with _insecure_ way, they should
use new SHA-3 standards, i.e. SHA-3 already has 2 SHAKE algorithms that
generate arbitrary length hash value,
SHAKE128(M, d) and SHAKE256(M, d).
No reason to encourage less secure HKDF usage to obtain arbitrary length
hash value.

Current hash_hkdf() signature does not make much sense with regard to
cryptographically, consistency and
expected usage.

Given that the function is live in the wild, massively changing the order
> of things and defaults is an instant red flag for myself, and I believe a
> lot of other people.
>

Aside from it should not be merged into PHP 7.1 in the first place.
There are only 2 (or 3) bug fix versions released. Fixing mistake ASAP is
better. IMHO.


To me this sounds more like an issue that could be relatively quickly
> improved by a documentation update that highlights how to securely use the
> function.
>

While documentation may work, it seems silly for me to write,

Even if "salt" is the last optional parameter, users must set appropriate
"salt" whenever it is possible for maximum key security.

for new function.

Yes, if there are more secure defaults that would be nice, but that ship
> has sailed, and the function was on it.


Thank you for your comment.
I would like to try to fix it at least.

To avoid this kind of confusions, we are better to have RFC if there is
strong objection.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 01, 2017 05:00AM
Hi all,

- insecure signature (it ignores strong RFC 5689 recommendation)
s/RFC 5689/RFC 5869/

On Sat, Apr 1, 2017 at 11:27 AM, Yasuo Ohgaki <[email protected]> wrote:

>
> Given that the function is live in the wild, massively changing the order
>> of things and defaults is an instant red flag for myself, and I believe a
>> lot of other people.
>>
>
> Aside from it should not be merged into PHP 7.1 in the first place.
> There are only 2 (or 3) bug fix versions released. Fixing mistake ASAP is
> better. IMHO.
>
>
> To me this sounds more like an issue that could be relatively quickly
>> improved by a documentation update that highlights how to securely use the
>> function.
>>
>
> While documentation may work, it seems silly for me to write,
>
> Even if "salt" is the last optional parameter, users must set
> appropriate "salt" whenever it is possible for maximum key security.
>

Another possible resolution could be reverting hash_hkdf() merge from 7.1
branch.
Basic hash_hkdf() operation could be done by hash_hmac() easily.

The merge should have had PHP RFC.
Reverting hash_hkdf() merge may work better.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Joe Watkins
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 12, 2017 12:50PM
Morning,

This RFC was left open for 5 days past the end of voting as declared on the
RFC.

I have closed the vote, and moved it out of voting section on RFC index.

Cheers
Joe

On Sat, Apr 1, 2017 at 3:50 AM, Yasuo Ohgaki <[email protected]> wrote:

> Hi all,
>
> - insecure signature (it ignores strong RFC 5689 recommendation)
> s/RFC 5689/RFC 5869/
>
> On Sat, Apr 1, 2017 at 11:27 AM, Yasuo Ohgaki <[email protected]> wrote:
>
> >
> > Given that the function is live in the wild, massively changing the order
> >> of things and defaults is an instant red flag for myself, and I believe
> a
> >> lot of other people.
> >>
> >
> > Aside from it should not be merged into PHP 7.1 in the first place.
> > There are only 2 (or 3) bug fix versions released. Fixing mistake ASAP is
> > better. IMHO.
> >
> >
> > To me this sounds more like an issue that could be relatively quickly
> >> improved by a documentation update that highlights how to securely use
> the
> >> function.
> >>
> >
> > While documentation may work, it seems silly for me to write,
> >
> > Even if "salt" is the last optional parameter, users must set
> > appropriate "salt" whenever it is possible for maximum key security.
> >
>
> Another possible resolution could be reverting hash_hkdf() merge from 7.1
> branch.
> Basic hash_hkdf() operation could be done by hash_hmac() easily.
>
> The merge should have had PHP RFC.
> Reverting hash_hkdf() merge may work better.
>
> Regards,
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 13, 2017 01:20AM
Hi Joe,

On Wed, Apr 12, 2017 at 7:46 PM, Joe Watkins <[email protected]> wrote:

> This RFC was left open for 5 days past the end of voting as declared on
> the RFC.
>

Thank you, I forgot about this.
IMHO, it's a shame for us we should have inconsistent and insecure function
signature for a new function.

I'm going to update the manual to add warning notes and example usages
like advanced CRFS token dedicated for specific URL with expiration time.

I can think of length option only usage, but I cannot think usage that could
be useful for majority of PHP users like advanced CSRF token.

Andrey,

Could you give us some length only and length/info only example
that could be useful for most PHP users.
It should be safe and recommended usage.
I suppose you should have some good examples.

Thank you.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Pieter Hordijk
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 13, 2017 10:20AM
----- Original Message -----
> From: "Yasuo Ohgaki" <[email protected]>
> To: "Joe Watkins" <[email protected]>, "Andrey Andreev" <[email protected].net>
> Cc: internals@lists.php.net
> Sent: Thursday, April 13, 2017 1:07:19 AM
> Subject: Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter

> Hi Joe,
>
> On Wed, Apr 12, 2017 at 7:46 PM, Joe Watkins <[email protected]> wrote:
>
>> This RFC was left open for 5 days past the end of voting as declared on
>> the RFC.
>>
>
> Thank you, I forgot about this.
> IMHO, it's a shame for us we should have inconsistent and insecure function
> signature for a new function.
>
> I'm going to update the manual to add warning notes and example usages
> like advanced CRFS token dedicated for specific URL with expiration time.
>
> I can think of length option only usage, but I cannot think usage that could
> be useful for majority of PHP users like advanced CSRF token.

Is this really something we need in our official docs instead of for example
on a personal blog?

To be honest I am afraid of ending up with something like the current state
of the session docs. Which are imo way too broad / opinionated, non English,
contains utterly confusing examples and / or flat out wrong and broken examples.
Above already resulted in a stream of docs bugs regarding session pages
and a lot of confused readers.

By all means describe how functions work, but don't confuse readers with things
most people won't ever need or are better suited as a (series of) blog posts /
Stack Overflow post(s).

My €0.02

cc-ing docs discussion to get them also involved in case somebody of the docs
team has an opinion.

> Andrey,
>
> Could you give us some length only and length/info only example
> that could be useful for most PHP users.
> It should be safe and recommended usage.
> I suppose you should have some good examples.
>
> Thank you.
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 13, 2017 10:50AM
Hi Peiter,

On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <[email protected]>
wrote:

> To be honest I am afraid of ending up with something like the current state
> of the session docs. Which are imo way too broad / opinionated, non
> English,
> contains utterly confusing examples and / or flat out wrong and broken
> examples.
> Above already resulted in a stream of docs bugs regarding session pages
> and a lot of confused readers.
>

You may consider my opinion as my personal opinion. I don't know of other
than
me who had that opinion then.

After our session discussion, it seems OWASP adopted most of discussed
elements in their doc ;)

https://www.owasp.org/index.php/Session_Management_Cheat_Sheet

Regards,

P.S. My opinion is based on RFC 5869. In addition, it's totally nonsense to
me to have completely different signature for hash_hkdf().
See the difference hash_hmac() and hash_pbkdf2(). hash_pbkdf2() is older
KDF function. I should have mention in the RFC :(

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 13, 2017 11:20AM
Hi Pieter,

On Thu, Apr 13, 2017 at 5:38 PM, Yasuo Ohgaki <[email protected]> wrote:

>
> On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <[email protected]>
> wrote:
>
>> To be honest I am afraid of ending up with something like the current
>> state
>> of the session docs. Which are imo way too broad / opinionated, non
>> English,
>> contains utterly confusing examples and / or flat out wrong and broken
>> examples.
>> Above already resulted in a stream of docs bugs regarding session pages
>> and a lot of confused readers.
>>
>
> You may consider my opinion as my personal opinion. I don't know of other
> than
> me who had that opinion then.
>
> After our session discussion, it seems OWASP adopted most of discussed
> elements in their doc ;)
>

I'm not exactly sure which part you consider as personal blog.

Current session management is too loose and insecure in many ways.
Since mandatory features for precise session management are not implemented,
the doc is intermediate.

I'm willing to improve the doc and appreciate improvement suggestions
always.
Feel free to send to my personal mail address.

Required information for precise and secure session management should be
in Precise Session Management RFC
https://wiki.php.net/rfc/precise_session_management

I appreciate if one could add missing documentation for precise session
management.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
wout van gils
[PHP-DEV] Re: Improve hash_hkdf() parameter
April 13, 2017 05:20PM
Kan iemand mij eindelijk eens uitschrijven.?
wout van gils
[PHP-DEV] Re: Improve hash_hkdf() parameter
April 13, 2017 05:20PM
Kan iemand mij eindelijk eens uitschrijven.?


??


________________________________
Van: Pieter Hordijk <[email protected]>
Verzonden: donderdag 13 april 2017 08:11
Aan: Yasuo Ohgaki
CC: Joe Watkins; Andrey Andreev; internals@lists.php.net; phpdoc@lists.php.net
Onderwerp: [PHP-DOC] Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter



----- Original Message -----
> From: "Yasuo Ohgaki" <[email protected]>
> To: "Joe Watkins" <[email protected]>, "Andrey Andreev" <[email protected].net>
> Cc: internals@lists.php.net
> Sent: Thursday, April 13, 2017 1:07:19 AM
> Subject: Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter

> Hi Joe,
>
> On Wed, Apr 12, 2017 at 7:46 PM, Joe Watkins <[email protected]> wrote:
>
>> This RFC was left open for 5 days past the end of voting as declared on
>> the RFC.
>>
>
> Thank you, I forgot about this.
> IMHO, it's a shame for us we should have inconsistent and insecure function
> signature for a new function.
>
> I'm going to update the manual to add warning notes and example usages
> like advanced CRFS token dedicated for specific URL with expiration time.
>
> I can think of length option only usage, but I cannot think usage that could
> be useful for majority of PHP users like advanced CSRF token.

Is this really something we need in our official docs instead of for example
on a personal blog?

To be honest I am afraid of ending up with something like the current state
of the session docs. Which are imo way too broad / opinionated, non English,
contains utterly confusing examples and / or flat out wrong and broken examples.
Above already resulted in a stream of docs bugs regarding session pages
and a lot of confused readers.

By all means describe how functions work, but don't confuse readers with things
most people won't ever need or are better suited as a (series of) blog posts /
Stack Overflow post(s).

My €0.02

cc-ing docs discussion to get them also involved in case somebody of the docs
team has an opinion.

> Andrey,
>
> Could you give us some length only and length/info only example
> that could be useful for most PHP users.
> It should be safe and recommended usage.
> I suppose you should have some good examples.
>
> Thank you.
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
Jan Ehrhardt
[PHP-DEV] Re: Improve hash_hkdf() parameter
April 13, 2017 05:50PM
wout van gils in php.internals (Thu, 13 Apr 2017 15:13:40 +0000):
>Kan iemand mij eindelijk eens uitschrijven.?

Dat moet je zelf doen:
http://php.net/mailing-lists.php
Onderaan.
--
Jan

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 13, 2017 11:30PM
Hi Pieter and all,

On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <[email protected]>
wrote:

> Is this really something we need in our official docs instead of for
> example
> on a personal blog?
>

I wrote draft doc patch.
Please verify.

Index: en/reference/hash/functions/hash-hkdf.xml
===================================================================
--- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317)
+++ en/reference/hash/functions/hash-hkdf.xml (作業コピー)
@@ -3,7 +3,7 @@
<refentry xml:id="function.hash-hkdf" xmlns="http://docbook.org/ns/docbook";
xmlns:xlink="http://www.w3.org/1999/xlink">;
<refnamediv>
<refname>hash_hkdf</refname>
- <refpurpose>Generate a HKDF key derivation of a supplied key
input</refpurpose>
+ <refpurpose>Derive secure new key from existing key by using
HKDF</refpurpose>
</refnamediv>
<refsect1 role="description">
&reftitle.description;
@@ -16,6 +16,20 @@
<methodparam
choice="opt"><type>string</type><parameter>salt</parameter><initializer>''</initializer></methodparam>
</methodsynopsis>

+ <para>
+ RFC 5869 defines HKDF (HMAC based Key Derivation Function) which
+ is general purpose KDF. HKDF could be useful for many PHP
+ applications that require temporary keys, such CSRF token,
+ pre-signed key for URI, password for password protected
+ URI, and so on.
+ </para>
+ <note>
+ <para>
+ When info and length
+ is not required for your program, more efficient
+ <function>hash_hmac</function> could be used instead.
+ </para>
+ </note>
</refsect1>
<refsect1 role="parameters">
&reftitle.parameters;
@@ -25,7 +39,7 @@
<term><parameter>algo</parameter></term>
<listitem>
<para>
- Name of selected hashing algorithm (i.e. "sha256", "sha512",
"haval160,4", etc..)
+ Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512",
"sha256", "sha512", "haval160,4", etc..)
See <function>hash_algos</function> for a list of supported
algorithms.
<note>
<para>
@@ -39,7 +53,7 @@
<term><parameter>ikm</parameter></term>
<listitem>
<para>
- Input keying material (raw binary). Cannot be empty.
+ Input keying material. Cannot be empty.
</para>
</listitem>
</varlistentry>
@@ -60,7 +74,8 @@
<term><parameter>info</parameter></term>
<listitem>
<para>
- Application/context-specific info string.
+ Application/context-specific info string. Info is intended for
+ public information such as user ID, protocol version, etc.
</para>
</listitem>
</varlistentry>
@@ -71,8 +86,32 @@
Salt to use during derivation.
</para>
<para>
- While optional, adding random salt significantly improves the
strength of HKDF.
+ While optional, adding random salt significantly improves the
+ strength of HKDF. Salt could be either secret or
+ non-secret. It is used as "Pre Shared Key" in many use cases.
+ Strong value is preferred. e.g. Use
<function>random_bytes</function>.
+ Optimal salt size is size of used hash algorithm.
</para>
+ <warning>
+ <para>
+ Although salt is the last optional parameter, salt is the
+ most important parameter for key security. Omitted salt is
+ indication of inappropriate design in most cases. Users must
+ set appropriate salt value whenever it is possible. Omit salt
+ only when it cannot be used.
+ </para>
+ <para>
+ Strong salt is mandatory and must be kept secret when input
+ key is weak, otherwise input key security will not be kept.
+ Even when input key is strong, providing strong salt is the
+ best practice for the best possible key security.
+ </para>
+ <para>
+ Salt must not be able to be controlled by users. i.e. User
+ must not be able to set salt value and get derived key. User
+ controlled salt allows input key analysis to attackers.
+ </para>
+ </warning>
</listitem>
</varlistentry>
</variablelist>
@@ -101,6 +140,99 @@
&reftitle.examples;
<para>
<example>
+ <title>URI specific CSRF token that supports expiration by
<function>hash_hkdf</function></title>
+ <programlisting role="php">
+<![CDATA[
+<?php
+define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration
+define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid
+
+/**************************************
+ * Implementation note
+ *
+ * It uses "counter" for CSRF expiration management.
+ * "counter" is very low entropy, but input key is strong and
+ * CSRF_TOKEN_SEED is short term key. It should be OK.
+ *
+ * This CSRF token implementation has pros and cons
+ *
+ * Pros
+ * - A CSRF token is valid only for specific URI.
+ * - No database is required for URI specific CSRF tokens.
+ * - Only CSRF token is required. i.e. No timestamp parameter.
+ * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE *
CSRF_TOKENS sec.
+ * - Even when user had long idle time, CSRF token is valid.
+ * - CSRF token will expire eventually.
+ * - Invalidating all active CSRF tokens could be done by
unset($_SESSION['CSRF_TOKEN_SEED']).
+ * It is recommended to reset CSRF tokens by login/logout event at
least.
+ * It may be good idea to invalidate all of older CSRF tokens when idle
time is long.
+ *
+ * Cons
+ * - There could be no CSRF expiration time.
+ *
+ * Precise CSRF token expiration is easy. Just add timestamp parameter
+ * as "info" and check it.
+ **************************************/
+
+session_start();
+if (empty($_SESSION['CSRF_TOKEN_SEED'])) {
+ $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32);
+ $_SESSION['CSRF_TOKEN_COUNT'] = 1;
+ $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
+}
+
+
+function csrf_get_token($uri) {
+ // Check expiration
+ if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) {
+ $_SESSION['CSRF_TOKEN_COUNT']++;
+ $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
+ }
+ // Equivalent(NOT exactly the same) value by using hash_hmac()
+ // return hash_hmac('sha3-256', hash_hmac('sha3-256',
$_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri);
+ return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri,
$_SESSION['CSRF_TOKEN_COUNT']);
+}
+
+function csrf_validate_token($csrf_token, $uri) {
+ for($i = 0; $i < CSRF_TOKENS; $i++) {
+ // Equivalent(NOT exactly the same) value by using hash_hmac()
+ // $token = hash_hmac('sha3-256', hash_hmac('sha3-256',
$_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri);
+ $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0,
$uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i);
+ if (hash_equals($csrf_token, $token)) {
+ return TRUE;
+ }
+ }
+ return FALSE;
+}
+
+
+//// Generating CSRF token ////
+// $uri is target URI that browser POSTs form data
+$uri = 'https://example.com/some_form/';
+$csrf_token = csrf_get_token($uri);
+// embed $csrf_token to your form
+
+//// Validating CSRF token ////
+$csrf_token = $_POST['csrf_token'] ?? '';
+if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) {
+ // Invalid CSRF token
+ throw new Exception('CSRF token validation error');
+}
+// valid request
+?>
+]]>
+ </programlisting>
+ <para>
+ Common CSRF token uses the same token value for a session and all
+ URI. This example CSRF token expires and is specific to a
+ URI. i.e. CSRF token http://example.com/form_A/ is not valid for
+ http://example.com/form_B/ Since token value is computed, no
+ database is required.
+ </para>
+ </example>
+ </para>
+ <para>
+ <example>
<title><function>hash_hkdf</function> example</title>
<programlisting role="php">
<![CDATA[
@@ -124,6 +256,30 @@
</para>
</example>
</para>
+ <para>
+ <example>
+ <title><function>hash_hkdf</function> bad example</title>
+ <para>
+ Users must not simply extend input key material length. HKDF does
+ not add additional entropy automatically. Therefore, weak key
+ remains weak unless strong salt is supplied. Following is bad
+ example.
+ </para>
+ <programlisting role="php">
+<![CDATA[
+<?php
+$inputKey = get_my_aes128_key(); // AES 128 bit key
+
+// Derive AES 256 key from AES 128 key
+$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption');
+// Users should not do this. $encryptionKey only has 128 bit
+// entropy while it should have 256 bit entropy.
+// To derive strong AES 256 key, strong enough salt is required.
+?>
+]]>
+ </programlisting>
+ </example>
+ </para>
</refsect1>

<refsect1 role="seealso">
@@ -130,6 +286,7 @@
&reftitle.seealso;
<para>
<simplelist>
+ <member><function>hash_hmac</function></member>
<member><function>hash_pbkdf2</function></member>
<member><link xlink:href="&url.rfc;5869">RFC 5869</link></member>
<member><link
xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland
implementation</link></member>

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 14, 2017 12:00AM
Hi all,

On Fri, Apr 14, 2017 at 6:22 AM, Yasuo Ohgaki <[email protected]> wrote:

>
> On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <[email protected]>
> wrote:
>
>> Is this really something we need in our official docs instead of for
>> example
>> on a personal blog?
>>
>
> I wrote draft doc patch.
> Please verify.
>

I used "very low entropy salt" for this CSRF token because "Input key is
strong, very low
entropy salt is acceptable". To avoid confusions, I revised the doc patch.

Index: en/reference/hash/functions/hash-hkdf.xml
===================================================================
--- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317)
+++ en/reference/hash/functions/hash-hkdf.xml (作業コピー)
@@ -3,7 +3,7 @@
<refentry xml:id="function.hash-hkdf" xmlns="http://docbook.org/ns/docbook";
xmlns:xlink="http://www.w3.org/1999/xlink">;
<refnamediv>
<refname>hash_hkdf</refname>
- <refpurpose>Generate a HKDF key derivation of a supplied key
input</refpurpose>
+ <refpurpose>Derive secure new key from existing key by using
HKDF</refpurpose>
</refnamediv>
<refsect1 role="description">
&reftitle.description;
@@ -16,6 +16,20 @@
<methodparam choice="opt"><type>string</type><parameter>salt</
parameter><initializer>''</initializer></methodparam>
</methodsynopsis>

+ <para>
+ RFC 5869 defines HKDF (HMAC based Key Derivation Function) which
+ is general purpose KDF. HKDF could be useful for many PHP
+ applications that require temporary keys, such CSRF token,
+ pre-signed key for URI, password for password protected
+ URI, and so on.
+ </para>
+ <note>
+ <para>
+ When info and length
+ is not required for your program, more efficient
+ <function>hash_hmac</function> could be used instead.
+ </para>
+ </note>
</refsect1>
<refsect1 role="parameters">
&reftitle.parameters;
@@ -25,7 +39,7 @@
<term><parameter>algo</parameter></term>
<listitem>
<para>
- Name of selected hashing algorithm (i.e. "sha256", "sha512",
"haval160,4", etc..)
+ Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512",
"sha256", "sha512", "haval160,4", etc..)
See <function>hash_algos</function> for a list of supported
algorithms.
<note>
<para>
@@ -39,7 +53,7 @@
<term><parameter>ikm</parameter></term>
<listitem>
<para>
- Input keying material (raw binary). Cannot be empty.
+ Input keying material. Cannot be empty.
</para>
</listitem>
</varlistentry>
@@ -60,7 +74,8 @@
<term><parameter>info</parameter></term>
<listitem>
<para>
- Application/context-specific info string.
+ Application/context-specific info string. Info is intended for
+ public information such as user ID, protocol version, etc.
</para>
</listitem>
</varlistentry>
@@ -71,8 +86,34 @@
Salt to use during derivation.
</para>
<para>
- While optional, adding random salt significantly improves the
strength of HKDF.
+ While optional, adding random salt significantly improves the
+ strength of HKDF. Salt could be either secret or
+ non-secret. It is used as "Pre Shared Key" in many use cases.
+ Strong value is preferred. e.g. Use <function>random_bytes</
function>.
+ Optimal salt size is size of used hash algorithm.
</para>
+ <warning>
+ <para>
+ Although salt is the last optional parameter, salt is the
+ most important parameter for key security. Omitted salt is
+ indication of inappropriate design in most cases. Users must
+ set appropriate salt value whenever it is possible. Omit salt
+ only when it cannot be used.
+ </para>
+ <para>
+ Strong salt is mandatory and must be kept secret when input
+ key is weak, otherwise input key security will not be kept.
+ When input key is strong, low entropy salt is acceptable.
+ However, providing strong salt is the best practice for the
+ best possible key security. Strong salt is strongly recommended
+ long life input keys.
+ </para>
+ <para>
+ Salt must not be able to be controlled by users. i.e. User
+ must not be able to set salt value and get derived key. User
+ controlled salt allows input key analysis to attackers.
+ </para>
+ </warning>
</listitem>
</varlistentry>
</variablelist>
@@ -101,6 +142,99 @@
&reftitle.examples;
<para>
<example>
+ <title>URI specific CSRF token that supports expiration by
<function>hash_hkdf</function></title>
+ <programlisting role="php">
+<![CDATA[
+<?php
+define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration
+define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid
+
+/**************************************
+ * Implementation note
+ *
+ * It uses "counter" for CSRF expiration management.
+ * "counter" is very low entropy, but input key is strong and
+ * CSRF_TOKEN_SEED is short term key. It should be OK.
+ *
+ * This CSRF token implementation has pros and cons
+ *
+ * Pros
+ * - A CSRF token is valid only for specific URI.
+ * - No database is required for URI specific CSRF tokens.
+ * - Only CSRF token is required. i.e. No timestamp parameter.
+ * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE *
CSRF_TOKENS sec.
+ * - Even when user had long idle time, CSRF token is valid.
+ * - CSRF token will expire eventually.
+ * - Invalidating all active CSRF tokens could be done by
unset($_SESSION['CSRF_TOKEN_SEED']).
+ * It is recommended to reset CSRF tokens by login/logout event at
least.
+ * It may be good idea to invalidate all of older CSRF tokens when idle
time is long.
+ *
+ * Cons
+ * - There could be no CSRF expiration time.
+ *
+ * Precise CSRF token expiration is easy. Just add timestamp parameter
+ * as "info" and check it.
+ **************************************/
+
+session_start();
+if (empty($_SESSION['CSRF_TOKEN_SEED'])) {
+ $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32);
+ $_SESSION['CSRF_TOKEN_COUNT'] = 1;
+ $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
+}
+
+
+function csrf_get_token($uri) {
+ // Check expiration
+ if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) {
+ $_SESSION['CSRF_TOKEN_COUNT']++;
+ $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
+ }
+ // Equivalent(NOT exactly the same) value by using hash_hmac()
+ // return hash_hmac('sha3-256', hash_hmac('sha3-256',
$_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri);
+ return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri,
$_SESSION['CSRF_TOKEN_COUNT']);
+}
+
+function csrf_validate_token($csrf_token, $uri) {
+ for($i = 0; $i < CSRF_TOKENS; $i++) {
+ // Equivalent(NOT exactly the same) value by using hash_hmac()
+ // $token = hash_hmac('sha3-256', hash_hmac('sha3-256',
$_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri);
+ $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0,
$uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i);
+ if (hash_equals($csrf_token, $token)) {
+ return TRUE;
+ }
+ }
+ return FALSE;
+}
+
+
+//// Generating CSRF token ////
+// $uri is target URI that browser POSTs form data
+$uri = 'https://example.com/some_form/';
+$csrf_token = csrf_get_token($uri);
+// embed $csrf_token to your form
+
+//// Validating CSRF token ////
+$csrf_token = $_POST['csrf_token'] ?? '';
+if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) {
+ // Invalid CSRF token
+ throw new Exception('CSRF token validation error');
+}
+// valid request
+?>
+]]>
+ </programlisting>
+ <para>
+ Common CSRF token uses the same token value for a session and all
+ URI. This example CSRF token expires and is specific to a
+ URI. i.e. CSRF token http://example.com/form_A/ is not valid for
+ http://example.com/form_B/ Since token value is computed, no
+ database is required.
+ </para>
+ </example>
+ </para>
+ <para>
+ <example>
<title><function>hash_hkdf</function> example</title>
<programlisting role="php">
<![CDATA[
@@ -124,6 +258,30 @@
</para>
</example>
</para>
+ <para>
+ <example>
+ <title><function>hash_hkdf</function> bad example</title>
+ <para>
+ Users must not simply extend input key material length. HKDF does
+ not add additional entropy automatically. Therefore, weak key
+ remains weak unless strong salt is supplied. Following is bad
+ example.
+ </para>
+ <programlisting role="php">
+<![CDATA[
+<?php
+$inputKey = get_my_aes128_key(); // AES 128 bit key
+
+// Derive AES 256 key from AES 128 key
+$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption');
+// Users should not do this. $encryptionKey only has 128 bit
+// entropy while it should have 256 bit entropy.
+// To derive strong AES 256 key, strong enough salt is required.
+?>
+]]>
+ </programlisting>
+ </example>
+ </para>
</refsect1>

<refsect1 role="seealso">
@@ -130,6 +288,7 @@
&reftitle.seealso;
<para>
<simplelist>
+ <member><function>hash_hmac</function></member>
<member><function>hash_pbkdf2</function></member>
<member><link xlink:href="&url.rfc;5869">RFC 5869</link></member>
<member><link xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland
implementation</link></member>

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Nikita Popov
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 14, 2017 11:30AM
On Thu, Apr 13, 2017 at 11:22 PM, Yasuo Ohgaki <[email protected]> wrote:

> Hi Pieter and all,
>
> On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <[email protected]>
> wrote:
>
> > Is this really something we need in our official docs instead of for
> > example
> > on a personal blog?
> >
>
> I wrote draft doc patch.
> Please verify.
>
> Index: en/reference/hash/functions/hash-hkdf.xml
> ===================================================================
> --- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317)
> +++ en/reference/hash/functions/hash-hkdf.xml (作業コピー)
> @@ -3,7 +3,7 @@
> <refentry xml:id="function.hash-hkdf" xmlns="http://docbook.org/ns/
> docbook"
> xmlns:xlink="http://www.w3.org/1999/xlink">;
> <refnamediv>
> <refname>hash_hkdf</refname>
> - <refpurpose>Generate a HKDF key derivation of a supplied key
> input</refpurpose>
> + <refpurpose>Derive secure new key from existing key by using
> HKDF</refpurpose>
> </refnamediv>
> <refsect1 role="description">
> &reftitle.description;
> @@ -16,6 +16,20 @@
> <methodparam
> choice="opt"><type>string</type><parameter>salt</
> parameter><initializer>''</initializer></methodparam>
> </methodsynopsis>
>
> + <para>
> + RFC 5869 defines HKDF (HMAC based Key Derivation Function) which
> + is general purpose KDF. HKDF could be useful for many PHP
> + applications that require temporary keys, such CSRF token,
> + pre-signed key for URI, password for password protected
> + URI, and so on.
> + </para>
> + <note>
> + <para>
> + When info and length
> + is not required for your program, more efficient
> + <function>hash_hmac</function> could be used instead.
> + </para>
> + </note>
> </refsect1>
> <refsect1 role="parameters">
> &reftitle.parameters;
> @@ -25,7 +39,7 @@
> <term><parameter>algo</parameter></term>
> <listitem>
> <para>
> - Name of selected hashing algorithm (i.e. "sha256", "sha512",
> "haval160,4", etc..)
> + Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512",
> "sha256", "sha512", "haval160,4", etc..)
> See <function>hash_algos</function> for a list of supported
> algorithms.
> <note>
> <para>
> @@ -39,7 +53,7 @@
> <term><parameter>ikm</parameter></term>
> <listitem>
> <para>
> - Input keying material (raw binary). Cannot be empty.
> + Input keying material. Cannot be empty.
> </para>
> </listitem>
> </varlistentry>
> @@ -60,7 +74,8 @@
> <term><parameter>info</parameter></term>
> <listitem>
> <para>
> - Application/context-specific info string.
> + Application/context-specific info string. Info is intended for
> + public information such as user ID, protocol version, etc.
> </para>
> </listitem>
> </varlistentry>
> @@ -71,8 +86,32 @@
> Salt to use during derivation.
> </para>
> <para>
> - While optional, adding random salt significantly improves the
> strength of HKDF.
> + While optional, adding random salt significantly improves the
> + strength of HKDF. Salt could be either secret or
> + non-secret. It is used as "Pre Shared Key" in many use cases.
> + Strong value is preferred. e.g. Use
> <function>random_bytes</function>.
> + Optimal salt size is size of used hash algorithm.
> </para>
> + <warning>
> + <para>
> + Although salt is the last optional parameter, salt is the
> + most important parameter for key security. Omitted salt is
> + indication of inappropriate design in most cases. Users must
> + set appropriate salt value whenever it is possible. Omit salt
> + only when it cannot be used.
> + </para>
> + <para>
> + Strong salt is mandatory and must be kept secret when input
> + key is weak, otherwise input key security will not be kept.
> + Even when input key is strong, providing strong salt is the
> + best practice for the best possible key security.
> + </para>
> + <para>
> + Salt must not be able to be controlled by users. i.e. User
> + must not be able to set salt value and get derived key. User
> + controlled salt allows input key analysis to attackers.
> + </para>
> + </warning>
> </listitem>
> </varlistentry>
> </variablelist>
> @@ -101,6 +140,99 @@
> &reftitle.examples;
> <para>
> <example>
> + <title>URI specific CSRF token that supports expiration by
> <function>hash_hkdf</function></title>
> + <programlisting role="php">
> +<![CDATA[
> +<?php
> +define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration
> +define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid
> +
> +/**************************************
> + * Implementation note
> + *
> + * It uses "counter" for CSRF expiration management.
> + * "counter" is very low entropy, but input key is strong and
> + * CSRF_TOKEN_SEED is short term key. It should be OK.
> + *
> + * This CSRF token implementation has pros and cons
> + *
> + * Pros
> + * - A CSRF token is valid only for specific URI.
> + * - No database is required for URI specific CSRF tokens.
> + * - Only CSRF token is required. i.e. No timestamp parameter.
> + * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE *
> CSRF_TOKENS sec.
> + * - Even when user had long idle time, CSRF token is valid.
> + * - CSRF token will expire eventually.
> + * - Invalidating all active CSRF tokens could be done by
> unset($_SESSION['CSRF_TOKEN_SEED']).
> + * It is recommended to reset CSRF tokens by login/logout event at
> least.
> + * It may be good idea to invalidate all of older CSRF tokens when idle
> time is long.
> + *
> + * Cons
> + * - There could be no CSRF expiration time.
> + *
> + * Precise CSRF token expiration is easy. Just add timestamp parameter
> + * as "info" and check it.
> + **************************************/
> +
> +session_start();
> +if (empty($_SESSION['CSRF_TOKEN_SEED'])) {
> + $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32);
> + $_SESSION['CSRF_TOKEN_COUNT'] = 1;
> + $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
> +}
> +
> +
> +function csrf_get_token($uri) {
> + // Check expiration
> + if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) {
> + $_SESSION['CSRF_TOKEN_COUNT']++;
> + $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
> + }
> + // Equivalent(NOT exactly the same) value by using hash_hmac()
> + // return hash_hmac('sha3-256', hash_hmac('sha3-256',
> $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri);
> + return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri,
> $_SESSION['CSRF_TOKEN_COUNT']);
> +}
> +
> +function csrf_validate_token($csrf_token, $uri) {
> + for($i = 0; $i < CSRF_TOKENS; $i++) {
> + // Equivalent(NOT exactly the same) value by using hash_hmac()
> + // $token = hash_hmac('sha3-256', hash_hmac('sha3-256',
> $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri);
> + $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0,
> $uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i);
> + if (hash_equals($csrf_token, $token)) {
> + return TRUE;
> + }
> + }
> + return FALSE;
> +}
> +
> +
> +//// Generating CSRF token ////
> +// $uri is target URI that browser POSTs form data
> +$uri = 'https://example.com/some_form/';
> +$csrf_token = csrf_get_token($uri);
> +// embed $csrf_token to your form
> +
> +//// Validating CSRF token ////
> +$csrf_token = $_POST['csrf_token'] ?? '';
> +if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) {
> + // Invalid CSRF token
> + throw new Exception('CSRF token validation error');
> +}
> +// valid request
> +?>
> +]]>
> + </programlisting>
> + <para>
> + Common CSRF token uses the same token value for a session and all
> + URI. This example CSRF token expires and is specific to a
> + URI. i.e. CSRF token http://example.com/form_A/ is not valid for
> + http://example.com/form_B/ Since token value is computed, no
> + database is required.
> + </para>
> + </example>
> + </para>
> + <para>
> + <example>
> <title><function>hash_hkdf</function> example</title>
> <programlisting role="php">
> <![CDATA[
> @@ -124,6 +256,30 @@
> </para>
> </example>
> </para>
> + <para>
> + <example>
> + <title><function>hash_hkdf</function> bad example</title>
> + <para>
> + Users must not simply extend input key material length. HKDF does
> + not add additional entropy automatically. Therefore, weak key
> + remains weak unless strong salt is supplied. Following is bad
> + example.
> + </para>
> + <programlisting role="php">
> +<![CDATA[
> +<?php
> +$inputKey = get_my_aes128_key(); // AES 128 bit key
> +
> +// Derive AES 256 key from AES 128 key
> +$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption');
> +// Users should not do this. $encryptionKey only has 128 bit
> +// entropy while it should have 256 bit entropy.
> +// To derive strong AES 256 key, strong enough salt is required.
> +?>
> +]]>
> + </programlisting>
> + </example>
> + </para>
> </refsect1>
>
> <refsect1 role="seealso">
> @@ -130,6 +286,7 @@
> &reftitle.seealso;
> <para>
> <simplelist>
> + <member><function>hash_hmac</function></member>
> <member><function>hash_pbkdf2</function></member>
> <member><link xlink:href="&url.rfc;5869">RFC 5869</link></member>
> <member><link
> xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland
> implementation</link></member>
>

Strong -1 on these docs changes. They are wrong and they will confuse users
about when and how HKDF should be used.

I have no idea where you got the idea that HKDF is supposed to be used for
CSRF token generation, but it isn't. I did not check whether your code is
correct and secure, but CSRF token generation is certainly not a common or
typical application of HKDF and as such should not be present in the
documentation.

Your "bad example" is actually pretty much the textbook use-case for HKDF.
The way you wrote it (get a AES-256 key from an AES-128 key) doesn't make
much sense, but the general principle of extracting two keys (for
encryption and authentication) from a single key is one of *the* use-cases
of HKDF. It is also, contrary to your statement in the documentation
snippet, perfectly cryptographically sound. A salt is not required for this
case. A salt *may* be beneficial, but for entirely different reasons (as
Scott pointed out, for many block cipher modes fixed encryption keys only
have a lifetime of around 2^64 encryptions, past which point IV collisions
are to be expected -- a salt in key derivation could mitigate this.)

Nikita
Pieter Hordijk
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 14, 2017 11:50AM
----- Original Message -----
> From: "Nikita Popov" <[email protected]>
> To: "Yasuo Ohgaki" <[email protected]>
> Cc: "Pieter Hordijk" <[email protected]>, "Joe Watkins" <[email protected]>, "Andrey Andreev"
> <[email protected]>, "internals" <[email protected]>, "phpdoc" <[email protected]>
> Sent: Friday, April 14, 2017 11:24:53 AM
> Subject: Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter

> On Thu, Apr 13, 2017 at 11:22 PM, Yasuo Ohgaki < [ mailto:[email protected] |
> yohgaki@ohgaki.net ] > wrote:

>> Hi Pieter and all,

>> On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk < [
>> mailto:[email protected] | info@pieterhordijk.com ] >
>> wrote:

>> > Is this really something we need in our official docs instead of for
>> > example
>> > on a personal blog?


>> I wrote draft doc patch.
>> Please verify.

>> Index: en/reference/hash/functions/hash-hkdf.xml
>> ===================================================================
>> --- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317)
>> +++ en/reference/hash/functions/hash-hkdf.xml (作業コピー)
>> @@ -3,7 +3,7 @@
>> <refentry xml:id="function.hash-hkdf" xmlns=" [ http://docbook.org/ns/docbook |
>> http://docbook.org/ns/docbook ] "
>> xmlns:xlink=" [ http://www.w3.org/1999/xlink | http://www.w3.org/1999/xlink ] ">
>> <refnamediv>
>> <refname>hash_hkdf</refname>
>> - <refpurpose>Generate a HKDF key derivation of a supplied key
>> input</refpurpose>
>> + <refpurpose>Derive secure new key from existing key by using
>> HKDF</refpurpose>
>> </refnamediv>
>> <refsect1 role="description">
>> &reftitle.description;
>> @@ -16,6 +16,20 @@
>> <methodparam
>> choice="opt"><type>string</type><parameter>salt</parameter><initializer>''</initializer></methodparam>
>> </methodsynopsis>

>> + <para>
>> + RFC 5869 defines HKDF (HMAC based Key Derivation Function) which
>> + is general purpose KDF. HKDF could be useful for many PHP
>> + applications that require temporary keys, such CSRF token,
>> + pre-signed key for URI, password for password protected
>> + URI, and so on.
>> + </para>
>> + <note>
>> + <para>
>> + When info and length
>> + is not required for your program, more efficient
>> + <function>hash_hmac</function> could be used instead.
>> + </para>
>> + </note>
>> </refsect1>
>> <refsect1 role="parameters">
>> &reftitle.parameters;
>> @@ -25,7 +39,7 @@
>> <term><parameter>algo</parameter></term>
>> <listitem>
>> <para>
>> - Name of selected hashing algorithm (i.e. "sha256", "sha512",
>> "haval160,4", etc..)
>> + Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512",
>> "sha256", "sha512", "haval160,4", etc..)
>> See <function>hash_algos</function> for a list of supported
>> algorithms.
>> <note>
>> <para>
>> @@ -39,7 +53,7 @@
>> <term><parameter>ikm</parameter></term>
>> <listitem>
>> <para>
>> - Input keying material (raw binary). Cannot be empty.
>> + Input keying material. Cannot be empty.
>> </para>
>> </listitem>
>> </varlistentry>
>> @@ -60,7 +74,8 @@
>> <term><parameter>info</parameter></term>
>> <listitem>
>> <para>
>> - Application/context-specific info string.
>> + Application/context-specific info string. Info is intended for
>> + public information such as user ID, protocol version, etc.
>> </para>
>> </listitem>
>> </varlistentry>
>> @@ -71,8 +86,32 @@
>> Salt to use during derivation.
>> </para>
>> <para>
>> - While optional, adding random salt significantly improves the
>> strength of HKDF.
>> + While optional, adding random salt significantly improves the
>> + strength of HKDF. Salt could be either secret or
>> + non-secret. It is used as "Pre Shared Key" in many use cases.
>> + Strong value is preferred. e.g. Use
>> <function>random_bytes</function>.
>> + Optimal salt size is size of used hash algorithm.
>> </para>
>> + <warning>
>> + <para>
>> + Although salt is the last optional parameter, salt is the
>> + most important parameter for key security. Omitted salt is
>> + indication of inappropriate design in most cases. Users must
>> + set appropriate salt value whenever it is possible. Omit salt
>> + only when it cannot be used.
>> + </para>
>> + <para>
>> + Strong salt is mandatory and must be kept secret when input
>> + key is weak, otherwise input key security will not be kept.
>> + Even when input key is strong, providing strong salt is the
>> + best practice for the best possible key security.
>> + </para>
>> + <para>
>> + Salt must not be able to be controlled by users. i.e. User
>> + must not be able to set salt value and get derived key. User
>> + controlled salt allows input key analysis to attackers.
>> + </para>
>> + </warning>
>> </listitem>
>> </varlistentry>
>> </variablelist>
>> @@ -101,6 +140,99 @@
>> &reftitle.examples;
>> <para>
>> <example>
>> + <title>URI specific CSRF token that supports expiration by
>> <function>hash_hkdf</function></title>
>> + <programlisting role="php">
>> +<![CDATA[
>> +<?php
>> +define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration
>> +define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid
>> +
>> +/**************************************
>> + * Implementation note
>> + *
>> + * It uses "counter" for CSRF expiration management.
>> + * "counter" is very low entropy, but input key is strong and
>> + * CSRF_TOKEN_SEED is short term key. It should be OK.
>> + *
>> + * This CSRF token implementation has pros and cons
>> + *
>> + * Pros
>> + * - A CSRF token is valid only for specific URI.
>> + * - No database is required for URI specific CSRF tokens.
>> + * - Only CSRF token is required. i.e. No timestamp parameter.
>> + * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE *
>> CSRF_TOKENS sec.
>> + * - Even when user had long idle time, CSRF token is valid.
>> + * - CSRF token will expire eventually.
>> + * - Invalidating all active CSRF tokens could be done by
>> unset($_SESSION['CSRF_TOKEN_SEED']).
>> + * It is recommended to reset CSRF tokens by login/logout event at
>> least.
>> + * It may be good idea to invalidate all of older CSRF tokens when idle
>> time is long.
>> + *
>> + * Cons
>> + * - There could be no CSRF expiration time.
>> + *
>> + * Precise CSRF token expiration is easy. Just add timestamp parameter
>> + * as "info" and check it.
>> + **************************************/
>> +
>> +session_start();
>> +if (empty($_SESSION['CSRF_TOKEN_SEED'])) {
>> + $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32);
>> + $_SESSION['CSRF_TOKEN_COUNT'] = 1;
>> + $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
>> +}
>> +
>> +
>> +function csrf_get_token($uri) {
>> + // Check expiration
>> + if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) {
>> + $_SESSION['CSRF_TOKEN_COUNT']++;
>> + $_SESSION['CSRF_TOKEN_EXPIRE'] = time();
>> + }
>> + // Equivalent(NOT exactly the same) value by using hash_hmac()
>> + // return hash_hmac('sha3-256', hash_hmac('sha3-256',
>> $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri);
>> + return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri,
>> $_SESSION['CSRF_TOKEN_COUNT']);
>> +}
>> +
>> +function csrf_validate_token($csrf_token, $uri) {
>> + for($i = 0; $i < CSRF_TOKENS; $i++) {
>> + // Equivalent(NOT exactly the same) value by using hash_hmac()
>> + // $token = hash_hmac('sha3-256', hash_hmac('sha3-256',
>> $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri);
>> + $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0,
>> $uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i);
>> + if (hash_equals($csrf_token, $token)) {
>> + return TRUE;
>> + }
>> + }
>> + return FALSE;
>> +}
>> +
>> +
>> +//// Generating CSRF token ////
>> +// $uri is target URI that browser POSTs form data
>> +$uri = ' [ https://example.com/some_form/ | https://example.com/some_form/ ] ';
>> +$csrf_token = csrf_get_token($uri);
>> +// embed $csrf_token to your form
>> +
>> +//// Validating CSRF token ////
>> +$csrf_token = $_POST['csrf_token'] ?? '';
>> +if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) {
>> + // Invalid CSRF token
>> + throw new Exception('CSRF token validation error');
>> +}
>> +// valid request
>> +?>
>> +]]>
>> + </programlisting>
>> + <para>
>> + Common CSRF token uses the same token value for a session and all
>> + URI. This example CSRF token expires and is specific to a
>> + URI. i.e. CSRF token [ http://example.com/form_A/ | http://example.com/form_A/
>> ] is not valid for
>> + [ http://example.com/form_B/ | http://example.com/form_B/ ] Since token value
>> is computed, no
>> + database is required.
>> + </para>
>> + </example>
>> + </para>
>> + <para>
>> + <example>
>> <title><function>hash_hkdf</function> example</title>
>> <programlisting role="php">
>> <![CDATA[
>> @@ -124,6 +256,30 @@
>> </para>
>> </example>
>> </para>
>> + <para>
>> + <example>
>> + <title><function>hash_hkdf</function> bad example</title>
>> + <para>
>> + Users must not simply extend input key material length. HKDF does
>> + not add additional entropy automatically. Therefore, weak key
>> + remains weak unless strong salt is supplied. Following is bad
>> + example.
>> + </para>
>> + <programlisting role="php">
>> +<![CDATA[
>> +<?php
>> +$inputKey = get_my_aes128_key(); // AES 128 bit key
>> +
>> +// Derive AES 256 key from AES 128 key
>> +$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption');
>> +// Users should not do this. $encryptionKey only has 128 bit
>> +// entropy while it should have 256 bit entropy.
>> +// To derive strong AES 256 key, strong enough salt is required.
>> +?>
>> +]]>
>> + </programlisting>
>> + </example>
>> + </para>
>> </refsect1>

>> <refsect1 role="seealso">
>> @@ -130,6 +286,7 @@
>> &reftitle.seealso;
>> <para>
>> <simplelist>
>> + <member><function>hash_hmac</function></member>
>> <member><function>hash_pbkdf2</function></member>
>> <member><link xlink:href="&url.rfc;5869">RFC 5869</link></member>
>> <member><link
>> xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland
>> implementation</link></member>

As I already said yesterday and which you chose to ignore. The manual should
describe the functions and the paramaters and that's it. Maybe add a
warning / note where it is warranted, or even link to some external resource
(e.g. owasp) when it really is needed.

However when reading the above changes I think: great blog post or nice
OSS library shared on github for people to use. Imo all these examples
and opinionated talk about things being mandatory have no place in our manual.

Look at the page of crypt (http://php.net/manual/en/function.crypt.php). There
is a short text in the intro about password hashing with password_hash and a
warning.

Look at the page of openssl_encrypt (http://php.net/manual/en/function.openssl-encrypt.php).
Nowhere on that page is being said it is mandatory to not use ECB mode.

So as far as I'm concerned I personally don't agree with the above changes, because
it's trying to use the manual for something it's not meant for (describing functions
and their parameters).

So please just keep it at that and write a blog post / open source library
for everything else.

I have the feeling you keep adding your own personal preferences to the manual.

Also note I am not even talking about the actual technical implications as Nikita
done below.

So a big -1 from me too.

> Strong -1 on these docs changes. They are wrong and they will confuse users
> about when and how HKDF should be used.

> I have no idea where you got the idea that HKDF is supposed to be used for CSRF
> token generation, but it isn't. I did not check whether your code is correct
> and secure, but CSRF token generation is certainly not a common or typical
> application of HKDF and as such should not be present in the documentation.

> Your "bad example" is actually pretty much the textbook use-case for HKDF.. The
> way you wrote it (get a AES-256 key from an AES-128 key) doesn't make much
> sense, but the general principle of extracting two keys (for encryption and
> authentication) from a single key is one of *the* use-cases of HKDF. It is
> also, contrary to your statement in the documentation snippet, perfectly
> cryptographically sound. A salt is not required for this case. A salt *may* be
> beneficial, but for entirely different reasons (as Scott pointed out, for many
> block cipher modes fixed encryption keys only have a lifetime of around 2^64
> encryptions, past which point IV collisions are to be expected -- a salt in key
> derivation could mitigate this.)


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Hello,

The PHP documentation has a separate place for detailed examples.
For example:

http://php.net/manual/en/book.inclued.php
http://php.net/manual/en/inclued.examples-implementation.php

The same could be done for ext/hash which today lacks an Examples
section:

http://php.net/manual/en/book.hash.php

This way the function's docs (hash_hkdf() in this case) remain as
reference material while detailed examples live elsewhere.

Regards,
Philip

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 15, 2017 02:00AM
Hi Nikita,

On Fri, Apr 14, 2017 at 6:24 PM, Nikita Popov <[email protected]> wrote:

> Strong -1 on these docs changes. They are wrong and they will confuse
> users about when and how HKDF should be used.
>
> I have no idea where you got the idea that HKDF is supposed to be used for
> CSRF token generation, but it isn't. I did not check whether your code is
> correct and secure, but CSRF token generation is certainly not a common or
> typical application of HKDF and as such should not be present in the
> documentation.
>
> Your "bad example" is actually pretty much the textbook use-case for HKDF.
> The way you wrote it (get a AES-256 key from an AES-128 key) doesn't make
> much sense, but the general principle of extracting two keys (for
> encryption and authentication) from a single key is one of *the* use-cases
> of HKDF. It is also, contrary to your statement in the documentation
> snippet, perfectly cryptographically sound. A salt is not required for this
> case. A salt *may* be beneficial, but for entirely different reasons (as
> Scott pointed out, for many block cipher modes fixed encryption keys only
> have a lifetime of around 2^64 encryptions, past which point IV collisions
> are to be expected -- a salt in key derivation could mitigate this.)
>

It seems you consider HKDF as very specific function for very specific
crypt task
which is wrong by the RFC 5869 intention.
The RFC 5869 explicitly mentions as

4. Applications of HKDF
HKDF is intended for use in a wide variety of KDF applications.

Why we must limit HKDF usage for certain crypt tasks even if it is
designed for _general_ Key Derivation tasks?

Key derivations in authentication is very common task.
CSRF token is "Key that validates the _authentic_ request".
It is obvious that expiration enabled URI specific CSRF token is
a lot secure than common static CSRF tokens that are valid for all requests.
How this could be bad example?

128 bit entropy key for AES 256 is simply bad practice like
$aes256key = hash('sha256', 'mypassword', true);

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 15, 2017 02:30AM
Hi Pieter,

On Fri, Apr 14, 2017 at 6:45 PM, Pieter Hordijk <[email protected]>
wrote:

>
> I have the feeling you keep adding your own personal preferences to the
> manual.


No, not at all.
My opinions are either based on concrete logic or
recommendations based reliable sources.

I improved hash_hkdf() manual farther more based on RFC 5869 descriptions.
https://gist.github.com/anonymous/ace4fa267f20041676f265fe58c3f1ea

Please verify it again.

Regards,

P.S. I'm a human, so I make mistakes. I appreciate if one could
point out when my logic is wrong.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 22, 2017 09:20PM
Hi all,

On Sat, Apr 15, 2017 at 9:17 AM, Yasuo Ohgaki <[email protected]> wrote:

> My opinions are either based on concrete logic or
> recommendations based reliable sources.
>
> I improved hash_hkdf() manual farther more based on RFC 5869 descriptions.
> https://gist.github.com/anonymous/ace4fa267f20041676f265fe58c3f1ea
>
> Please verify it again.
>

I would like to finish documentation.

RFC 5869 clearly states HKDF is a generic key derivation function.

Omitting salt when key does not have enough entropy is obvious
bad practice or mistake. Even when key has enough entropy, long
life key (IKM) requires good salt for the best key security. These
could be understood from the RFC and other basic crypt theory.

I'll commit the doc in a few days if there is no more comments on this.

Andrey, (Or anyone who vote no for the PHP RFC)

Could you show some good example hash_hkdf() usages that justify
current function signature? I suppose you should have few common and
good examples.

I cannot think of any common/good example that uses length only or
length/info only. No good example is shown so far.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Niklas Keller
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 22, 2017 09:40PM
2017-04-22 21:14 GMT+02:00 Yasuo Ohgaki <[email protected]>:

> Hi all,
>
> On Sat, Apr 15, 2017 at 9:17 AM, Yasuo Ohgaki <[email protected]> wrote:
>
> > My opinions are either based on concrete logic or
> > recommendations based reliable sources.
> >
> > I improved hash_hkdf() manual farther more based on RFC 5869
> descriptions.
> > https://gist.github.com/anonymous/ace4fa267f20041676f265fe58c3f1ea
> >
> > Please verify it again.
> >
>
> I would like to finish documentation.
>
> RFC 5869 clearly states HKDF is a generic key derivation function.
>
> Omitting salt when key does not have enough entropy is obvious
> bad practice or mistake. Even when key has enough entropy, long
> life key (IKM) requires good salt for the best key security. These
> could be understood from the RFC and other basic crypt theory.
>
> I'll commit the doc in a few days if there is no more comments on this.
>

What the... there were multiple concerns regarding the changes already. I'm
hereby expressing another strong -1 on these.

Regards, Niklas


> Andrey, (Or anyone who vote no for the PHP RFC)
>
> Could you show some good example hash_hkdf() usages that justify
> current function signature? I suppose you should have few common and
> good examples.
>
> I cannot think of any common/good example that uses length only or
> length/info only. No good example is shown so far.
>
> Regards,
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 22, 2017 09:50PM
Hi Niklas,

On Sun, Apr 23, 2017 at 4:32 AM, Niklas Keller <[email protected]> wrote:

>
> What the... there were multiple concerns regarding the changes already.
> I'm hereby expressing another strong -1 on these.
>

Instead of posting your feeling, please post logic behind your idea.
Most of the changes are based on what is _written_ in the RFC 5869

I'm a bit tired with arguments without valid logic.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Andrey Andreev
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 23, 2017 12:30AM
Hi,

On Sat, Apr 22, 2017 at 10:37 PM, Yasuo Ohgaki <[email protected]> wrote:
> Hi Niklas,
>
> On Sun, Apr 23, 2017 at 4:32 AM, Niklas Keller <[email protected]> wrote:
>>
>>
>> What the... there were multiple concerns regarding the changes already.
>> I'm hereby expressing another strong -1 on these.
>
>
> Instead of posting your feeling, please post logic behind your idea.
> Most of the changes are based on what is _written_ in the RFC 5869
>
> I'm a bit tired with arguments without valid logic.

You're tired? Yasuo, the reason why you're not receiving replies
unless you say "I'll commit in a few days if there are no more
comments" is because everybody is tired of talking to you.

If you want examples, search GitHub for PHP code utilizing HKDF - you
will see that most projects use it without a salt, including
https://github.com/defuse/php-encryption - pretty much the best PHP
userspace crypto library today. And I'm only saying "most" because I
can't be bothered to go through literally all of them; I've found NONE
that do use the salt.
You will also find zero projects using it for CSRF protection.

The vote ended with 1 Yes (you) and 14 No; not a single person has
agreed with you so far, and most have explicitly stated strong
disagreement with your proposed changes. Yet you insist on pushing
your *personal opinion*, ignoring everybody else and acting as if ~80
mails haven't already been exchanged.

How is it even possible that you still believe that everybody is wrong
and you alone are right? Give it up already.

Cheers,
Andrey.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 25, 2017 02:40AM
Hi Andrey,

On Sun, Apr 23, 2017 at 7:24 AM, Andrey Andreev <[email protected]> wrote:

>
> You're tired? Yasuo, the reason why you're not receiving replies
> unless you say "I'll commit in a few days if there are no more
> comments" is because everybody is tired of talking to you.
>
> If you want examples, search GitHub for PHP code utilizing HKDF - you
> will see that most projects use it without a salt, including
> https://github.com/defuse/php-encryption - pretty much the best PHP
> userspace crypto library today. And I'm only saying "most" because I
> can't be bothered to go through literally all of them; I've found NONE
> that do use the salt.
>

Wrong.
I don't think the author wouldn't make such mistake, so I checked.

/**
* Derives authentication and encryption keys from the secret, using a
slow
* key derivation function if the secret is a password.
*
* @param string $salt
*
* @throws Ex\EnvironmentIsBrokenException
*
* @return DerivedKeys
*/
public function deriveKeys($salt)
{
if (Core::ourStrlen($salt) !== Core::SALT_BYTE_SIZE) {
throw new Ex\EnvironmentIsBrokenException('Bad salt.');
}

if ($this->secret_type === self::SECRET_TYPE_KEY) {
$akey = Core::HKDF(
Core::HASH_FUNCTION_NAME,
$this->secret->getRawBytes(),
Core::KEY_BYTE_SIZE,
Core::AUTHENTICATION_INFO_STRING,
$salt
);
$ekey = Core::HKDF(
Core::HASH_FUNCTION_NAME,
$this->secret->getRawBytes(),
Core::KEY_BYTE_SIZE,
Core::ENCRYPTION_INFO_STRING,
$salt
);
return new DerivedKeys($akey, $ekey);
} elseif ($this->secret_type === self::SECRET_TYPE_PASSWORD) {





> You will also find zero projects using it for CSRF protection.
>

You obviously does not understand HKDF RFC at all. (And don't read my reply)
It seems you consider HKDF as a specific KDF, but it is _not_.

HKDF is designed as general purpose KDF. It is clearly stated in RFC 5869

4 <https://tools.ietf.org/html/rfc5869#section-4>;. Applications of HKDF

HKDF is intended for use in a wide variety of KDF applications.


Just because you cannot think of how general purpose KDF could be used
for other purposes, it does not mean it should not be used other purposes.
Especially when it is designed for general purpose in the first place.


The vote ended with 1 Yes (you) and 14 No; not a single person has
> agreed with you so far, and most have explicitly stated strong
> disagreement with your proposed changes. Yet you insist on pushing
> your *personal opinion*, ignoring everybody else and acting as if ~80
> mails haven't already been exchanged.
>

> How is it even possible that you still believe that everybody is wrong
> and you alone are right? Give it up already.


Prove my idea in the manual (or my RFC) is wrong by logic, rather than FUD.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 25, 2017 02:50AM
On Tue, Apr 25, 2017 at 9:28 AM, Yasuo Ohgaki <[email protected]> wrote:

> I don't think the author wouldn't make such mistake, so I checked.
>

Oops. Double denial.
I thought the author wouldn't make such mistake, so I checked.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Andrey Andreev
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 25, 2017 12:20PM
Hi,

On Tue, Apr 25, 2017 at 3:28 AM, Yasuo Ohgaki <[email protected]> wrote:
>>
>> If you want examples, search GitHub for PHP code utilizing HKDF - you
>> will see that most projects use it without a salt, including
>> https://github.com/defuse/php-encryption - pretty much the best PHP
>> userspace crypto library today. And I'm only saying "most" because I
>> can't be bothered to go through literally all of them; I've found NONE
>> that do use the salt.
>
>
> Wrong.
> I don't think the author wouldn't make such mistake, so I checked.
>
> /**
> * Derives authentication and encryption keys from the secret, using a
> slow
> * key derivation function if the secret is a password.
> *
> * @param string $salt
> *
> * @throws Ex\EnvironmentIsBrokenException
> *
> * @return DerivedKeys
> */
> public function deriveKeys($salt)
> {
> if (Core::ourStrlen($salt) !== Core::SALT_BYTE_SIZE) {
> throw new Ex\EnvironmentIsBrokenException('Bad salt.');
> }
>
> if ($this->secret_type === self::SECRET_TYPE_KEY) {
> $akey = Core::HKDF(
> Core::HASH_FUNCTION_NAME,
> $this->secret->getRawBytes(),
> Core::KEY_BYTE_SIZE,
> Core::AUTHENTICATION_INFO_STRING,
> $salt
> );
> $ekey = Core::HKDF(
> Core::HASH_FUNCTION_NAME,
> $this->secret->getRawBytes(),
> Core::KEY_BYTE_SIZE,
> Core::ENCRYPTION_INFO_STRING,
> $salt
> );
> return new DerivedKeys($akey, $ekey);
> } elseif ($this->secret_type === self::SECRET_TYPE_PASSWORD) {
>
>

Fair enough, it uses a salt somewhere I didn't see - as I said, I
didn't check literally everything.
It doesn't use it here:
https://github.com/defuse/php-encryption/blob/0364e3ea20d2382e709034e972d474f551c3273c/src/Crypto.php#L124

>>
>> You will also find zero projects using it for CSRF protection.
>
>
> You obviously does not understand HKDF RFC at all. (And don't read my reply)
> It seems you consider HKDF as a specific KDF, but it is _not_.
>

I'm telling you nobody uses it for CSRF and you can't disprove that,
but somehow that means I don't understand RFC 5869?!

> HKDF is designed as general purpose KDF. It is clearly stated in RFC 5869
>
> 4. Applications of HKDF
>
> HKDF is intended for use in a wide variety of KDF applications.
>
>
> Just because you cannot think of how general purpose KDF could be used
> for other purposes, it does not mean it should not be used other purposes.
> Especially when it is designed for general purpose in the first place.
>

First of all, KDF is a *cryptographic* term.
The fact that you don't know this should disqualify you of even being
involved in this discussion, and it is laughable that you're trying to
tell anybody that they don't understand RFC5869.

Secondly, you're cherry-picking a single sentence, out of context and
twisting its meaning to serve your personal agenda.
Here's the entire paragraph in question:

HKDF is intended for use in a wide variety of KDF applications.
These include the building of pseudorandom generators from imperfect
sources of randomness (such as a physical random number generator
(RNG)); the generation of pseudorandomness out of weak sources of
randomness, such as entropy collected from system events, user's
keystrokes, etc.; the derivation of cryptographic keys from a shared
Diffie-Hellman value in a key-agreement protocol; derivation of
symmetric keys from a hybrid public-key encryption scheme; key
derivation for key-wrapping mechanisms; and more. All of these
applications can benefit from the simplicity and multi-purpose nature
of HKDF, as well as from its analytical foundation.

Link: https://tools.ietf.org/html/rfc5869#section-4

And finally, but what is most important as far as PHP documentation goes:

- Can HKDF be somehow used for CSRF protection? Sure, a lot of things
*happen* to be usable for different things they weren't intended for.
- Should HKDF be used for CSRF protection? Maybe, if you want an
overkill solution - a simple HMAC is more than sufficient if you want
to couple your CSRF tokens with the server state.
- Is CSRF token generation what HKDF was made for? Absolutely NOT,
and you know it.

It's none of our business to *invent* new use cases and document them
as if that's what everybody should do.
For all I care, use it however you wish in your own code, but the PHP
documentation is not your personal blog.

>> The vote ended with 1 Yes (you) and 14 No; not a single person has
>> agreed with you so far, and most have explicitly stated strong
>> disagreement with your proposed changes. Yet you insist on pushing
>> your *personal opinion*, ignoring everybody else and acting as if ~80
>> mails haven't already been exchanged.
>>
>>
>> How is it even possible that you still believe that everybody is wrong
>> and you alone are right? Give it up already.
>
>
> Prove my idea in the manual (or my RFC) is wrong by logic, rather than FUD.
>

I just did, but since you insist - there's more.

Your RFC lists multiple examples of using *user-provided plain-text
passwords* for IKM, and you describe those as valid use cases.

Well, I was prepared to compare HKDF to PBKDF2, as the latter also
uses HMACs and a salt, but I don't even have to do that.
Because guess what Section 4 of RFC 5869 says about passwords? The
very same "Applications of HKDF" section that you cherry-pick from:

On the other hand, it is anticipated that some applications will not
be able to use HKDF "as-is" due to specific operational requirements,
or will be able to use it but without the full benefits of the
scheme. One significant example is the derivation of cryptographic
keys from a source of low entropy, such as a user's password. The
extract step in HKDF can concentrate existing entropy but cannot
amplify entropy. In the case of password-based KDFs, a main goal is
to slow down dictionary attacks using two ingredients: a salt value,
and the intentional slowing of the key derivation computation. HKDF
naturally accommodates the use of salt; however, a slowing down
mechanism is not part of this specification. Applications interested
in a password-based KDF should consider whether, for example, [PKCS5]
meets their needs better than HKDF.

Link: https://tools.ietf.org/html/rfc5869#section-4

And this doesn't stop at passwords. Please note that this paragraph
explicitly states this:

The extract step in HKDF can concentrate existing entropy but
cannot amplify entropy.

Which means that it is NOT designed to do key stretching, or in other
words it should NOT be relied upon to produce strong outputs from weak
inputs - the exact scenario for which you wanted to make salts
non-optional.

---------------------

Is this FUD? Do we all still not understand HKDF, while you're the
only person on the planet who does?

And will you at least stop with the "I will commit unless somebody
comments" emails?
When everybody is unanimously against your changes, that means don't do them.

Cheers,
Andrey.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 30, 2017 01:20AM
Hi Andrey,

On Tue, Apr 25, 2017 at 7:17 PM, Andrey Andreev <[email protected]> wrote:

> Hi,
>
> On Tue, Apr 25, 2017 at 3:28 AM, Yasuo Ohgaki <[email protected]> wrote:
> >>
> >> If you want examples, search GitHub for PHP code utilizing HKDF - you
> >> will see that most projects use it without a salt, including
> >> https://github.com/defuse/php-encryption - pretty much the best PHP
> >> userspace crypto library today. And I'm only saying "most" because I
> >> can't be bothered to go through literally all of them; I've found NONE
> >> that do use the salt.
> >
> >
> > Wrong.
> > I don't think the author wouldn't make such mistake, so I checked.
> >
> > /**
> > * Derives authentication and encryption keys from the secret, using
> a
> > slow
> > * key derivation function if the secret is a password.
> > *
> > * @param string $salt
> > *
> > * @throws Ex\EnvironmentIsBrokenException
> > *
> > * @return DerivedKeys
> > */
> > public function deriveKeys($salt)
> > {
> > if (Core::ourStrlen($salt) !== Core::SALT_BYTE_SIZE) {
> > throw new Ex\EnvironmentIsBrokenException('Bad salt.');
> > }
> >
> > if ($this->secret_type === self::SECRET_TYPE_KEY) {
> > $akey = Core::HKDF(
> > Core::HASH_FUNCTION_NAME,
> > $this->secret->getRawBytes(),
> > Core::KEY_BYTE_SIZE,
> > Core::AUTHENTICATION_INFO_STRING,
> > $salt
> > );
> > $ekey = Core::HKDF(
> > Core::HASH_FUNCTION_NAME,
> > $this->secret->getRawBytes(),
> > Core::KEY_BYTE_SIZE,
> > Core::ENCRYPTION_INFO_STRING,
> > $salt
> > );
> > return new DerivedKeys($akey, $ekey);
> > } elseif ($this->secret_type === self::SECRET_TYPE_PASSWORD) {
> >
> >
>
> Fair enough, it uses a salt somewhere I didn't see - as I said, I
> didn't check literally everything.
> It doesn't use it here:
> https://github.com/defuse/php-encryption/blob/
> 0364e3ea20d2382e709034e972d474f551c3273c/src/Crypto.php#L124
>
>
It is in
public static function legacyDecrypt($ciphertext, $key)
which is legacy(old and not recommended) way.


> >>
> >> You will also find zero projects using it for CSRF protection.
> >
> >
> > You obviously does not understand HKDF RFC at all. (And don't read my
> reply)
> > It seems you consider HKDF as a specific KDF, but it is _not_.
> >
>
> I'm telling you nobody uses it for CSRF and you can't disprove that,
> but somehow that means I don't understand RFC 5869?!
>

> > HKDF is designed as general purpose KDF. It is clearly stated in RFC 5869
> >
> > 4. Applications of HKDF
> >
> > HKDF is intended for use in a wide variety of KDF applications.
> >
> >
> > Just because you cannot think of how general purpose KDF could be used
> > for other purposes, it does not mean it should not be used other
> purposes.
> > Especially when it is designed for general purpose in the first place.
> >
>
> First of all, KDF is a *cryptographic* term.
>
The fact that you don't know this should disqualify you of even being
> involved in this discussion, and it is laughable that you're trying to
> tell anybody that they don't understand RFC5869.
>

KDF is "Key Derivation Function".
HKDF is designed as "General KDF" as RFC 5869 explicitly states in Section
4.


>
> Secondly, you're cherry-picking a single sentence, out of context and
> twisting its meaning to serve your personal agenda.
> Here's the entire paragraph in question:
>

> HKDF is intended for use in a wide variety of KDF applications.
> These include the building of pseudorandom generators from imperfect
> sources of randomness (such as a physical random number generator
> (RNG)); the generation of pseudorandomness out of weak sources of
> randomness, such as entropy collected from system events, user's
> keystrokes, etc.; the derivation of cryptographic keys from a shared
> Diffie-Hellman value in a key-agreement protocol; derivation of
> symmetric keys from a hybrid public-key encryption scheme; key
> derivation for key-wrapping mechanisms; and more. All of these
> applications can benefit from the simplicity and multi-purpose nature
> of HKDF, as well as from its analytical foundation.
>
> Link: https://tools.ietf.org/html/rfc5869#section-4
>

Again, RFC 5869 explicitly states it is a "General KDF". There are
specific examples, but there is _NO_ statement that defines HKDF
for specific purpose.

It even explains strange (but valid) KDF usage as CSPRNG.

My CSRF token example is obviously a good KDF application example.
What makes you think it is a bad one?


> And finally, but what is most important as far as PHP documentation goes:
>
> - Can HKDF be somehow used for CSRF protection? Sure, a lot of things
> *happen* to be usable for different things they weren't intended for.
>

"HKDF is intended for use in a wide variety of KDF applications."
Nothing wrong for using wide variety of KDF tasks.


> - Should HKDF be used for CSRF protection? Maybe, if you want an
> overkill solution - a simple HMAC is more than sufficient if you want
> to couple your CSRF tokens with the server state.
>

Overkill? Not at all. Should I list reasons why advanced CSRF token by HKDF
is required/better?

I repeat an explanation in my PHP RFC.
NOTE: String concatenations have risk.

If moderately secure hashing is acceptable, one could use hash() for
URI specific CSRF token with expiration like

$csrf_token = hash('sha3-256', $csrf_token_seed . $expire_timestamp .
$the_uri);

This is obviously worse way because of the "String concatenation".
i.e. $csrf_token_seed . $expire_timestamp . $the_uri
Better way is not to concatenate string.

$csrf_token = hash_hmac('sha3-256', $csrf_token_seed, $expire_timestamp .
$the_uri);

This is still worse way because of the "String concatenation".
i.e. $expire_timestamp . $the_uri // Where $expire_timestamp is low
entropy salt.
Again, better way is not to concatenate string.

$csrf_token = bin2hex(hash_hkdf('sha3-256', $csrf_token_seed, 0,
$the_uri, $expire_timestamp ));



> - Is CSRF token generation what HKDF was made for? Absolutely NOT,
> and you know it.
>

Absolutely valid because it is "Designed as general purpose KDF".

Even though RFC 5869 explicitly states HKDF is not designed for "slow
hashing" like PBKDF2(hash_pbkdf2) or crypt(hash_password), it
could be used for password hashing _IF_ OKM and/or Salt could be
stored in _secure_ system(s). i.e. OKM and/or Salt could be stored
in other secure server(s).


> It's none of our business to *invent* new use cases and document them
> as if that's what everybody should do.
> For all I care, use it however you wish in your own code, but the PHP
> documentation is not your personal blog.
>

Did you read my reply to Nikita?
CSRF token is a "Authentication Key" that validates request is _authentic_.
Therefore, the CSRF example is one of a perfect HKDF application example
even with your claim "HKDF is for cryptographic task".

You insist HKDF is only for specific cryptographic tasks (I still don't
know
what you mean by this exactly. No valid examples yet.), how
_authentication_
can be non cryptographic task?


>> The vote ended with 1 Yes (you) and 14 No; not a single person has
> >> agreed with you so far, and most have explicitly stated strong
> >> disagreement with your proposed changes. Yet you insist on pushing
> >> your *personal opinion*, ignoring everybody else and acting as if ~80
> >> mails haven't already been exchanged.
> >>
> >>
> >> How is it even possible that you still believe that everybody is wrong
> >> and you alone are right? Give it up already.
> >
> >
> > Prove my idea in the manual (or my RFC) is wrong by logic, rather than
> FUD.
> >
>
> I just did, but since you insist - there's more.
>

You don't prove any.

You only states "How RFC 5869 should be interpreted with your
understanding".

Previous hash(), hash_hmac() and hash_hkdf() example is a logical
explanation
why hash_hkdf()/HKDF is needed. i.e. String concatenations involve risks
even
with cryptographic hashes. Separating HMAC "Key" into "Salt" and "Info"
makes
HKDF useful to wide variety of applications. Optional "length" parameter
adds
more use cases which are rare.

RFC 5869's HKDF is made for general purpose KDF obviously. Does RFC
5869 limit or specify the usage? ABSOLUTELY NOT.

You seem misunderstood usage examples showed in RFC is the only HKDF
applications. You seem misunderstood common mistake warnings, too.


Your RFC lists multiple examples of using *user-provided plain-text
> passwords* for IKM, and you describe those as valid use cases as follows:
>
> Well, I was prepared to compare HKDF to PBKDF2, as the latter also
> uses HMACs and a salt, but I don't even have to do that.
> Because guess what Section 4 of RFC 5869 says about passwords? The
> very same "Applications of HKDF" section that you cherry-pick from:
>
> On the other hand, it is anticipated that some applications will not
> be able to use HKDF "as-is" due to specific operational requirements,
> or will be able to use it but without the full benefits of the
> scheme. One significant example is the derivation of cryptographic
> keys from a source of low entropy, such as a user's password. The
> extract step in HKDF can concentrate existing entropy but cannot
> amplify entropy. In the case of password-based KDFs, a main goal is
> to slow down dictionary attacks using two ingredients: a salt value,
> and the intentional slowing of the key derivation computation. HKDF
> naturally accommodates the use of salt; however, a slowing down
> mechanism is not part of this specification. Applications interested
> in a password-based KDF should consider whether, for example, [PKCS5]
> meets their needs better than HKDF.
>
> Link: https://tools.ietf.org/html/rfc5869#section-4
>
> And this doesn't stop at passwords. Please note that this paragraph
> explicitly states this:
>
> The extract step in HKDF can concentrate existing entropy but
> cannot amplify entropy.
>
> Which means that it is NOT designed to do key stretching, or in other
> words it should NOT be relied upon to produce strong outputs from weak
> inputs - the exact scenario for which you wanted to make salts
> non-optional.
>

I agree no KDF including HKDF will provide automatically secure keys by
itself.
This is the reason why the manual explicitly should state best practices
and
common mistakes. (And insisting "salt" as the first required parameter)

The explanations are warnings for "HKDF as slow hashing" and "non secret
salt".

Applications, that _must_ store "Salt" and "Resulted Hash"(OKM) in the
_same_ DB, must not use HKDF "as-is".

OKM and/or Salt must be stored in _secure_ system(s) when HKDF(or HMAC)
is used for user entered password. This is explained already in this mail.

"Non secret salt" never increase entropy. (This is common KDF application
mistake)
With HKDF, PRK must have enough entropy which attacker cannot guess
with reasonable computation time.
i.e. HMAC(IKM, Salt) must be secure.

Strong key derivations, that make strong encryption keys from poor keys
like
user entered password, are very _common_ tasks. Users must do whenever
it is possible.

e.g.
$secure_aes256_key = hex2bin(hash_hmac('sha3-256', $poor_user_password,
$strong_SECRET_salt));
OR with hash_hkdf()
$secure_aes256_key = hash_hkdf('sha3-256', $poor_user_password, 0, '',
$strong_SECRET_salt);

Salt obviously must be _SECRET_ (and have enough entropy), otherwise
simple brute force attack works.

I'm not sure how many times I showed this logically obvious example.

Random strong _SECRET_ salt does produce strong PRK regardless of IKM,
therefore increase OKM entropy. Explanation is omitted because it is
obvious.


> Is this FUD? Do we all still not understand HKDF, while you're the
> only person on the planet who does?
>

You are insisting false FUD that HKDF is for specific task(s) and valid
KDF usages as invalid. In this mail, you are insisting HKDF cannot
increase OKM entropy while random _secret_ salt can.

No cryptographers will recommend "Omitting salt", but recommend it
whenever it's possible.

No cryptographers will consider HKDF based advanced CSRF token as
invalid HKDF application.

If any, please let me know.

Anyway, salt as the last optional parameter is total nonsense, claim that
KDF
based CSRF as invalid HKDF application as well.

And will you at least stop with the "I will commit unless somebody
> comments" emails?
> When everybody is unanimously against your changes, that means don't do
> them.
>

Not everybody, I presume.
I don't see logically correct objections yet.

Regards,

P.S.
I've presented more than enough good HKDF usage examples that users
should/must provide proper salt in my PHP RFC.

You show _no_ examples that are valid/recommended/common HKDF
usage only with length and length/info. Please give some examples that
could be common and useful like advanced CSRF token by HKDF.

I don't need your view of HKDF RFC or usage, but I do need good practical
examples that justify your point of view. Please don't waste of your/my
time,
just give some good examples in next reply. Thanks.

--
Yasuo Ohgaki
yohgaki@ohgaki.net
Yasuo Ohgaki
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 30, 2017 01:30AM
On Sun, Apr 30, 2017 at 8:14 AM, Yasuo Ohgaki <[email protected]> wrote:

> I don't need your view of HKDF RFC or usage, but I do need good practical
> examples that justify your point of view. Please don't waste of your/my
> time,
> just give some good examples in next reply. Thanks.
>

BTW, valid (yet not common/proper) example that I can think of is,

<?php
$strong_512bit_key = random_bytes(64);
$strong_256bit_key = hash_hkdf('sha3-512', $strong_512bit_key, 32);
?>

while it does not even require HKDF, though.

<?php
$strong_512bit_key = random_bytes(64);
$strong_256bit_key = hash('sha3-256', $strong_512bit_key);
?>

should be good enough.

Even with "Info", following HMAC is enough.

<?php
$strong_512bit_key = random_bytes(64);
$strong_256bit_key = hash_hmac('sha3-256', $strong_512bit_key, $some_info);
?>

--
Yasuo Ohgaki
yohgaki@ohgaki.net
[email protected]
Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
April 30, 2017 09:00AM
Am 30.04.2017 um 01:26 schrieb Yasuo Ohgaki:
> On Sun, Apr 30, 2017 at 8:14 AM, Yasuo Ohgaki <[email protected]> wrote:
>
>> I don't need your view of HKDF RFC or usage, but I do need good practical
>> examples that justify your point of view. Please don't waste of your/my
>> time,
>> just give some good examples in next reply. Thanks.
>>
>
> BTW, valid (yet not common/proper) example that I can think of is

...... PLEASE STOP riding that dead horse - it's even annoying for users
following the devel-list how you argue on that opic over months - nonody
shares your view, that's it - accept it

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Sorry, only registered users may post in this forum.

Click here to login