[PHP-DEV] Q: ZEND_HANDLE_STREAM and wincache extension on PHP 5.4

PHP Internals folks--

My name is Eric Stenson, and I'm a developer at Microsoft working on IIS.
I've been given the task of upgrading our php_wincache extension to work
on PHP5.4, and I've run into a problem.

The problem I'm running into is the php_cgi!main() on PHP5.4 has changed
behavior.  The php_cgi!main() function is seeing us return a ZEND_HANDLE_STREAM,
and it's assuming that the zend_file_handle.handle.stream.handle (void *)
is a (php_stream*), when in fact it's php_wincache's (fcache_handle *).
It then attempts to access the php_stream.ops (a v-table-ish struct), which
is not in the fcache_handle, and it jumps off into an invalid address and
promptly AV's.

I'm trying to understand why our php_wincache!fcache_useval() thinks it's
okay to set zend_file_handle.handle.stream.handle to (fcache_handle *).
I'm having trouble understanding what the responsibilities are of a PHP
extension that extends the file system as php_wincache does.  Could you
point me at any developer documents for PHP extension developers that explains
what php_wincache *should* be doing?

I'm very much a novice at PHP code, or writing a PHP extension, so I feel
like I'm flailing around without understanding what PHP extensions are
supposed to do.  Any pointers, guidance, and architecture advice would be
massively helpful!

Thank you!
          --E.


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

On Thu, 15 Mar 2012 19:56:11 +0100, Eric Stenson <[email protected]>
wrote:

>
> The problem I'm running into is the php_cgi!main() on PHP5.4 has changed
> behavior. The php_cgi!main() function is seeing us return a
> ZEND_HANDLE_STREAM,
> and it's assuming that the zend_file_handle.handle.stream.handle (void *)
> is a (php_stream*), when in fact it's php_wincache's (fcache_handle *).
> It then attempts to access the php_stream.ops (a v-table-ish struct),
> which is not in the fcache_handle, and it jumps off into an invalid
> address and
> promptly AV's.
>
> [...]

I think you should contact Dmitry, as he was the one that committed
r301058. That code looks very strange. It seems to want to break the
abstraction of zend_stream and assume that handle stores a php_stream (to
use e.g. zend_stream_getc instead of php_stream_getc).

But I don't think the assumption done to break the abstraction is correct.
For instance, see

http://lxr.php.net/opengrok/xref/PHP_TRUNK/ext/phar/phar.c#3364

which has:

file_handle->handle.stream.handle = phar;

and *phar is of type phar_archive_data, which is not a php_stream (or an
augmentation thereof).

--
Gustavo Lopes

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

> -----Original Message-----
> From: Gustavo Lopes [mailto:[email protected]]
[...]
> For instance, see
>
> http://lxr.php.net/opengrok/xref/PHP_TRUNK/ext/phar/phar.c#3364
>
> which has:
>
> file_handle->handle.stream.handle = phar;
>
> and *phar is of type phar_archive_data, which is not a php_stream (or an
> augmentation thereof).

Thank you, Gustavo! Is Dmitry on the internals mailing list?

I was talking with Pierre yesterday, and there was some question about whether
there was some distinction between streams that were created as a result of
zend_compile_file versus just those files that were called by calling
zend_stream_open_function.

Does the phar extension make any distinction between when streams are
opened? Or, is it by virtue of only handling files that only end with
extensions that aren't compiled that it's skipping the code in
php_cgi!main()?

Thx!

--E.

On Fri, Mar 16, 2012 at 4:52 PM, Eric Stenson <[email protected]> wrote:

> Does the phar extension make any distinction between when streams are
> opened?  Or, is it by virtue of only handling files that only end with
> extensions that aren't compiled that it's skipping the code in
> php_cgi!main()?

The problem here is only about cgi calling a script. What phar does
will happen during the script execution. So it is very different in
that matter.

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

On Fri, 16 Mar 2012 16:52:39 +0100, Eric Stenson <[email protected]>
wrote:

>
> Thank you, Gustavo! Is Dmitry on the internals mailing list?

It's best you send him an e-mail to dmitry@zend.com

> I was talking with Pierre yesterday, and there was some question about
> whether there was some distinction between streams that were created as
> a result of zend_compile_file versus just those files that were called
> by calling
> zend_stream_open_function.

What would this distinction be and what would be the implication?

Conceptually, comparing zend_compile_file and zend_stream_function is
already a little fuzzy because they don't do the same thing;
zend_compile_file has already been provided a zend_file_handle. What may
make them comparable is that the zend_file_handle may be just a file name
and so effectively the file will have to be opened by the
zend_compile_file implementation (see also zend_stream_fixup).

>
> Does the phar extension make any distinction between when streams are
> opened? Or, is it by virtue of only handling files that only end with
> extensions that aren't compiled that it's skipping the code in
> php_cgi!main()?

I think the reason phar causes no problem is because CGI will not be
opening phar:// files as a first script, which is all php_cgi!main() deals
with. So phar's zend_stream_function will in this case end up falling back
on the default implementation and cause no problem.

In any case, it's true that default implementation of
zend_stream_open_function (php_stream_open_for_zend_ex) does store a
php_stream* in handle. However, this clearly isn't the general case (even
it were, breaking the abstraction would already be dubious). This is
definitely a bug.

--
Gustavo Lopes

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

hi,

On Fri, Mar 16, 2012 at 7:49 PM, Gustavo Lopes <[email protected]> wrote:

> It's best you send him an e-mail to dmitry@zend.com

No, it is best to send a mail to internals and cc him.

>> I was talking with Pierre yesterday, and there was some question about
>> whether there was some distinction between streams that were created as a
>> result of zend_compile_file versus just those files that were called by
>> calling
>> zend_stream_open_function.
>
>
> What would this distinction be and what would be the implication?

See the wincache code please. This comment was only about the wincache
implementation. Also it is only about knowing whether we are in RINIT
or at runtime, as in, compile file being called from the SAPI or from
require/include&co.


> I think the reason phar causes no problem is because CGI will not be opening
> phar:// files as a first script, which is all php_cgi!main() deals with. So
> phar's zend_stream_function will in this case end up falling back on the
> default implementation and cause no problem.

Yes, see my reply earlier.

> In any case, it's true that default implementation of
> zend_stream_open_function (php_stream_open_for_zend_ex) does store a
> php_stream* in handle. However, this clearly isn't the general case (even it
> were, breaking the abstraction would already be dubious). This is definitely
> a bug.

It is not necessary a bug but a breakage in the ABI, and not
documented. I discussed it with Dmitry and it was not seen as a bug
but as an optimization. The problem in wincache is somehow special due
the file caching mechanism, which mix up both SAPI file open and
runtime file open, which ends in the current issue.

Cheers,
--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Hi Eric,

Sorry, I don't read @internals every day. Please CC me if you need quick
answer.

Really I don't see the reason why wincache stores the PHP file source.
Storing compiled opcodes must be enough. To avoid re-reading of already
cached file. opcode cache should override zend_stream_open() and return
stream of type ZEND_HANDLE_FILENAME.

Anyway, I don't see a way to fix PHP to support custom streams without
BC break, because zend_streams doesn't provide handlers for seeking.

The best solution would be handling shebang lines by parser/scanner. It
even was done few years ago, but than was reverted.

Thanks. Dmitry.


On 03/15/2012 10:56 PM, Eric Stenson wrote:
> PHP Internals folks--
>
> My name is Eric Stenson, and I'm a developer at Microsoft working on IIS.
> I've been given the task of upgrading our php_wincache extension to work
> on PHP5.4, and I've run into a problem.
>
> The problem I'm running into is the php_cgi!main() on PHP5.4 has changed
> behavior. The php_cgi!main() function is seeing us return a ZEND_HANDLE_STREAM,
> and it's assuming that the zend_file_handle.handle.stream.handle (void *)
> is a (php_stream*), when in fact it's php_wincache's (fcache_handle *).
> It then attempts to access the php_stream.ops (a v-table-ish struct), which
> is not in the fcache_handle, and it jumps off into an invalid address and
> promptly AV's.
>
> I'm trying to understand why our php_wincache!fcache_useval() thinks it's
> okay to set zend_file_handle.handle.stream.handle to (fcache_handle *).
> I'm having trouble understanding what the responsibilities are of a PHP
> extension that extends the file system as php_wincache does. Could you
> point me at any developer documents for PHP extension developers that explains
> what php_wincache *should* be doing?
>
> I'm very much a novice at PHP code, or writing a PHP extension, so I feel
> like I'm flailing around without understanding what PHP extensions are
> supposed to do. Any pointers, guidance, and architecture advice would be
> massively helpful!
>
> Thank you!
> --E.
>
>

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

On Wed, 21 Mar 2012 07:46:31 +0100, Dmitry Stogov <[email protected]> wrote:

> Anyway, I don't see a way to fix PHP to support custom streams without
> BC break, because zend_streams doesn't provide handlers for seeking.
>

Couldn't you at least check whether e.g. the closer function pointer is
php_zend_stream_closer and only then assume it wraps a php_stream? If it
was not, you'd have to abandon shebang detection, but I think that's a
minor problem.

--
Gustavo Lopes

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php