Welcome! Log In Create A New Profile

Advanced

[PHP-DEV] Request-global hashtable initialization

Posted by Heinz Wiesinger 
Heinz Wiesinger
[PHP-DEV] Request-global hashtable initialization
April 12, 2018 10:00PM
Hi everyone!

I ventured into writing my first PHP extension and did pretty well so far
following the existing documentation and example code inside the php source.

However, I hit an odd bug in my code that I can't quite explain. I'm obviously
missing something and I hope someone here can point me in the right direction
:)

I'd like to set up a request global hashtable. For now I'm running
ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then have
a user function that's supposed to add entries to that hashtable and another
user function that reads entries from the hashtable.

I tested with code passed directly on the command line (using php -r) and that
works just like I would expect it too. The value gets written to the hashtable
and I can read it afterwards from the other function. I also verified it's the
same value that I first stored, so from what I can tell the basic
functionality is good.

When I take the same testcode and write it into a file, and then run php on
that file, execution aborts trying to allocate almost 2GB of memory when
trying to store the value in the hashtable (using zend_hash_update()). I can
see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume the
hashtable is setup just fine. Yet something is different.

What's the difference between " php -r '' " and "php test.php" in that regard?

I uploaded my current state of the extension to github in case someone would
like to take a closer look at the code: https://github.com/pprkut/autoload-psr

Thanks in advance!

Grs,
Heinz
Nikita Popov
Re: [PHP-DEV] Request-global hashtable initialization
April 12, 2018 10:40PM
On Thu, Apr 12, 2018 at 9:53 PM, Heinz Wiesinger <[email protected]>
wrote:

> Hi everyone!
>
> I ventured into writing my first PHP extension and did pretty well so far
> following the existing documentation and example code inside the php
> source.
>
> However, I hit an odd bug in my code that I can't quite explain. I'm
> obviously
> missing something and I hope someone here can point me in the right
> direction
> :)
>
> I'd like to set up a request global hashtable. For now I'm running
> ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then
> have
> a user function that's supposed to add entries to that hashtable and
> another
> user function that reads entries from the hashtable.
>
> I tested with code passed directly on the command line (using php -r) and
> that
> works just like I would expect it too. The value gets written to the
> hashtable
> and I can read it afterwards from the other function. I also verified it's
> the
> same value that I first stored, so from what I can tell the basic
> functionality is good.
>
> When I take the same testcode and write it into a file, and then run php on
> that file, execution aborts trying to allocate almost 2GB of memory when
> trying to store the value in the hashtable (using zend_hash_update()). I
> can
> see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume
> the
> hashtable is setup just fine. Yet something is different.
>
> What's the difference between " php -r '' " and "php test.php" in that
> regard?
>
> I uploaded my current state of the extension to github in case someone
> would
> like to take a closer look at the code: https://github.com/pprkut/
> autoload-psr
>
> Thanks in advance!
>
> Grs,
> Heinz
>

https://github.com/pprkut/autoload-psr/blob/27cfc53c864116cf0b65b8f7e69464e15b3fdef5/autoload_psr.c#L175
should be using ZVAL_STR_COPY. The hashtable API does not automatically
incref stored values.

I'd suggest running under USE_ZEND_ALLOC=0 valgrind to check for further
memory issues. Differences in the mode of execution are likely just
incidental.

Nikita
Heinz Wiesinger
Re: [PHP-DEV] Request-global hashtable initialization
April 13, 2018 08:20AM
On Thursday, 12 April 2018 22:34:52 CEST Nikita Popov wrote:
> On Thu, Apr 12, 2018 at 9:53 PM, Heinz Wiesinger <[email protected]>
>
> wrote:
> > Hi everyone!
> >
> > I ventured into writing my first PHP extension and did pretty well so far
> > following the existing documentation and example code inside the php
> > source.
> >
> > However, I hit an odd bug in my code that I can't quite explain. I'm
> > obviously
> > missing something and I hope someone here can point me in the right
> > direction
> >
> > :)
> >
> > I'd like to set up a request global hashtable. For now I'm running
> > ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then
> > have
> > a user function that's supposed to add entries to that hashtable and
> > another
> > user function that reads entries from the hashtable.
> >
> > I tested with code passed directly on the command line (using php -r) and
> > that
> > works just like I would expect it too. The value gets written to the
> > hashtable
> > and I can read it afterwards from the other function. I also verified it's
> > the
> > same value that I first stored, so from what I can tell the basic
> > functionality is good.
> >
> > When I take the same testcode and write it into a file, and then run php
> > on
> > that file, execution aborts trying to allocate almost 2GB of memory when
> > trying to store the value in the hashtable (using zend_hash_update()). I
> > can
> > see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume
> > the
> > hashtable is setup just fine. Yet something is different.
> >
> > What's the difference between " php -r '' " and "php test.php" in that
> > regard?
> >
> > I uploaded my current state of the extension to github in case someone
> > would
> > like to take a closer look at the code: https://github.com/pprkut/
> > autoload-psr
> >
> > Thanks in advance!
> >
> > Grs,
> > Heinz
>
> https://github.com/pprkut/autoload-psr/blob/27cfc53c864116cf0b65b8f7e69464e1
> 5b3fdef5/autoload_psr.c#L175 should be using ZVAL_STR_COPY. The hashtable
> API does not automatically incref stored values.
>
> I'd suggest running under USE_ZEND_ALLOC=0 valgrind to check for further
> memory issues. Differences in the mode of execution are likely just
> incidental.

Thanks for the pointers! I tried running it under valgrind, but couldn't find
anything. So I tried without valgrind, but with USE_ZEND_ALLOC=0 and found
that I can't reproduce the problem like that. Looks to me that something I do
doesn't play well with the zend allocator.

Incidentally valgrind also doesn't tell me about the issue with ZVAL_STR that
you pointed out. Maybe I'm using it wrong? I tried

USE_ZEND_ALLOC=0 valgrind --tool=memcheck --leak-check=full php -
dextension=modules/autoload_psr.so test2.php

Grs,
Heinz
Heinz Wiesinger
Re: [PHP-DEV] Request-global hashtable initialization
April 14, 2018 11:40AM
On Friday, 13 April 2018 08:16:24 CEST Heinz Wiesinger wrote:
> On Thursday, 12 April 2018 22:34:52 CEST Nikita Popov wrote:
> > On Thu, Apr 12, 2018 at 9:53 PM, Heinz Wiesinger <[email protected]>
> >
> > wrote:
> > > Hi everyone!
> > >
> > > I ventured into writing my first PHP extension and did pretty well so
> > > far
> > > following the existing documentation and example code inside the php
> > > source.
> > >
> > > However, I hit an odd bug in my code that I can't quite explain. I'm
> > > obviously
> > > missing something and I hope someone here can point me in the right
> > > direction
> > >
> > > :)
> > >
> > > I'd like to set up a request global hashtable. For now I'm running
> > > ALLOC_HASHTABLE() and zend_hash_init() inside PHP_GINIT_FUNCTION. I then
> > > have
> > > a user function that's supposed to add entries to that hashtable and
> > > another
> > > user function that reads entries from the hashtable.
> > >
> > > I tested with code passed directly on the command line (using php -r)
> > > and
> > > that
> > > works just like I would expect it too. The value gets written to the
> > > hashtable
> > > and I can read it afterwards from the other function. I also verified
> > > it's
> > > the
> > > same value that I first stored, so from what I can tell the basic
> > > functionality is good.
> > >
> > > When I take the same testcode and write it into a file, and then run php
> > > on
> > > that file, execution aborts trying to allocate almost 2GB of memory when
> > > trying to store the value in the hashtable (using zend_hash_update()). I
> > > can
> > > see PHP_GINIT_FUNCTION being executed in that case as well so I'd assume
> > > the
> > > hashtable is setup just fine. Yet something is different.
> > >
> > > What's the difference between " php -r '' " and "php test.php" in that
> > > regard?
> > >
> > > I uploaded my current state of the extension to github in case someone
> > > would
> > > like to take a closer look at the code: https://github.com/pprkut/
> > > autoload-psr
> > >
> > > Thanks in advance!
> > >
> > > Grs,
> > > Heinz
> >
> > https://github.com/pprkut/autoload-psr/blob/27cfc53c864116cf0b65b8f7e69464
> > e1 5b3fdef5/autoload_psr.c#L175 should be using ZVAL_STR_COPY. The
> > hashtable API does not automatically incref stored values.
> >
> > I'd suggest running under USE_ZEND_ALLOC=0 valgrind to check for further
> > memory issues. Differences in the mode of execution are likely just
> > incidental.
>
> Thanks for the pointers! I tried running it under valgrind, but couldn't
> find anything. So I tried without valgrind, but with USE_ZEND_ALLOC=0 and
> found that I can't reproduce the problem like that. Looks to me that
> something I do doesn't play well with the zend allocator.
>
> Incidentally valgrind also doesn't tell me about the issue with ZVAL_STR
> that you pointed out. Maybe I'm using it wrong? I tried
>
> USE_ZEND_ALLOC=0 valgrind --tool=memcheck --leak-check=full php -
> dextension=modules/autoload_psr.so test2.php

I moved the hashtable initialization out of PHP_GINIT_FUNCTION and do it on-
demand now in the respective user functions. That seems to have done the
trick, as segfaults and oom messages are gone now.

Grs,
Heinz
Sorry, only registered users may post in this forum.

Click here to login