Welcome! Log In Create A New Profile

Advanced

[PHP-DEV] generating code from AST

Posted by Adam Baratz 
Adam Baratz
[PHP-DEV] generating code from AST
March 06, 2017 10:40PM
I'm exploring how to automate some basic kinds of refactor operations. One
approach I'm considering:

- Generate an AST
- Rearrange it as needed
- Turn it back into userland code

Is this something anyone's explored?

Thanks,
Adam
Sara Golemon
Re: [PHP-DEV] generating code from AST
March 06, 2017 10:50PM
On Mon, Mar 6, 2017 at 3:37 PM, Adam Baratz <[email protected]> wrote:
> I'm exploring how to automate some basic kinds of refactor operations. One
> approach I'm considering:
>
> - Generate an AST
> - Rearrange it as needed
> - Turn it back into userland code
>
> Is this something anyone's explored?
>
The engine's internals actually have an ast export method which does
this alread (it's used by the assert message generator). astkit
makes use of this in an extension to do much of what you describe:
https://github.com/sgolemon/astkit

-Sara

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Sara Golemon
Re: [PHP-DEV] generating code from AST
March 06, 2017 10:50PM
On Mon, Mar 6, 2017 at 3:40 PM, Sara Golemon <[email protected]> wrote:
> On Mon, Mar 6, 2017 at 3:37 PM, Adam Baratz <[email protected]> wrote:
>> I'm exploring how to automate some basic kinds of refactor operations. One
>> approach I'm considering:
>>
>> - Generate an AST
>> - Rearrange it as needed
>> - Turn it back into userland code
>>
>> Is this something anyone's explored?
>>
> The engine's internals actually have an ast export method which does
> this alread (it's used by the assert message generator). astkit
> makes use of this in an extension to do much of what you describe:
> https://github.com/sgolemon/astkit
>
Hrmmm, I seem to have not updated the README.md, but it is in there:
https://github.com/sgolemon/astkit/blob/master/astkit-node.c#L204-L214

-Sara

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Sebastian Bergmann
Re: [PHP-DEV] generating code from AST
March 07, 2017 07:00AM
Am 06.03.2017 um 22:40 schrieb Sara Golemon:
> https://github.com/sgolemon/astkit

There's also https://github.com/nikic/php-ast.

But wouldn't it be great if PHP 7.2 shipped with a built-in,
enabled-by-default extension for AST operations?

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
David Walker
Re: [PHP-DEV] generating code from AST
March 07, 2017 07:10AM
On Mon, Mar 6, 2017 at 22:55 Sebastian Bergmann <[email protected]> wrote:

> Am 06.03.2017 um 22:40 schrieb Sara Golemon:
> > https://github.com/sgolemon/astkit
>
> There's also https://github.com/nikic/php-ast.
>
> But wouldn't it be great if PHP 7.2 shipped with a built-in,
> enabled-by-default extension for AST operations?


I'd be happy if this were to go forward. When/if there are changes to AST,
having a tool like these in core would require them to be updated at the
same time.

Would this concept also want to extend to a VLD-esque extension as well for
the same reasons? I'd be receptive for abilities to expose the core
internal ast/ops to userland more easily as a core extension

However I'm sure there are valid reasons why VLD (or similar) have not been
included; and would wonder if those would also end up applying to the AST

--
Dave

>
Sebastian Bergmann
Re: [PHP-DEV] generating code from AST
March 07, 2017 07:10AM
Am 07.03.2017 um 07:01 schrieb David Walker:
> When/if there are changes to AST, having a tool like these in core would require them to be updated at the same time.

Exactly.

> Would this concept also want to extend to a VLD-esque extension as well for
> the same reasons? I'd be receptive for abilities to expose the core
> internal ast/ops to userland more easily as a core extension

Having a bytecode disassembler in core would also be nice, yes.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Derick Rethans
Re: [PHP-DEV] generating code from AST
March 07, 2017 11:40AM
On Tue, 7 Mar 2017, Sebastian Bergmann wrote:

> Am 07.03.2017 um 07:01 schrieb David Walker:
>
> > When/if there are changes to AST, having a tool like these in core
> > would require them to be updated at the same time.
>
> Exactly.
>
> > Would this concept also want to extend to a VLD-esque extension as well for
> > the same reasons? I'd be receptive for abilities to expose the core
> > internal ast/ops to userland more easily as a core extension
>
> Having a bytecode disassembler in core would also be nice, yes.

Because installing an extension is too hard? It sounds like reinventing
the wheel.

cheers,
Derick

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Rowan Collins
Re: [PHP-DEV] generating code from AST
March 07, 2017 09:00PM
On 07/03/2017 10:33, Derick Rethans wrote:
>> Having a bytecode disassembler in core would also be nice, yes.
> Because installing an extension is too hard? It sounds like reinventing
> the wheel.

Not necessarily reinventing it, just pre-attaching it to the carriage ;)
By which I mean, there's no reason "bundling a VLD-esque extension"
couldn't mean "bundling VLD".

But as Dave said:

> However I'm sure there are valid reasons why VLD (or similar) have not been
> included; and would wonder if those would also end up applying to the AST


Basically, we're back to the age-old discussion of exactly what are the
pros and cons of "bundling" an extension, given the way PHP is packaged
by vendors in practice.

Regards,

--
Rowan Collins
[IMSoP]


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Sebastian Bergmann
Re: [PHP-DEV] generating code from AST
March 08, 2017 10:00AM
Am 07.03.2017 um 11:33 schrieb Derick Rethans:
> Because installing an extension is too hard?

No. To ensure that userland functionality that is based on compiler
internals (token stream, abstract syntax tree, bytecode) does not fall out
of sync with the compiler.

We already have an extension that is built-in and enabled-by-default for
the token stream. Why not have one for the abstract syntax tree and
bytecode representations?

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Nikita Nefedov
Re: [PHP-DEV] generating code from AST
March 08, 2017 01:10PM
> On 8 Mar 2017, at 08:49, Sebastian Bergmann <[email protected]> wrote:
>
> Am 07.03.2017 um 11:33 schrieb Derick Rethans:
>> Because installing an extension is too hard?
>
> No. To ensure that userland functionality that is based on compiler
> internals (token stream, abstract syntax tree, bytecode) does not fall out
> of sync with the compiler.
>
> We already have an extension that is built-in and enabled-by-default for
> the token stream. Why not have one for the abstract syntax tree and
> bytecode representations?

To be honest, that’s exactly what we might want to avoid. Last time someone wanted to do actual improvements to our lexing they were stopped because changing tokens would mean breaking code that relies on `token_get_all()` which is a core extension user land function.

It might be though that it’s only the problem with lexing and it should not be a problem for AST because it is, ahem, abstract.
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Sorry, only registered users may post in this forum.

Click here to login