For quite some time now there have been appearances of a new type of
declarative modules on CPAN. Some of these (but not all) are
MooseX::Method::Signatures, giving you a method
keyword with signatures and type validation; TryCatch
which provides you with a modern and elegant way to catch exceptions; and
MooseX::Declare, a whole extendable, declarative framework around the
Moose ecosystem.
To many people who don’t have the time or the interest to live on the edge of advancements in the Perl community, this might seem like magic trickery, much like source filters. And because of this many people keep a safe distance from all these developments, like they rightfully do with source filters. You should never use something before you know how it works on a deep enough level for decision making. This is why I decided to write this post, and to show how all of this is made possible.
The heavy lifter with all these syntax extensions is Devel::Declare, which was thought up and implemented first and largely by the mind of Matt S Trout, core team member of Catalyst and inventor of DBIx::Class It allows Perl modules to install hooks for barewords that will be executed when perl encounters the token in the code. There are two things that are important to notice here:
The hooks are defined in Perl space. That means you use Perl to extend Perl. There is no visible XS involved in developing these extensions. I have no idea whatsoever about C, C++ or XS. Paint me happy if it’s Scheme, Perl or JavaScript, but otherwise I wouldn’t trust me further than I can throw myself. But even though I have no experience in any of these low-level languages, I am able to contribute regularly to
MooseX::Declare, and build a (yet unreleased) extension of it called CatalystX::Declarative. This is a huge advantage since it means a lot more people can contribute, extend, document, or fix bugs. Including me, of course.The hooks will only fire on barewords. This means that the handling code does not have to parse Perl to find the invocations of its syntax. The keywords can for example be safely used in strings. The parsing of the special syntax only starts at the place where the syntax is used. This is a huge advantage over source filters, who always have to deal with the file as a whole, and not with individual invocations. This small scope in which these extensions act also allows safe combination of multiple extensions at once.
MooseX::Declarefor instance reusesMooseX::Method::Signaturesfor methods and modifiers.
After being invoked, these extensions use Devel::Declare to walk the
statement from the invocation, stripping and interpreting parts as they walk along.
When all is done, they will inject a bit of generated Perl code where the invocation occured.
Using methods as an example, this
method foo ($x) {
# method body
}
will be roughly translated to this
method {
# autogenerated code (lexicals, validation, etc.)
# method body
};
With this approach, it never actually has to parse the body. How the trick with
the injected semicolon after the block works took me a while to understand. It
simply injects a call to B::Hooks::EndOfScope that will fire when
the block was compiled. After that happens it picks up again and inserts the
semicolon. Here comes some internal magic into the game, because the
extension will use Devel::Declare to give perl a code reference
in this place to use, and this registration handler will get the actual code
body. This is not as complicated as it sounds, but it allows certain meta data
to be accessible when the method is registered. This is especially useful
with Moose’ usage of the MOP.
The rest is all history. The block is passed as a code reference as usual in Perl and installed under a proper name in the calling class.
Devel::Declare provides lots of utilities for parsing tokens and
handling the injection of code, and it is still young and in development. While
the above might seem amazing enough, this is only where it starts.
Devel::Declare has the potential to hook into more than just
constant keywords, and people are already thinking about ways to make the parsing
of syntaxes declarative by itself. This might result in something similar to what
Perl 6 provides with grammars and rules, except in Perl 5.
I hope this article will help some of you to overcome their fears from these new, shiny but all the while very useful extensions, or even draw the one or the other to start contributing to one of the existing modules or the whole movement itself.
Excellent post! very informative