Dawn of a new Age in Perl: How Devel::Declare extends Perl's syntax

For quite some time now there have been appearances of a new type of declarative modules on CPAN. Some of these (but not all) are MooseX::Method::Signatures, giving you a method keyword with signatures and type validation; TryCatch which provides you with a modern and elegant way to catch exceptions; and MooseX::Declare, a whole extendable, declarative framework around the Moose ecosystem.

To many people who don’t have the time or the interest to live on the edge of advancements in the Perl community, this might seem like magic trickery, much like source filters. And because of this many people keep a safe distance from all these developments, like they rightfully do with source filters. You should never use something before you know how it works on a deep enough level for decision making. This is why I decided to write this post, and to show how all of this is made possible.

The heavy lifter with all these syntax extensions is Devel::Declare, which was thought up and implemented first and largely by the mind of Matt S Trout, core team member of Catalyst and inventor of DBIx::Class It allows Perl modules to install hooks for barewords that will be executed when perl encounters the token in the code. There are two things that are important to notice here:

  • The hooks are defined in Perl space. That means you use Perl to extend Perl. There is no visible XS involved in developing these extensions. I have no idea whatsoever about C, C++ or XS. Paint me happy if it’s Scheme, Perl or JavaScript, but otherwise I wouldn’t trust me further than I can throw myself. But even though I have no experience in any of these low-level languages, I am able to contribute regularly to MooseX::Declare, and build a (yet unreleased) extension of it called CatalystX::Declarative. This is a huge advantage since it means a lot more people can contribute, extend, document, or fix bugs. Including me, of course.

  • The hooks will only fire on barewords. This means that the handling code does not have to parse Perl to find the invocations of its syntax. The keywords can for example be safely used in strings. The parsing of the special syntax only starts at the place where the syntax is used. This is a huge advantage over source filters, who always have to deal with the file as a whole, and not with individual invocations. This small scope in which these extensions act also allows safe combination of multiple extensions at once. MooseX::Declare for instance reuses MooseX::Method::Signatures for methods and modifiers.

After being invoked, these extensions use Devel::Declare to walk the statement from the invocation, stripping and interpreting parts as they walk along. When all is done, they will inject a bit of generated Perl code where the invocation occured. Using methods as an example, this

  method foo ($x) {
      # method body
  }

will be roughly translated to this

  method {
      # autogenerated code (lexicals, validation, etc.)

      # method body
  };

With this approach, it never actually has to parse the body. How the trick with the injected semicolon after the block works took me a while to understand. It simply injects a call to B::Hooks::EndOfScope that will fire when the block was compiled. After that happens it picks up again and inserts the semicolon. Here comes some internal magic into the game, because the extension will use Devel::Declare to give perl a code reference in this place to use, and this registration handler will get the actual code body. This is not as complicated as it sounds, but it allows certain meta data to be accessible when the method is registered. This is especially useful with Moose’ usage of the MOP.

The rest is all history. The block is passed as a code reference as usual in Perl and installed under a proper name in the calling class.

Devel::Declare provides lots of utilities for parsing tokens and handling the injection of code, and it is still young and in development. While the above might seem amazing enough, this is only where it starts. Devel::Declare has the potential to hook into more than just constant keywords, and people are already thinking about ways to make the parsing of syntaxes declarative by itself. This might result in something similar to what Perl 6 provides with grammars and rules, except in Perl 5.

I hope this article will help some of you to overcome their fears from these new, shiny but all the while very useful extensions, or even draw the one or the other to start contributing to one of the existing modules or the whole movement itself.

No TrackBacks

TrackBack URL: http://www.catalyzed.org/mt/mt-tb.fcgi/29

1 Comment

| Leave a comment

Excellent post! very informative

Leave a comment

All comments are moderated. Spammers don't waste your time

Sponsored By


Ionzero: Rescue your dev project.
OpenID accepted here Learn more about OpenID

Following

Not following anyone

Note to spammers: all comments are moderated. Don't waste your time