Syntax error in injected code when extending MooseX::Declare

Question

This is a big one, so please bear with me. There's a pot of gold at the end.

For mostly experimental reasons, I'm trying to make a custom extension of MooseX::Declare that does some extra magic that is useful for a specific hobby project. For example, I want to make the class keyword inject a bit of extra stuff, like importing useful utilities from List::Util and the like, turning on various extra pragmas (besides strict and warnings) , automatically import my global Config object, and so on.

So I wrote the following test and set off to see if I could get it to work. Amazingly, I was able to get 99% of the way there, but now I've run into a problem that I can't figure out. My custom class keyword dies with a syntax error in the injected code.

#!/usr/bin/env perl

use MyApp::Setup;

class Foo { 
    use Test::More tests => 1;

    has beer => ( is => 'ro', default => 'delicious' );
    method something { 
        is $self->beer, 'delicious';
    }
}


Foo->new->something;

MyApp::Setup looks like the following. In the future it will do some more stuff, but right now it just calls import on my MX::D subclass:

package MyApp::Setup;

use strict;
use warnings;

use MyApp::MooseX::Declare;

sub import { 
    goto &MyApp::MooseX::Declare::import;
}

1;

And that class looks like this:

package MyApp::MooseX::Declare;

use Moose;

use MyApp::MooseX::Declare::Syntax::Keyword::Class;
use MyApp::MooseX::Declare::Syntax::Keyword::Role;
use MyApp::MooseX::Declare::Syntax::Keyword::Namespace;

extends 'MooseX::Declare';

sub import {
    my ($class, %args) = @_;

    my $caller = caller;

    for my $keyword ( __PACKAGE__->keywords ) {
        warn sprintf 'setting up keyword %s', $keyword->identifier;
        $keyword->setup_for($caller, %args, provided_by => __PACKAGE__ );
    }
}

sub keywords { 
    # override the 'class' keyword with our own
    return
      ( MyApp::MooseX::Declare::Syntax::Keyword::Class->new( identifier => 'class' ),
        MyApp::MooseX::Declare::Syntax::Keyword::Role->new( identifier => 'role' ),
        MyApp::MooseX::Declare::Syntax::Keyword::Namespace->new( identifier => 'namespace' ) );
}

1;

I set up the three keyword classes to just include an extra role that replaces MX::D::Syntax::NamespaceHandling.

package MyApp::MooseX::Declare::Syntax::Keyword::Class;

use Moose;

extends 'MooseX::Declare::Syntax::Keyword::Class';
with 'MyApp::MooseX::Declare::Syntax::NamespaceHandling';

1;

(The other two are identical.)

In the real MX::D, the NamespaceHandling stuff is composed into a separate role called MooseSetup, which is itself composed into the keyword class. Doing it all in one place seems to work; I don't know if the slight deviation in structure is the source of my problem, though. At one point I had my own version of MooseSetup, but that led to composition conflicts that I couldn't figure out.

Finally, the meat and potatoes is my version of NamespaceHandling, which overrides the parse method. The bulk of it is just copy-and-pasted from the original.

package MyApp::MooseX::Declare::Syntax::NamespaceHandling;

use Moose::Role;
use Carp 'croak';
use Moose::Util 'does_role';
use MooseX::Declare::Util 'outer_stack_peek';


with 'MooseX::Declare::Syntax::NamespaceHandling';

# this is where the meat is!

sub parse {
    my ($self, $ctx) = @_;

    # keyword comes first
    $ctx->skip_declarator;

    # read the name and unwrap the options
    $self->parse_specification($ctx);

    my $name = $ctx->namespace;

    my ($package, $anon);

    # we have a name in the declaration, which will be used as package name
    if (defined $name) {
        $package = $name;

        # there is an outer namespace stack item, meaning we namespace below
        # it, if the name starts with ::
        if (my $outer = outer_stack_peek $ctx->caller_file) {
            $package = $outer . $package
                if $name =~ /^::/;
        }
    }

    # no name, no options, no block. Probably { class => 'foo' }
    elsif (not(keys %{ $ctx->options }) and $ctx->peek_next_char ne '{') {
        return;
    }

    # we have options and/or a block, but not name
    else {
        $anon = $self->make_anon_metaclass
            or croak sprintf 'Unable to create an anonymized %s namespace', $self->identifier;
        $package = $anon->name;
    }

    warn "setting up package [$package]";

    # namespace and mx:d initialisations
    $ctx->add_preamble_code_parts(
        "package ${package}",
        sprintf(
            "use %s %s => '%s', file => __FILE__, stack => [ %s ]",
            $ctx->provided_by,
            outer_package => $package,
            $self->generate_inline_stack($ctx),
       ),
    );

    # handle imports and setup here (TODO)


    # allow consumer to provide specialisations
    $self->add_namespace_customizations($ctx, $package);

    # make options a separate step
    $self->add_optional_customizations($ctx, $package);

    # finish off preamble with a namespace cleanup
    # we'll use namespace::sweep instead

    #$ctx->add_preamble_code_parts(
    #    $ctx->options->{is}->{dirty}
    #        ? 'use namespace::clean -except => [qw( meta )]'
    #        : 'use namespace::autoclean'
    #);

    # clean up our stack afterwards, if there was a name
    $ctx->add_cleanup_code_parts(
        ['BEGIN',
            'MooseX::Declare::Util::outer_stack_pop __FILE__',
        ],
    );

    # actual code injection
    $ctx->inject_code_parts(
        missing_block_handler => sub { $self->handle_missing_block(@_) },
    );

    # a last chance to change things
    $self->handle_post_parsing($ctx, $package, defined($name) ? $name : $anon);
}


1;

When I run the test, everything seems to go great -- I get the warning messages indicating that the right methods are being called and that the package "Foo" is being set up. Then it dies with:

syntax error at t/default.t line 5, near "{package Foo"

So it seems like something is injecting some code right before or after the package declaration that is causing a syntax error, but I can't figure out what. I've tried randomly playing with the various items in the parse sub (I don't actually know what they all do at this point) but I can't seem to eliminate or even change the error. And of course there's no way (that I know of) to actually inspect the generated code, which might yield a clue.

Thanks for your help.

Some updates: After looking around inside MooseX::Declare::Context, I added some print statements to see exactly what was being injected via the call to inject_code_parts. This is the actual code that gets generated (tidied):

 package Foo; 

 use MyApp::MooseX::Declare outer_package => 'Foo', file => __FILE__, stack => [ 
     MooseX::Declare::StackItem->new(q(identifier), q(class), q(handler), 
     q(MyApp::MooseX::Declare::Syntax::Keyword::Class), q(is_dirty), q(0), 
     q(is_parameterized), q(0), q(namespace), q(Foo)) ];; 

 BEGIN { Devel::Declare::Context::Simple->inject_scope('BEGIN { 
   MooseX::Declare::Util::outer_stack_pop __FILE__ }') }; ;

I can't say I know what all that does (especially the outer_stack_pop thing), but it all looks syntactically OK to me. I still think something is injecting code before all this that causes the syntax error.

Answer 1

Well, that was a hell of a debugging session, but I finally traced the problem and got it figured out. After cracking open both MooseX::Declare::Context and Devel::Declare::Context::Simple (to which the former delegates) I was able to trace the flow and through copious dumping to STDOUT I realized that some of the extra handlers from MooseSetup.pm, which I thought I had correctly composed into my keyword classes, were not actually there. The resulting code being injected thus did not have the proper shadow/cleanup stuff attached.

Anyway, I now have what appears to be a fully working customized MooseX::Declare! I'm really psyched about this -- it means I can type

use MyApp::Setup; 

class MyApp::Foo { ... }

and that one class statement sets up a whole mess of application-specific boilerplate. Rad.