Search code examples
javascriptnode.jsrequirejskarma-runnerpegjs

Configure Karma to load pegjs with requirejs

Trying to test a project using PegJS and requirejs. I have a couple of source files, implemented as AMD module (define) which loads through the require API. Below the directory structure: 

js/
   somefile.js
   main.js
   parser.js
test/
   parser.spec.js

I've written a parser.js module to load a PegJS grammar file and use PegJS to create a peg parser:

define(function() {
  'use strict';

  var PEG = require('pegjs');
  var grammarFile = 'grammar.peg'

return {
  parse: function(fs, content, debug) {
    var grammar = fs.readFileSync(grammarFile, 'utf8').toString();
    // Build parser from grammar
    var parser = PEG.buildParser(grammar, { trace: debug });
    [...]

This works fine with a main.js executed on the command line with node. Now I want to test my project using karma, jasmine and PhantomJS. I have a karma.conf.js like this:

frameworks: ['jasmine', 'requirejs'],
files: [
  { pattern: './test/**/*.spec.js', included: false },
  { pattern: './js/**/*.js', included: false},
  './test/test-main.js',
],

Also have a require bootstrap file called test-main.js which is configured this way:

'use strict';

var allTestFiles = [];
var TEST_REGEXP = /(spec|test)\.js$/i;

// Get a list of all the test files to include
Object.keys(window.__karma__.files).forEach(function(file) {
  console.log(file);
  if (TEST_REGEXP.test(file)) {
    // Normalize paths to RequireJS module names.
    // If you require sub-dependencies of test files to be loaded as-is (requiring file extension)
    // then do not normalize the paths
    var normalizedTestModule = file.replace(/^\/base\/|\.js$/g, '');
    allTestFiles.push(file);
  }
});

require.config({
  // Karma serves files under /base, which is the basePath from your config file
  baseUrl: '/base/js',
  // dynamically load all test files
  deps: allTestFiles,
  // we have to kickoff jasmine, as it is asynchronous
  callback: window.__karma__.start
});

Now, when I launch my test (grunt karma), I got this error:

PhantomJS 1.9.8 (Linux 0.0.0) ERROR: Error{message: 'Module name "pegjs" has not been loaded yet for context: _. Use require([])

So I try to include pegjs in the files loaded by Karma this way karma.conf.js:

files: [
  { pattern: 'node_modules/pegjs/lib/**/*.js', included: true  },
  { pattern: './test/**/*.spec.js', included: false },
  { pattern: './js/**/*.js', included: false},
  './test/test-main.js'
],

When I do this, I still get an error:

Error: Module name "utils/arrays" has not been loaded yet for context: _. Use require([])

Looking inside pegjs module, there is indeed an arrays.js file:

compiler/
compiler.js
grammar-error.js
parser.js
peg.js
utils/
  arrays.js
  classes.js
  objects.js

So trying to include arrays too:

files: [
  { pattern: 'node_modules/pegjs/lib/utils/arrays.js', included: true },
  { pattern: 'node_modules/pegjs/lib/**/*.js', included: true  },
  { pattern: './test/**/*.spec.js', included: false },
  { pattern: './js/**/*.js', included: false},
  './test/test-main.js'
],

I get:

ReferenceError: Can't find variable: module
at /blabla/node_modules/pegjs/lib/utils/arrays.js:108

Because of:

108 module.exports = arrays;

So, intead of loading the npm module, I tried to load the bower module this way:

files: [
  { pattern: 'bower_components/pegjs/peg-0.9.0.js', included: true },
  { pattern: './test/**/*.spec.js', included: false },
  { pattern: './js/**/*.js', included: false},
  './test/test-main.js'
],

And here you go again:

PhantomJS 1.9.8 (Linux 0.0.0) ERROR: Error{message: 'Module name "pegjs" has not been loaded yet for context: _. Use require([])

Also tried not to include pegjs in the karma generated web page:

files: [
  { pattern: 'bower_components/pegjs/peg-0.9.0.js', included: false },
  { pattern: './test/**/*.spec.js', included: false },
  { pattern: './js/**/*.js', included: false},
  './test/test-main.js'
],

But it fails with:

PhantomJS 1.9.8 (Linux 0.0.0) ERROR: 'There is no timestamp for /base/bower_components/pegjs/peg-0.9.0!'

Tried to put the bower_component folder inside the js folder but no luck.

So I don't know were to go from here... Couldn't find anything relevant on Google or here. It seems to be a specific problem to requirejs/pegjs with karma... Any idea is welcome.

UPDATE following dan's answer:

So I switch from a synchronous require to a asynchronous require in parser.js:

define(['../bower_components/pegjs/peg-0.9.0'], function(PEG) {
  'use strict';

  var grammarFile = 'grammar.peg'

return {
  parse: function(fs, content, debug) {
    var grammar = fs.readFileSync(grammarFile, 'utf8').toString();
    // Build parser from grammar
    var parser = PEG.buildParser(grammar, { trace: debug });
    [...]

Tried to include the pegjs bower component in karma.conf.js:

{ pattern: 'bower_components/pegjs/peg-0.9.0.js', included: false },

or not include it:

{ pattern: 'bower_components/pegjs/peg-0.9.0.js', included: true },

But always get the same error:

Error: Script error for "/blabla/bower_components/pegjs/peg-0.9.0", needed by: /blabla/js/parser.js
http://requirejs.org/docs/errors.html#scripterror
at /blabla/node_modules/requirejs/require.js:140

Yes the file exists:

$ file /home/aa024149/share/logofrjs/bower_components/pegjs/peg-0.9.0.js 
/blabla/bower_components/pegjs/peg-0.9.0.js: ASCII text, with very long lines

UPDATE2: Finally understood and found an acceptable solution.

Solution

  • So with the help of the various answers and comments from dan and pieceOpiland I finally came to a way to do what I want.

    So first, pegjs, like many javascript libraries comes in two formats: npm modules and bower modules.

    Npm modules are used for script made for node and called from the command line. Bower modules are used for script loaded in a browser.

    First misunderstanding from my part was that the 'require' would work in node and in the browser indistinctly. This is wrong. It seems the only way to require a module so that it works in the browser is through a asynchronous call like:

    require(['module'], function(module) {
  ...
});

    Another misunderstanding was that I could load npm modules in the browser. That some kind of magic would operate for the various npm files to be loaded with my page. That might be possible but only using some special tool like browserify. Without special transformation, only the bower version can be loaded in the browser. Additionally, pegjs bower module is made in a way so that global variables are defined like so:

    var PEG = {
 ...
}

module.exports = PEG;

    Basically, the bower module plugs a global variable (actually several global variables) to the top level scope.

    So instead of having my client code (the one running in the browser AND in node) loading the module, I actually load the module in either:

    1. main.js through a synchronous require to the npm module like so: var PEG = require('pegjs');
    2. main-browser.js through the global variable which becomes available when you load the bower pegjs script (through a <script> tag)

    Both those 'mains' are then injecting the PEG variable to my parser function.

    For karma to work, I just need to include the pegjs bower module in the generated page (karma.conf.js extract):

    files: [
 { pattern: 'bower_components/pegjs/peg-0.9.0.js', included: true },
 { pattern: './test/**/*.spec.js', included: false },
 { pattern: './js/**/*.js', included: false},
 './test/test-main.js',
],