Bundle npm module 'cheerio' in K6 test

I am trying to create some tests using K6 framework from LoadImpact, but I am struggelig with including external NPM module following the instructions on their documentation site.

On loadImpacts documentations site they include a detailed example on just what I am after, modules that enable me to parse xml from a soap service response. But, I am unable to get this working! Now, I am a total javascript newbie, but I have been coding for many years and would really like to solve this. The can be found here: https://docs.k6.io/docs/modules#section-npm-modules can anyone get this working? I need to run this on servers isolated from the Internet, so I am totaly dependent on creating the packages and transfer the required files.

According to the documentation a package is created like this

-- bundle `cheerio` npm module
git clone git@github.com:cheeriojs/cheerio.git
npm install browserify index.js -s cheerio > cheerio.js

My first question: In the folder I am residing when running this command a 'cheerio.js' file is created along with a a 'cheerio' folder and a 'node_modules' folder. the cheerio.js in my "root" directory only contains the following:

+ cheerio@0.22.0
+ index.js@0.0.3
+ browserify@16.2.3
updated 3 packages and audited 2829 packages in 2.221s
found 0 vulnerabilities

Back to LoadImpacts example on how to reference this package in a k6 javascript:

import cheerio from "./vendor/cheerio.js";
export default function() 
{
  const res = http.get("https://loadimpact.com/");
  const $ = cheerio.load(res.body);

What file is this, and where in the structure generated by browserify can I find it? I have tried to change this to point to 'index.js' in the 'cheerio' folder or cheerio.js found in 'cheerio/lib'. I will then receive a complaint about the first line in cheerio.js which defines a "parse" variable it cannot find: var parse = require("./parse'), if I change this to var parse = require("./parse.js') it goes on to complain about missing 'htmlparser2' which I can also find in this structure, but it seems like the entire dependency structure is not working.

Can anybody give me some guidance on how to create a browserify package with dependencies for cheerio and how/what I need to copy to my k6 project to make this work like on the loadImpact site.

Solution

The k6 docs for this definitely need some clarification, which I'll later do. The vendor folder currently mentioned there isn't something special, the docs are just missing a step to copy the cheerio.js and xml2js.js files that were generated by browserify to a new vendor folder in your k6 project.

For now, I'll try to offer a simplified explanation on how to achieve the same thing in a simpler way:

Create a new empty folder and go to it in a terminal
Run npm install browserify cheerio there (ignore the npm warnings about missing package.json or description)
Run ./node_modules/.bin/browserify ./node_modules/cheerio/ -s cheerio > cheerio.js in that folder
The resulting cheerio.js file in the folder root should be the file you import from the k6 script:

import http from "k6/http";
import cheerio from "./cheerio.js";

export default function () {
    const res = http.get("https://loadimpact.com/");

    const $ = cheerio.load(res.body);

    console.log($('head title').text())
}

That should be it for a single npm library.

And if you need to use multiple npm packages, it might be better to invest some time into bundling them in a single browserified .js file. For example, if you need both the cheerio and the xml2js libraries mentioned in the k6 docs, you can do something like this:

Create a new empty folder

Add something like the following package.json file in it:

{
  "name": "k6-npm-libs-demo",
  "version": "0.0.1",
  "description": "just a simple demo of how to use multiple npm libs in k6",
  "main": "npm-main.js",
  "dependencies": {},
  "devDependencies": {
    "browserify": "*",
    "cheerio": "*",
    "xml2js": "*"
  },
  "scripts": {
    "install": "./node_modules/.bin/browserify npm-main.js -s npmlibs  > vendored-libs.js"
  },
  "author": "",
  "license": "ISC"
}

Of course, if you need different libraries than cheerio and xml2js, you need to adjust the devDependencies options.

Add an npm-main.js file like this (again, adjusting for the libraries you want):
```
exports.xml2js = require('xml2js');
exports.cheerio = require('cheerio');
```

Open that folder in a terminal and run npm install. That should result in the creation of a vendored-libs.js file in the root of the folder, which you can use in k6 like this:

import http from "k6/http";
import { cheerio, xml2js } from "./vendored-libs.js";

export default function () {
    const res = http.get("https://loadimpact.com/");

    const $ = cheerio.load(res.body);
    console.log($('head title').text())

    var xmlString = '<?xml version="1.0" ?>' +
        '<items xmlns="http://foo.com">' +
        ' <item>Foo</item>' +
        ' <item color="green">Bar</item>' +
        '</items>'

    xml2js.parseString(xmlString, function (err, result) {
        console.log(JSON.stringify(result));
    });
}