Search code examples
jenkinsjasmineprotractorgulp-protractorjasmine-reporters

protractor-jasmine2-html-reporter doesn't consolidate results for all test when tests are shared using 'shardTestFiles': true in conf file


Recently we have configured our e2e-tests to be on Jenkins & soon we realized that we have to use shared test files: true options as complete suite run is taking very long time for us peeking 9-10hrs on daily basis. but when we configured below two options in conf file. tests are running fine but final report displays only the last specs run results in save path. consolidate all options is not giving the full reports.

please find our conf file details. any help will be appreciated.

Edit the conf file as per solution provide by Aditya. please help

   var Jasmine2HtmlReporter = require('protractor-jasmine2-html-reporter');
var log4js = require('log4js');
var params = process.argv;
var args = process.argv.slice(3);

exports.config = {
  //seleniumServerJar: './node_modules/gulp-protractor/node_modules/protractor/selenium/selenium-server-standalone-2.48.2.jar',
  seleniumAddress: 'http://localhost:4444/wd/hub',
  allScriptsTimeout: 100000,
  framework: 'jasmine2',

  onPrepare: function () {

    return new Promise(function(fulfill, reject) {
      browser.getCapabilities().then(function(value) {
        reportName = value.get(Math.random(8,2)) + '_' + value.get('browserName') + '_' + Math.floor(Math.random() * 1E16);
        jasmine.getEnv().addReporter(
          new Jasmine2HtmlReporter({
            //cleanDestination: false,
            savePath: __dirname+'/target',
            //docTitle: 'Web UI Test Report',
            screenshotsFolder: 'image',
            //takeScreenshots: true,
            takeScreenshotsOnlyOnFailures: true,
            consolidate: true,
            consolidateAll: true,
             preserveDirectory: true,
            //fixedScreenshotName: true,
            filePrefix: reportName + ".html"
          })
        );
        fulfill();
      });
    });

    // browser.manage().timeouts().implicitlyWait(11000);
    var width = 768;
    var height = 1366;
    browser.driver.manage().window().setSize(768, 1366);
    browser.ignoreSynchronization = false; 
  },

  afterLaunch: function afterLaunch() {
    var fs = require('fs');
    var output = '';
    fs.readdirSync('target/').forEach(function (file) {
      if (!(fs.lstatSync('target/' + file).isDirectory()))
        output = output + fs.readFileSync('target/' + file);
    });
    fs.writeFileSync('target/ConsolidatedReport.html', output, 'utf8');

  },

  suites:{

    example:['./test/e2e/specs/**/*Spec.js',]
  },


  /*  capabilities: {
      'browserName': 'chrome'
    },*/

  multiCapabilities: [
    {
      'browserName': 'chrome'
    },
    {
      'browserName': 'firefox'
    }
  ],


  resultJsonOutputFile:'./results.json',

  // Options to be passed to Jasmine-node.
  jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 100000
  }
};

Solution

  • The limitation is with the 'Jasmine2HtmlReporter' as it overwrites the html report file when tests run in parallel. But avoiding this is definitely possible and there are couple of ways of doing it. Pick up the right way based on your convenience

    1) Tinker with 'index.js' of Jasmine2HtmlReporter to append file instead of PhantomJs overwrite its using

    2) Generate unique HTML reports by configuring Jasmine2HTML reporter from onPrepare() function and consolidate all the reports later

    SOLUTION 1: The current code base of Jasmine2HtmlReporter - index.js uses two functions - phantomWrite() & nodeWrite() to write data. Refer here

    I have created a new function - appendwrite() to append instead of overwriting and have modified code to pickup this function Check out my github code forked out of protractor-jasmine2-html-reporter

            function appendwrite(path, filename, text){
                var fs = require("fs");
                var nodejs_path = require("path");
                require("mkdirp").sync(path); // make sure the path exists
                var filepath = nodejs_path.join(path, filename);
                fs.appendFileSync(filepath,text)
                return;
            }
    

    And modify the self.writeFile function in 'node_modules/protractor-jasmine2-html-reporter/index.js' to pickup the new function

            try {
                appendwrite(path, filename, text);
                //phantomWrite(path, filename, text);
                return;
            } catch (e) { errors.push('  PhantomJs attempt: ' + e.message); }
            try {
                nodeWrite(path, filename, text);
                return;
            } catch (f) { errors.push('  NodeJS attempt: ' + f.message); }
    

    And Comment the below code which cleans reports on new run so that you dont see any error cleanup error - CleanUpCode

        rmdir(self.savePath);
    

    SOLUTION 2: Generate separate reports based on sessionID for parallel instances by configuring the Jasmine reporter in OnPrepare function

    onPrepare: function() {
            return new Promise(function (fulfill, reject) {
                browser.getCapabilities().then(function (value) {
                    reportName = value.get('webdriver.remote.sessionid') + '_' + value.get('browserName') + '_' + Math.floor(Math.random()*1E16);
                    jasmine.getEnv().addReporter(
                        new Jasmine2HtmlReporter({
                            savePath: 'target/',
                            screenshotsFolder: 'images',
                            consolidate: true,
                            consolidateAll: true,
                            filePrefix: reportName + ".html"
                        })
                    );
                    fulfill();
                })
            });
        },
    

    Step 2: Consolidate the reports generated across parallel instances in afterLaunch() method after complete tests are done and all webdriver sessions are closed

    afterLaunch: function afterLaunch() {
            var fs = require('fs');
            var output = '';
           fs.readdirSync('target/').forEach(function(file){
               if(!(fs.lstatSync('target/' + file).isDirectory()))
                output = output + fs.readFileSync('target/' + file);
           });
            fs.writeFileSync('target/ConsolidatedReport.html', output, 'utf8');
        },
    

    You will see reports generated something like below with one ConsolidatedReport also PS: Please ignore any typo and syntax errors. this is just to serve as an example and can be customized

    enter image description here

    EDIT1: The 'sessionID' we are using to name the HTML report is the webdriver remote sessionID and if you have doubt that it may not remain unique through multiple sessions, Just generate a random number for the individual HTML reports and consolidate later

    I have modified the code above