Search code examples
groovysoapui

Creating a Test Report from Project level tear down script

I have generate a report based on an execution of a test suite where it creates a folder directory and insert a file displaying the report. This is compiled within a TearDown Script at Test Suite level. Below is the code:

def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context) 
def dataFolder = groovyUtils.projectPath
def failedTestCases = 0
def succeedTestCases = 0
def totalTestCases = 0
def testCaseFailed = ""
def testCaseSucceed = ""
def date = new Date()
def folderTime = date.format("yyyy-MM-dd HH-mm-ss")
def hotelId = context.getProperty('hotelid')
def hotelname = context.getProperty('hotelname')
def hoteltype = context.getProperty('hoteltype')
//def propertyValues = ""
//def correlationid = messageExchange.modelItem.testStep.testCase.testSuite.Project.namegetPropertyValue("correlationid")
//Create a folder directory for the responses
RootResultFolder = dataFolder + "\\Test Reports" + "\\xxx_WebAPI - " + folderTime + "\\"
CreateResultFolder = new File(RootResultFolder)
CreateResultFolder.mkdir()

//context.setProperty("RootResultFolder", RootResultFolder)

def fileName = "WebAPI Test Report.txt"
def rootFolder = RootResultFolder + fileName 
def logFile = new File(rootFolder)



if(logFile.exists())
{

 log.info("Error a file named " + fileName + "already exisits")
}
    else
{
runner.results.each { testCaseResult ->
    def name = testCaseResult.testCase.name
    totalTestCases++
    if(testCaseResult.status.toString() == 'FAILED'){
        failedTestCases ++
        testCaseFailed += "- $name - HAS FAILED \n\n"
       //propertyValues += "hotelid - $hotelid, hotelname - $hotelname, hoteltype - $hoteltype \n\n"
        testCaseResult.results.each{ testStepResults ->
            testStepResults.messages.each() { msg -> log.info msg } 
        }
    }else{
        succeedTestCases ++
        testCaseSucceed += "- $name - SUCCEED \n\n"
        testCaseResult.results.each{ testStepResults ->
        testStepResults.messages.each() { msg -> log.info msg } 
        }
    }
}
}


logFile.write   "TOTAL TEST CASES SUCCEED: $succeedTestCases of $totalTestCases" + "\n\n" +
            testCaseSucceed + "---\n\n" +
            "TOTAL TEST CASES FAILED: $failedTestCases of $totalTestCases" + "\n\n" +
            testCaseFailed + "\n\n"

What I actually want to do is move the code from Test Suite level and place it in the tear down script at Project level. Now when I run the code from there, it does not generate the file, I'm assuming I need to place the correct paths in as I am not moving to test suite to test case but from project to test suite to testcase to test steps.

My question is really on syntax, I want to develop a report when the whole project is run, it outputs the following results:

  • Project Name - is it success or failed. If one suite failed then project fails else it passes
  • Test Suite - Take name of each test suite in project and if passes then place 'Succeed' next to name of test suite else place 'Failed' next to name of test suite
  • Name of all test cases within test suite. Like the one in screenshot really, 'succeed' next to test cases that have passed and 'failed' next to those that haven't.
  • Finally the property values. If a test case has failed, capture the property values for that failed test case so we can track which values were entered that caused the failure of the test.

Can somebody help me with the relevant syntax to perform these so then I can peice it into my code and manipulate?

UPDATE:

def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context) 
def dataFolder = groovyUtils.projectPath
def date = new Date()
def folderTime = date.format("yyyy-MM-dd HH-mm-ss")

//Create a folder directory for the responses
RootResultFolder = dataFolder + "\\Test Reports" + "\\xxx - " + folderTime + "\\"
CreateResultFolder = new File(RootResultFolder)
CreateResultFolder.mkdir()*/

//context.setProperty("RootResultFolder", RootResultFolder)

def reportFileName = "WebAPI Test Report.txt"
def rootFolder = RootResultFolder + reportFileName 
def logFile = new File(rootFolder)
Solution

  • If you look at the TearDown Script of the project, it shows as below i.e., the variables already initialized by soapui.

    enter image description here

    Issue with your script
    So if you look at it, there is runner variable. Also the same variable is available at TearDown script of test suite level. But, these are instances of different Objects. The script used in OP was of suite level which you aware and that is why you are not seeing in the result.

    Here is the project level TearDown Script and following in line comments.

    /**
*
* Below is the TearDown script for SoapUI Project level
* Which create a custom report in a  given file
* Modify the variable "reportFileName" below
*
**/
//Modify the file as needed for report file
//def reportFileName = '/tmp/abctestreport.txt'
//Adding the below as user wants specific directory
//Get the project path

def dataFolder =  new com.eviware.soapui.support.GroovyUtils(context).projectPath

//Create today's date for storing response
def today = new Date().format("yyyy-MM-dd")

def filePrefix = "${dataFolder}/TestReports/xxx_WebAPI_${today}" as String

def fileNamePart = new Date().format("yyyy-MM-dd'T'HH.mm.ss")

//creating filename dynamically.
def reportFileName = "${filePrefix}/xxx_WebAPI_TestReport_${fileNamePart}.txt" as String


//NOTE: Not required to edit beyond this point

/**
* This class holds the test case details
**/
class TestCaseResultHolder {
    def log
    Map<String, String> properties = [:]
    boolean status

    def createProperties(testCase) {
        testCase.getPropertyNames().each { key ->
            properties[key] = testCase.getPropertyValue(key)
        }       
    }

    def getCaseResult(caseRunner, caseName) {
        log.info "Checking test case status ${caseName}"
        if ( caseRunner.status.toString() == 'FAILED' ){
            log.error "Test case $caseName has failed"
            for ( stepResult in caseRunner?.results ){
                stepResult.messages.each() { msg -> log.info msg }
            }
            return false
        } else {
            log.info "${caseName} is passed"
        }
        true
    }

    def buildCaseResult(caseRunner, caseName) {
        status = getCaseResult(caseRunner, caseName)
        if (!status) {
            createProperties(caseRunner.testCase)
        }
    }

}

/**
* This class holds the test suite details
**/
class SuiteResultsHolder {

    def log
    Map<String, TestCaseResultHolder> casaeResults = [:]
    int testCaseCount = 0
    int passedCasesCount = 0
    int failedCasesCount = 0

    def buildSuiteResults(suiteRunner, suiteName){      
        log.info "Building results of test suite ${suiteName}"
        for ( caseRunner in suiteRunner?.results ) {
            def caseName = caseRunner.testCase.name
            testCaseCount++
            def tcHolder = new TestCaseResultHolder(log: log)
            tcHolder.buildCaseResult(caseRunner, caseName)          
            casaeResults[caseName] = tcHolder
            if (tcHolder.status) {
                passedCasesCount++
            } else {
                failedCasesCount++
            }
        }
    }

    def getStatus() {
        (0 < failedCasesCount) ? false : true
    }

}

/**
* This class holds the project details
**/
class ProjectResultsHolder {

    def log
    Map<String, SuiteResultsHolder> suiteResults = [:]
    int suiteCount = 0
    int passedSuitecount = 0
    int failedSuiteCount = 0

    def buildProjectResults(projectRunner, projectName) {
        log.info "Building results of test project ${projectName}"          
        for(suiteRunner in projectRunner?.results) {
            def suiteName =  suiteRunner.testSuite.name
            suiteCount++
            def suiteResultsHolder = new SuiteResultsHolder(log: log)
            suiteResultsHolder.buildSuiteResults(suiteRunner, suiteName)
            suiteResults[suiteName] = suiteResultsHolder
            if (suiteResultsHolder.status) {
                passedSuitecount++
            } else {
                failedSuiteCount++
            }
        }
    }

    def getStatus() {
        (0 < failedSuiteCount) ? false : true
    }

}

//Get the status string based on boolean
def getResult(status){ status == true ? 'SUCCEED' : 'FAILED'}

//Draws a line
def drawLine(def letter = '=', def count = 70) { letter.multiply(count)}

//Gets the summary report
def getSummaryReport(project, projectResultHolder) {
    def report = new StringBuffer()
    report.append(drawLine()).append('\n')
    report.append("\t\t\tTest Execution Summary\n")
    report.append(drawLine('-', 60)).append('\n')
    report.append("Project : ${project.name}\n")
    report.append("Result : ${getResult(projectResultHolder.status)}\n")
    report.append("Total test suites executed: ${projectResultHolder.suiteCount}\n")
    report.append("Test suites passed: ${projectResultHolder.passedSuitecount}\n")
    report.append("Test suites failed: ${projectResultHolder.failedSuiteCount}\n")
    report.append(drawLine()).append('\n')
    report
}

//Gets the test case report
def getTestCaseReport(testCaseReport) {
    def report = new StringBuffer()
    report.append(drawLine('-', 60)).append('\n')
    report.append("\t\tTest Case Details:\n")
    report.append(drawLine('-', 60)).append('\n')
    testCaseReport.each { kase, tcReport ->
        report.append("Name : ${kase}\n")
        report.append("Status : ${getResult(tcReport.status)}\n")
        if (!tcReport.status) {
            report.append("Properties : ${tcReport.properties.toString()}\n")
        }
    }
    report
}

//Get the detailed report
def getDetailedReport(projectResultHolder) {
    def report = new StringBuffer()
    report.append(drawLine()).append('\n')
    report.append("\t\t\tTest Execution Detailed Report\n")
    report.append(drawLine()).append('\n')
    projectResultHolder.suiteResults.each { suite, details ->
        report.append("Test Suite : ${suite}\n")
        report.append("Result : ${getResult(details.status)}\n")
        report.append("Total Cases : ${details.testCaseCount}\n")
        report.append("Cases Passed : ${details.passedCasesCount}\n")
        report.append("Cases Failed: ${details.failedCasesCount}\n")
        report.append(getTestCaseReport(details.casaeResults))
        report.append(drawLine()).append('\n')
        report.append(drawLine()).append('\n')
    }
    report
}

//Save the contents to a file
def saveToFile(file, content) {
    if (!file.parentFile.exists()) {
        file.parentFile.mkdirs()
        log.info "Directory did not exist, created"
    }
    file.write(content) 
    assert file.exists(), "${file.name} not created"
}

def holder = new ProjectResultsHolder(log: log)
holder.buildProjectResults(runner, project.name)

def finalReport = new StringBuffer()
finalReport.append(getSummaryReport(project, holder))
finalReport.append(getDetailedReport(holder))

def reportFile = new File(reportFileName)
saveToFile(reportFile, finalReport.toString())

    And here is the generated output:
    enter image description here