It has been way too long since I have posted anything about SoapUI and that is mainly because I have not been very active is using it at work for a long time now. But I recently received an email through this site, stating appreciation  for my SoapUI posts. They also happened to mention that they are now looking for a way to do result reporting with SoapUI Free Edition and it gave me the idea to write this post. Result reporting is one of the biggest selling points of SoapUI Pro, but with a little will and a little time result reports can be pretty easy to produce in SoapUI Free Edition too.

I would love to be able to post the scripts that I have already written, but since I produced them while on projects for my employer, they are proprietary content and I cannot post them in public. So, I will try my best to provide some direction on how I implemented reporting. I actually have two types of reporting that I am able to enable in my SoapUI projects. One would do CSV output of Test Case results and one that will produce HTML log files that contain all the details about a test case if it fails. The CSV file is rather straight forward but as you can imagine the HTML file is rather complex. I will focus on the CSV file example to show how easy it is to produce some reporting, but at the end will also give some examples of more detailed reporting.

Where to start?

The first part would be understanding what type of results you want to capture. The second part that has to be tackled is understanding where in the SoapUI structure that you can start capturing the desired results with Groovy scripts.

When I started I decided that I wanted to be able to do reporting on each Test Case and I wanted to be able to get the reporting even if I were to run a Test Case individually from the TestCase Editor. After some research I found that I can capture the results of a Test Case run in the TearDown Script of each TestCase. The TearDown Script is invoked with the testRunner variable and it is the variable that contains all of the status information from the run of the Test Case. So, within the TearDown script you could do something like this:

try {
    //Check a project level property to see if the reports should be produced.
    if(context.expand('${#Project#ProduceReports}') == 'true') {
        //The path and file to persist results
        def resultDir = new File("C:\\SoapUIResults");
        if(!resultDir.exists()) {
            resultDir.mkdirs();
        }
        def resultsFile = new File(resultDir, "CSVReport.csv");

        //If the file does not already exist, we want to create it, otherwise we want to append
        if(!resultsFile.exists()) {
            resultsFile.createNewFile();
            //Header values
            resultsFile.write('"NAME","RESULT","MESSAGES","DATE"');
        }

        //Write the result values
        resultsFile.append('\n');    //Newline
        resultsFile.append('"' + testRunner.testCase.name + '",');    //Test Case Name
        resultsFile.append('"' + testRunner.status + '",');    //Overall Test Case result

        //There can be multiple messages, so set up a loop
        resultsFile.append('"');    //Start of messages.
        for(result in testRunner.getResults()) {
            //There can be lots of messages, so limit amount recorded to avoid going over
            //the buffer size for a CSV field
            msgCount = 0;
            for(message in result.getMessages()) {
                msgCount++;
                if(msgCount < 10) {
                    resultsFile.append(message + ';\n');
                }
            }
        }

        resultsFile.append('",');    //End of messages.

        def currentDate = new Date().format("yyyy-MM-dd hh:mm");
        resultsFile.append('"' + currentDate + '"');
    }
} catch(e) {
    log.error("An error occurred: " + e.toString());
}

The only left to do is to add the 'ProduceReports' project level property that is referenced in the first 'if' statement. I like to use project level properties to work as configuration values. Being able to turn certain functionality on/off for the whole project in one place. So, go to the project level and add a property named 'ProduceReports' and give it a value of 'true'.

Hopefully with the naming and the comments I used in the script it all makes sense. You could also just copy and paste it into one of your Test Case's TearDown scripts and give it a try. Make your TestCase fail and see the messages that are produced.

Many of you might be saying "That is all fine and dandy, but I have hundreds of TestCases and I don't want to maintain that script in all of my Test Cases." I agree, it is not optimal and not good code reuse if you try to put that code in every single TestCase. So, at this point I am going to use some practices I discussed in my entry about Script Libraries in SoapUI Free. I encourage reading that, but I hope to provide enough details to make sense without needing to read that entry.

Create a new Test Suite in your project named 'Scripts' (disable this Test Suite as we do not want it to run if the whole project is run), create a Test Case in that Test Suite and name it 'Reporting' and add a Groovy Script step to that Test Case and name it 'CSVFile'. Now take the code above, exactly as is, and paste it into the 'CSVFile' Groovy Script step. In the 'Reporting' Test Case add another Groovy Script step named 'CSVFile TearDown'. In that step enter on simple line:

"testRunner.testCase.testSuite.project.testSuites["Scripts"].testCases["Reporting"].testSteps["CSVFile"].run(testRunner, context);"

That single line runs the 'CSVFile' script that will generate the CSV file of results. That single line will now be the only code that you need in each Test Case's TearDown script, and this is the only place that you will need to maintain that code. How? We will create one additional Groovy Script step that has a sole purpose to populate all of the Test Case TearDown scripts with that code. So, on the 'Reporting' Test Case, add one more Groovy Script step and name it 'Populate TearDown Script'. Populate that step with the following:

try {
    def tearDown = testRunner.testCase.testSteps["CSVFile TearDown"].getScript();

    def project = testRunner.testCase.testSuite.project;

    //Loop through each Test Suite
    for(suite in project.getTestSuiteList()) {
        //Do not want to add script to test cases in Scripts Suite.
        if(!suite.name.startsWith("Scripts")) {
            //Loop through each Test Case
            for(tcase in suite.getTestCaseList()) {
                tcase.setTearDownScript(tearDown);
            }
        }
    }
} catch(e) {
    log.error(e.toString());
}

Now, if you run that script by itself, all of the Test Cases in the current project will be populated with the single line of code that is in the 'CSVFile TearDown' script step. This would be the only manual thing you would need to do. If you add new Test Cases to your project you would want to re-run that script to make sure the new Test Cases get the needed TearDown script. Also, if you change the 'CSVFile TearDown' at all, you would want to re-run the 'Populate TearDown Script' script to get those changes into the Test Cases.

Nice for such a small amount of code, right? But at the same time, it may not be detailed enough for what you want. No need to worry, there is a lot more information available. In the 'CSVFile' script a lot of the information you would want would be available in the 'for' loop that is looping through all of the results ("for(result in testRunner.getResults())"). Here is an example of some of the data that you may want to report on:

for(result in testRunner.getResults()) {
    def testStep = result.getTestStep();

    def type = testStep.config.type    //Returns the type of test step (i.e. 'groovy', 'delay', 'properties', 'request', etc.)
    
    switch(type){    //determine what type of step it is and collect different info based on the step type
        case 'delay':    //Delay test step
            def delayTime = testStep.getDelayString();
            break;
        case 'properties':    //Properties test step
            for(testProp in testStep.getPropertyList()) {    //Go through each property in the property step
                 def propName = testProp.getName();
                 def propValue = testProp.getValue();
            }
            break;
        case 'request':    //SOAP Request/Response test step
            def xmlRequest = testStep.properties["Request"].value;    //The Xml tab content of a Soap Request Test Step
            def rawRequest = result.getRequestContent();    //The raw request that is sent to the endpoint
            def rawResponse = result.getResponseContent();    //The raw response that is recieved from the service
            break;
    }
}

That is a small sampling. I hope this entry helps. As always, if you have any questions just let me know.

Comments

re: Reporting in SoapUI Free Edition

Wednesday, October 26, 2016 4:56:21 AM

Hi,My report is not generated using this script, can you please share image tutorial also this will be great help for me.

re: Reporting in SoapUI Free Edition

Wednesday, April 5, 2017 7:47:46 AM

Hey Doan,

Thanks for this post. I able to create the csv report using above script.

But the customization of report, I could not update the CSVFile successfully like fetching response content for each test step in the CSV file.

Please help me how can have responses logged in result column on CSV file

re: Reporting in SoapUI Free Edition

Wednesday, April 5, 2017 8:26:58 AM

Hello all,

As you might be able to tell, this is a rather old post now (from 2013). So, newer versions of SoapUI might not work the same way. Also, I no longer work with SoapUI as my work assignments have moved me on to other things. Best of luck with finding a solution.

You must sign in to this site to post comments.
Already Registered?
Login
Not Yet Registered?
Register

Support and Encourage Me

 


 

Proud member of the mojoPortal team

 

Security Plus Logo


 

I use and recommend SmarterASP.NET for my ASP.NET websites

With a 60 day free trial and then a 60 day money back guarantee what do you have to lose? Give them a try!