Integration of SOA testing tool with continuous build integration tool

Introduction
In this era of Web 2.0 and SaaS, SOA is being more popular than ever in product developments. Thus most of the developments are related to implementing web services for exposing their core/business logic. As the web services are being popular, the testing demands are increasing for web services. Also, apart from one time testing, demand for regression testing of web services is increasing rapidly. To achieve this we have to automate the web service testing and that can be achieved with SOA testing tools like SOAPUI etc.
Apart from this the demand of regression does not stop here but many product managers are interested in running regression test for each and every build and test each and every nightly build. In this case, QA department have to run the web services test cases after each nightly build and for that we have to integrate and automate testing of nightly build using continuous integration tool like Hudson etc.
What is SOA testing and how does it work? There are many tools that are available for SOA testing and continuous integration, but I will consider SOAPUI as SOA testing tool and Hudson as continuous integration tool here because both are very much used in market now a days. Following sections will describe advantages and disadvantages of SOAPUI, Hudson along with integration.

SOA testing tool — SOAPUI
SOAPUI is the open source web service testing tool, it supports testing of both SOAP and REST based web services with specific steps for SOAP request and REST request. The main thing of SOAPUI is it supports groovy script. SOAPUI is developed by eviware. Same eviware has developed SOAPUI PRO which is licensed version and it have some special features like data driven testing related steps like data source, data loop etc. Also, SOAPUI support is many types of assertions like script based assertions etc. Apart from this PRO version provides facility to create library for common functions. For this we have to put library in EXT folder of SOAPUI installation.

On technical side, SOAPUI is developed in JAVA so we can extend the soapui runner and can override the events of SOAPUI which is the good feature for event handling and related actions.And most important thing of SOAPUI is we can create our own type of formats for reports and also it supports JUnit style report which will make our integration very easy.

Apart from this Hudson supports for defining next job and previous job for any job. So we can run one job on completion of the job. Like we can run test job after build job.

Continuous integration tool — Hudson
Hudson have mostly all the features and functionality which should be present in any continuous integration tool. It included features like CVS and SVN integration, post-build and pre-build actions.
Also, Hudson supports JUnit style report and based on that displays the analysis of results with Age, increase or decrease of failures etc.

Integration of SOAPUI and Hudson
For integration, we have to consider following requirements and same will be used to decide integration style.
Requirements:

1. SOAPUI should be run from Hudson
2. Generate reports in JUnit style and incorporate with Hudson.
3. Show logs in console output.
4. Run any project with different global properties of SOAPUI

1. SOAPUI should be run from Hudson

Hudson have good feature of execution of batch of commands of windows and Linux and SOAPUI is providing test runner of its own, we can run SOAPUI from Hudson easily. We can run it as same way as we can run soapui from command line. So it can be integrated easily and using Hudson we can view console output as well.
For this first create a new job in Hudson. This link will help you understanding the Hudson management;
After creating a job, go to configure section of the job and “Add build step” to run shell script / batch file as below to run SOAPUI from Hudson.

2. Generate reports in JUnit style and incorporate with Hudson

Hudson have inbuilt functionality of digesting JUnit style report and SOAPUI generates reports in JUnit style and Hudson generates good trending and result reports based on it. To enable JUnit style report in Hudson we have to select “Publish JUnit test result report” option and it will ask for JUnit style report xml file path. If you are creating different JUnit style report in different folder than give that path using wild cards and Hudson will check for all the JUnit style report and will show the consolidated results and trends. The path should be relative to WORKSPACE.

3. Show logs in console output

As described in point-1, it is necessary to have log information handy for test and Hudson have special area to show console output.

4. Run any project with different global properties of SOAPUI

To make our SOA test tool dynamic, we must support run time property setup option and SOAPUI command line runner supports this via global properties. Because we can setup global properties values during run time and which we can use to achieve dynamic nature of our test framework and as we know Hudson support execution of tool through command line or shell scripting.
For example, I want to run SOA test daily after nightly build of my application. Here we are assuming that our SOAUI test project is available in SVN and application source is also available in SVN. So for this the steps for setting up job will be as below:

1. Create one new job in Hudson for building the code.
2. Configure SVN from which we have to download the latest source of application.
3. Configure build steps like ant script or batch file or shell script.
4. Configure schedule to run the job daily.
5. Save job.
6. Create one more new job in Hudson for testing the build which was setup in previous step.
7. Configure SVN path to get latest test project from SVN.
8. Configure the batch commands to invoke and run the soapui tests.
9. Set build job as triggering job to start this test job.
10. Set email address to whom we want to send the test report.
11. Save the job.

Now the source build job will start at scheduled time and completion of build it will start the test job.

Conclusion
There are many options in Hudson and SOAPUI to achieve maximum flexibility of building and testing the jobs and apart from this we have integrated functional UI testing with Hudson.