Test Framework

Globus Toolkit 3.0 - Last Updated 07/15/2003

1 Introduction

This document describes the test framework available in GT3 Core. The framework can be used to run tests against core distributions to verify correct installations and setups, and also provides a set of APIs and tools to facilitate testing of components built on top of GT3 Core. This test framework is currently only available in source distributions. 

2 Tool Dependencies

Junit 3.8.1 - The tests are built on top of the JUnit test toolkit.  You need to download JUnit from http://www.junit.org where you also can find further documentation about the JUnit APIs and tools. Then put junit.jar in the GT3 distribution lib directory, in the ant lib directory or in anywhere in your classpath before running the tests.

Ant 1.5.3 - All the tests are run via ant, and also extend some ant tasks, so you need to download and install ant from http://ant.apache.org 

3 Running the Tests

Go to the  impl/java directory in your GT3 Core source distribution. Now to run all tests type ant testAll. If you do not have a GSI security proxy set up on your machine you cannot run the security tests. To run all tests but the security tests type ant test. Also make sure that you have a clean source distribution before running the tests by typing ant cleanAll. Below is a table summarizing the basic test targets available.

testAll runs all tests
test runs all tests but security tests
testSecurity runs security tests
testCompatibility runs OGSI compatibility tests
testTools runs tests of grid service generation tools
testPackage runs individual package test. See section 5.

All these tests will run against a local server started in the same JVM as the clients on a dynamic port. You can also run the tests against a remote server. In order to do so you need to specify the test.server.url property and pass it into ant as follows: ant <test>Server -Dtest.server.url=http://<host>:<port>/ogsa/services/. Where <test> can be any of test, testSecurity, testCompatibilty or testPackage depending on what tests you want to run. If you run the server tests without this property set it will default to http://127.0.0.1:8080/ogsa/services/

4 Tests Conventions

Tests of a component are put in a subdirectory called test of that component. This directory should container test suite classes, collecting all the individual tests of the components. The suites are named after conventions based on when they should be run and what kind of functionality they test. Below is a summary of suite name conventions:

PackageTests will automatically be run by the test target
PackagePostTests will automatically be run by the test target, and guaranteed to run after any PackageTests tests
PackagePostPostTests will automatically be run by the test target, and guaranteed to run after any PackagePostTests tests
SecurityTests contains test cases requiring security credentials, and will be run using the testSecurity targets
CompatibilityTests contains test cases used to test OGSI compatibility, and may not be used to test non-standard functionality. These tests are run with the ant testCompatibilty targets
ToolsTests run by testTools, and used to test tools for building Grid services

5 Running Individual Tests

In order to just run a single test suite you need to run the testPackage ant task as follows:

ant testPackage -Dpackage.test=<package test> -Dtest.pattern=<test pattern>

<package test> must be the full path to the test suite, e.g org/globus/ogsa/encoding/test/PackageTests
<test pattern> Only test cases in the suite matching this pattern will be run. If omitted all test cases will be run. Currently the only supported pattern matching is substring comparison.

6 Writing a Tests

First of all pick an name for your test suite that follows the conventions in section 3. To take full advantage of the test framework features you should make your suite extend from the GridTestSuite class, which is an extension of the Junit TestSuite class. You may also extend from the JUnit class directly if you don't need to make use of the GridTestSuite extensions, such as collocated and remote server test support, and test case pattern matching.

Here is a simple example of a PackageTest class:

package org.globus.ogsa.encoding.test;

import junit.framework.Test;
import org.globus.ogsa.server.test.GridTestSuite;

public class PackageTests extends GridTestSuite {
    public PackageTests(String name) {
        super(name);
    }
    public static Test suite() throws Exception {
        GridTestSuite suite = new PackageTests("SerializationTests");
        suite.addTestSuite(TestSerialization.class);
        return suite;
    }
}

Similarly when implementing your test case you can extend from the GridTestCase class which in turn extends the JUnit TestCase class. The GridTestCase class gives you access to a static TEST_SERVER variable of type org.globus.ogsa.server.test.TestServer, which can be used to run collocated or remote tests depending on configuration. Typically you would use the TEST_SERVER url to compose handles of services to call. Please see the JUnit documentation at http://www.junit.org for details on how to write test cases.

7 Stress Test Framework

In order to stress arbitrary tests with multithreaded clients and clients running in multiple JVMs, we have developed a stressJava ant target that extends the ant java target. The stressJava task supports the following attributes (ant property names within paranthesis):

className (-Dstress.class) The class name of the stress test client to run. It must implement the org.globus.ogsa.tools.ant.StressTest interface.
threads (-Dstress.threads) Number of threads used to run the test (per process)
processes (-Dstress.processes) Number of processes used to run the test
repeats (-Dstress.repeats) Number of times within a thread the run method of the test case will be run
concurrent (-Dstress.concurrent) yes/no. If no disables concurrent threads in a process
args (-Dservice.root -Dservice.port -Dstress.path -Dstress.args) stress test specific arguments passed in to the init method of the test

We currently provide 5 built in ant stress test targets, described below:

stressService stress tests a SOAP service invocation, without parameter serialization, nor deserialization
stressFactory stress tests Grid service instance creation
stressStringSerialization test of string array serialization (-Dstress.args should contain the array size)
stressIntSerialization test of int array serialization (-Dstress.args should contain the array size)
stressCustomSerialization test of java bean (xsd:complexType) array serialization (-Dstress.args should contain the array size)

All tests will dump an xml based performance result log in a file when completed.

To, for example,  run the factory test from 5 processes with 10 concurrent threads in each process creating 20 services you can run the following command:

ant stressFactory -Dstress.processes=5 -Dstress.threads=10 -Dstress.repeats=20 -Dstress.concurrent=yes