The autotest instance

The autotest package provides test data and expected results to test the services app of EOxServer. The package content need to be copied in a minimal EOxServer instance from where the tests can be run.

Installation

In order to run the tests, a new EOxServer instance has to be created with the eoxserver-admin.py script which creates a basic directory and file structure for a minimal EOxServer instance:

eoxserver-admin.py create_instance autotest

Note

Any valid instance name may be used instead of autotest. Just make sure to adjust the following commands.

Use the --init_spatialite to initialize a SQLite database needed for running the autotest tests against SQLite:

eoxserver-admin.py create_instance autotest --init_spatialite

Now the EOxServer instance can be filled with its content, downloaded for example from the EOxServer project download page and unpacked into the previously created instance:

wget http://eoxserver.org/export/head/downloads/EOxServer_autotest-<version>.tar.gz
tar xvfz EOxServer_autotest-<version>.tar.gz
cp -R EOxServer_autotest-<version>/* autotest

Note

The version needs to be adjusted to the version of EOxServer under test. If testing the current development branch or the latest of any stable branch please download the autotest data directly from the repository.

In order to successfully run all tests two configuration directives in conf/eoxserver.conf need to be adjusted:

[services.auth.base]
pdp_type=dummypdp

[services.ows.wcst11]
allowed_actions=Add,Delete

Currently there are two configuration directives in conf/eoxserver.conf in the testing directive which allow to skip certain test cases known to be problematic on some systems. Please refer to the corresponding section in the configuration options documentation.

Note

The reference platform used during the continuous integration builds is based on Ubuntu 12.04 64bit. If the testing is performed on a different platform please disable the binary comparison of rasters (see above).

Two configuration directives in settings.py, which is automatically generated by the eoxserver-admin.py script, are particular important for running the tests:

  • FIXTURE_DIRS has to include the directory holding the autotest fixtures which is usually data/fixtures
  • TEST_RUNNER should be set to eoxserver.testing.core.EOxServerTestRunner in order to be able to run test cases by regular expression search (see eoxserver.testing.core)

The autotest instance is now installed and ready for some testing!

Running tests

Most of the tests in EOxServer use the Django test framework, which itself is built upon Python’s unittest framework.

To run tests against a component of EOxServer simply run:

cd autotest/autotest
python ../manage.py test <component>

where <component> is one of services, core, backends, coverages and processes. If all components shall be tested in one pass, just omit the <component> parameter. Detailed information about running Django tests can be found in the according chapter of the Django documentation.

Running single tests

Single tests or groups of tests can be run by appending the test name or beginning of the test name to the component:

python manage.py test services.WCS20GetCapabilities

XML Validation

In order to speed up the tests and also to pass certain tests it is highly recommended to make usage of locally stored schemas via XML Catalog:

wget http://eoxserver.org/export/head/downloads/EOxServer_schemas-<version>.tar.gz
tar xvfz EOxServer_schemas-<version>.tar.gz
export XML_CATALOG_FILES=`pwd`"/EOxServer-<version>/schemas/catalog.xml"

This allows the underlying libxml2 to vastly improve the performance of parsing schemas and the validation of XML trees against them. Also, several schemas contain small errors which makes it impossible to correctly use them in a real validation scenario. The self contained schemas package provides only useable schemas.

Running the autotest instance

First the configuration of the instance has to be finalized. After the successful Database Setup it needs to be initialized:

cd autotest
python manage.py syncdb

Either a Django superuser needs to be defined while running the command or the auth_data.json loaded as described in the next section.

Loading test data

Test data is provided as fixtures plus image files. To register all available test data simply run:

cd autotest
python manage.py loaddata auth_data.json initial_rangetypes.json \
                          testing_base.json testing_coverages.json \
                          testing_asar_base.json testing_asar.json \
                          testing_reprojected_coverages.json

The following fixtures are provided:

  • initial_data.json - Base data to enable components. Loaded with syncdb.
  • auth_data.json - An administration account.
  • initial_rangetypes.json - Range types for RGB and gray-scale coverages.
  • testing_base.json - Range type for the 15 band uint16 test data.
  • testing_coverages.json - Metadata for the MERIS test data.
  • testing_asar_base.json - Range type for the ASAR test data.
  • testing_asar.json - Metadata for the ASAR test data.
  • testing_reprojected_coverages.json - Metadata for the reprojected MERIS test data.
  • testing_rasdaman_coverages.json - Use this fixtures in addition when rasdaman is installed and configured.
  • testing_backends.json - This fixtures are used for testing the backend layer only and shouldn’t be loaded in the test instance.

Running the development web server

Django provides a lightweight development web server which can be used to run the autotest instance:

cd autotest
python manage.py runserver

The autotest instance is now available via a standard web browser at http://localhost:8000/

The Admin Client is available at http://localhost:8000/admin or via the Admin Client link from the start page. Note that if the auth_data.json has been loaded there is a superuser login available with username and password “admin”.

Sample service requests are described in the Demonstration section.

Selenium

The Selenium testing framework is a powerful tool to create and run GUI test cases for any browser and HTML based application. It uses low-level mechanisms, such as simulating simple user input, to automate the browser and to test the application.

Currently the only browser supported is Firefox using the Selenium IDE plugin. It is basically a tool to record and play test cases and it also supports exporting the test scripts to several scripting languages as Java, Ruby, Python and Selenese, a basic HTML encoding.

../../_images/selenium-ide.png

Before the test cases can be run, ensure that the databases backends and coverages are empty and the EOxServer is run by either its developement server or within a webserver environment. To clear the databases in question type:

python manage.py reset coverages backends

and confirm the deletion. But be aware that this deletes all data previously entered in the database.

The autotest instance provides two test suites, one for the Admin interface and one for the Webclient interface. To open a testsuite with Selenium IDE navigate to File->Open Test Suite... and open the suite of your choice.

To start the test run click on the Play entire test suite button. Alternatively, you can choose a single test case by double clicking it and then press the Play current test case button. Note: especially in the admin test suite several test cases have dependencies on other test cases to be run first, so many test cases will fail when its dependencies are not fullfilled. The best option is to play the entire test suite as a whole and view the results afterwards.

Note that the test speed should be decreased in order to allow enough time to fill the pages and thus pass the tests.

Don’t forget to adjust the base URL when the autotest instance is not run locally.

Note that when testing the admin interface, before the tests can be rerun, the database has to be emptied, as explained in the example above.