JavaScript based functional testing

The problem

Every software developer very well knows that producing new code leads to producing new bugs (problems in the software).
The good software developers know that writing tests for their code doesn"t solve completely the problem but at least prevents regressions. Unfortunately developers often do not have time to write the tests, or are too lazy to do it, or just don"t like the way they do it and try to postpone it.

In this article I"ll describe a prototype of a mini framework that can be used to create functional tests for web applications that run blazing fast, are easy to debug and more imporatantly are very close to the real way a web application is being used.

Note: this article is not Wicket specific. The described solution can be used for any kind of web application.

The solution

A well known library for writing functional tests for web applications is WebDriver/Selenium2.
My problem with it is that it is:

slow - starting the browser instance takes time, issuing commands to it takes time, HtmlUnitDriver with JavaScript enabled is very slow...
unreliable - every Driver implementation behaves differently. For me only FirefoxDriver works well
well, let"s not say more bad words for WebDriver. Some of its developers may have the same opinion for Apache Wicket :-)

In this article I"m going to describe how to use QUnit to run functional tests for Wicket Examples.

Apache Wicket project already uses QUnit for its unit tests since version 6.0.0 and so far they seems to serve us very well.

The main idea is that the tests run the application in an iframe, i.e. each test tells the iframe what is its initial source url, then the test can enter data in input fields, click links and buttons and finally assert the result of the previous actions.

CORS

The first problem is Cross origin requests - the tests need to be in the same domain as the application to be able to manipulate it.
There are (at least) two possible solutions:

run the application with the proper Access-Control-Allow-Origin header value
create a new application that merges the tests in the original application

The first approach is too obtrusive. It makes the application aware of its clients (the tests).
I prefer the second solution - by using Maven War Overlays we can create a Maven project that contains the tests and injects the original application in itself. This way the tests are available in my-tests folder in the web context for example.

QUnit Setup

To create QUnit tests you need to start with their HTML template. For our functional tests we need to put our additional iframe in its body. It is good to make the iframe big enough to be able to see what happens when you debug a test. In non-debug mode everything happens so fast that you cannot see anything ;-)

Here is the HTML template:

 

<!DOCTYPE html>
<html>

<head>
	<title>Wicket Examples Functional tests</title>
	<meta http-equiv="content-type" content="text/html; charset=UTF-8">
	<link rel="stylesheet" href="lib/qunit-1.10.0.css" type="text/css" media="screen" />
    <script type="text/javascript" src="lib/jquery.min.js"></script>
    <script type="text/javascript" charset="utf-8">
        $q = jQuery.noConflict(true),
                $ = null,
                jQuery = null;
    </script>
    <script type="text/javascript" src="lib/qunit-1.10.0.js"></script>

	<!-- common helpers -->
    <script type="text/javascript" src="Helpers.js"></script>

	<!-- the modules under test -->
    <script type="text/javascript" src="tests/helloworld.js"></script>
    <script type="text/javascript" src="tests/echo.js"></script>
    <script type="text/javascript" src="tests/forminput.js"></script>
    
    <script type="text/javascript" src="tests/ajax/form.js"></script>
</head>

<body>
	<h1 id="qunit-header">Wicket Examples JS tests</h1>

	<h2 id="qunit-banner"></h2>

	<div id="qunit-testrunner-toolbar"></div>

	<h2 id="qunit-userAgent"></h2>

    <!-- Show the iframe to see how the tests run -->
    <iframe id="applicationFrame" width="100%" height="500px" src="../"></iframe>

    <ol id="qunit-tests"></ol>

    <div id="qunit-fixture">

    </div>
</body>
</html>

The interesting things here are:

we preserve current page"s (the tests" page) jQuery in $q variable and unset the original jQuery and $ variables just to avoid name clashes later in our tests.
we add Helpers.js which is a set of helper functions which are used by all our tests
we add <script> for all actual tests
and finally we add the iframe that will run the application

And here is how the tests themselves look like:

helloworld.js:

$q(document).ready(function() {
	"use strict";

	module("Hello World");

	asyncTest("hello world", function () {
		expect(2);

		onPageLoad(function($) {

			var $message = $("#message");
			equal($message.length, 1, "The greeting is there");
			equal($message.text(), "Hello World!", "The greeting is correct")
			
			start();
		});
		getIframe().attr("src", "/helloworld");
	});

});

The idea is that we use QUnit asyncronous tests because often our tests will need to go over several pages in the application and we should not allow QUnit to go to the next test before the current test is finished.

So what does this test do ?
It registers a callback for the load event for the iframe and tells the iframe to go to the page we need to test. When the callback fires we make our assertions and then notify QUnit that we are ready with QUnit#start(). We can register several nested onPageLoad callbacks if the test needs to go over several pages.

The Ajax tests work similarly just they listen for Wicket"s Global . See ajax/form.js from the demo application for example.

Demo

A demo application that integrates wicket-examples.war can be found at my GitHub account.
To run it:

git clone git@github.com:martin-g/blogs.git
mvn install
cd functional-qunit
mvn jetty:run
in a browser open http://localhost:8080/js-test/all.html </ul>
The tests can be integrated in Continious Integration setup easily by using PhantomJS and/or Grunt.js (also based on PhantomJS).
To run the tests with Grunt.js check the header content of grunt.js in the project.

Credits

The original idea is borrowed from User interfaces and unit testing with QUnit

Wicket in Action