Tag Archive | junit

Verify HTML documents in junit tests with jsoup

Assume that you are developing an application that creates some kind of fancy HTML report for its users. When it comes down to writing your unit tests, you have two choices:

  • You test the generated report against a complete report prepared beforehand.
  • You parse the HTML document and test parts of it separately.

The first choice seems to be simple at first glance, because you have manually validated that the prepared report is correct. Writing such kind of tests is also easy as it boils down to the following pattern:

String preparedReport = loadReportFromSomeWhere();
assertThat(generatedReport, is(preparedReport));

But what happens when you change a small part of the report generating code? You will have to change probably some or even all of the prepared reports. Hence the second choice is in these cases the better one, as you only have to adjust the test cases that are affected (and that you would have to change anyhow).

Here is the part where jsoup comes in handy. jsoup is a Java library developed for parsing HTML documents, but in contrast to other options for parsing XML like structures it supports CSS selectors like those used in JavaScript libraries like jquery. This way you don’t have to write tons of code in order to verify exactly the part of the report that your current unit test is concerned with.

To demonstrate how jsoup can be used, we assume that our application has a simple HtmlReport class that can be used to create a valid HTML document using the builder pattern (https://en.wikipedia.org/wiki/Builder_pattern):

String html = HtmlReport.create()
	.addHeader1("title", "Testing HTML documents with jsoup")
	.addSection("intro", "This section explains what the text is all about.")
	.addHeader2("jsoup", "jsoup in a nutshell")
	.addSection("pjsopu", "This section explains jsoup in detail.")
	.addList("jsoup_adv", Arrays.asList("find data using CSS selectors", "manipulate HTML elements"))

To keep it simple, the report just consists of a header element (h1) followed by a section (p) and a paragraph with a header h2 that contains an HTML list (ul). The first argument to each method is the id of the HTML element. This way we can use it later on to address exactly the element we want and beyond that support the formatting of all elements (the CSS designer will love us).

The first thing we want to test is that the document contains an h2 element with id “title”:

<h1 id="title">Testing HTML documents with jsoup</h1>

Using jsoup this verification becomes a two liner:

Document doc = Jsoup.parse(html);
assertThat(doc.select("h1#title").text(), is("Testing HTML documents with jsoup"));

While we let jsoup parse the document in the first line, we can use the provided method select() to query for the element using the selector h1#title, i.e. we are asking for an h1 element with id title. The same way we can assure that we have a paragraph with the correct content:

assertThat(doc.select("p#intro").text(), is("This section explains what the text is all about."));

A little bit more tricky is to verify that the list with id jsoup_adv is written in the correct order. For that we have to use the pseudo selector :eq(n) that allows use to query for a specific index position of a sibling:

assertThat(doc.select("ul#jsoup_adv > li:eq(0)").text(), is("find data using CSS selectors"));
assertThat(doc.select("ul#jsoup_adv > li:eq(1)").text(), is("manipulate HTML elements"));

The selector ul#jsoup_adv > li:eq(0) asks for the first (:eq(0)) li elements that is a direct child of an ul element with id jsoup_adv.

Beyond that one can even use regular expression to find for example all h2 elements whose text ends with the string “nutshell”:

assertThat(doc.select("h2:matches(.*nutshell$)").size(), is(1));

Conclusion: Using jsoup for parsing HTML documents in junit tests makes the verification of HTML documents much easier and robust. If one is used to and likes CSS selectors like they are used by jquery, then jsoup is worth a look.

Testing System.in and System.out with system-rules

Writing unit tests is an integral part of software development. One problem you have to solve when your class under test interacts with the operating system, is to simulate its behaviours. This can be done by using mocks instead of the real objects provided by the Java Runtime Environment (JRE). Libraries that support mocking for Java are for example mockito or jMock.

Mocking objects is a great thing when you have complete control over their instantiation. When dealing with standard input and standard output this is a little bit tricky, but not impossible as java.lang.System lets you replace the standard InputStream and OutputStream.


In order that you do not have to replace the streams before and after each test case manually, you can utilize org.junit.rules.ExternalResource. This class provides the two methods before() and after() that are called, like their names suggest, before and after each test case. This way you can easily setup and cleanup resources that all of your tests within one class need. Or, to come back to the original problem, replace the input and output stream for java.lang.System.

Exactly what I have described above, is implemented by the library system-rules. To see how it works, lets start with a simple example:

public class CliExample {
	private Scanner scanner = new Scanner(System.in, "UTF-8");

	public static void main(String[] args) {
		CliExample cliExample = new CliExample();

	private void run() {
		try {
			int a = readInNumber();
			int b = readInNumber();
			int sum = a + b;
		} catch (InputMismatchException e) {
			System.err.println("The input is not a valid integer.");
		} catch (IOException e) {
			System.err.println("An input/output error occurred: " + e.getMessage());

	private int readInNumber() throws IOException {
		System.out.println("Please enter a number:");
		String nextInput = scanner.next();
		try {
			return Integer.valueOf(nextInput);
		} catch(Exception e) {
			throw new InputMismatchException();

The code above reads two intergers from standard input and prints out its sum. In case the user provides an invalid input, the program should output an appropriate message on the error stream.

In the first test case, we want to verify that the program correctly sums up two numbers and prints out the result:

public class CliExampleTest {
	public final StandardErrorStreamLog stdErrLog = new StandardErrorStreamLog();
	public final StandardOutputStreamLog stdOutLog = new StandardOutputStreamLog();
	public final TextFromStandardInputStream systemInMock = emptyStandardInputStream();

	public void testSuccessfulExecution() {
		CliExample.main(new String[]{});
		assertThat(stdOutLog.getLog(), is("Please enter a number:\r\nPlease enter a number:\r\n5\r\n"));

To simulate System.in we utilize system-rules’ TextFromStandardInputStream. The instance variable is initialized with an empty input stream by calling emptyStandardInputStream(). In the test case itself we provide the intput for the application by calling provideText() with a newline at the appropriate points. Then we call the main() method of our application. Finally we have to assert that the application has written the two input statements and the result to the standard input. The latter is done through an instance of StandardOutputStreamLog. By calling its method getLog() we retrieve everything that has been written to standard output during the current test case.

The StandardErrorStreamLog can be used alike for the verification of what has been written to standard error:

	public void testInvalidInput() throws IOException {
		CliExample.main(new String[]{});
		assertThat(stdErrLog.getLog(), is("The input is not a valid integer.\r\n"));

Beyond that system-rules also offers rules for the work with System.getProperty(), System.setProperty(), System.exit() and System.getSecurityManager().

Conclusion: With system-rules testing command line applications with unit tests becomes even more simpler than using junit’s Rules itself. All the boilerplate code to update the system environment before and after each test case comes within some easy to use rules.

PS: You can find the complete sources here.

Testing HTML5 canvas applications with sikuli and arquillian

HTML5 introduces a great new element that can be used to draw arbitrary content on a pane: the canvas element. What has been a standard feature for fat client applications for decades is now introduced to the world of web applications. Web developers no longer need to use proprietary plugins to draw images or charts in their applications.

But when it comes for testing, this new feature imposes new challenges to the web development community. How to test that the canvas element is in an appropriate state at some point in time? Standard technologies like selenium focus on the markup that is generated by the web server and not on the pixels drawn on the canvas.

More promising in this field are technologies that use image processing to verify that an application renders its data correctly. One of these frameworks is sikuli. Sikuli is an open-source research project that was started at the MIT and is now maintained by Raimund Hocke.

To give a more practical introduction, let’s assume we have a simple web application that uses the HTML5 canvas element to implement some simple image processing functionality like a grayscale, a brighten and a threshold filter as well as an undo button (the code for this application can be found as usual on github):

The installation of sikuli is (of course) platform dependent. The installer, which can be downloaded from the sikuli download page, is a Java Swing application that asks you about your typical usage pattern. As we do not want to use the python IDE, we choose option 4 from the list of options. The actual jar file is then downloaded and prepared for our OS. After the installation process has finished, we find an OS dependent jar file within the installation directory. As our example project uses maven as build system, we have to introduce a system scope dependency after we have copied the library into the lib folder:


When sikuli is used for the first time, it extracts some native libraries into a new folder, here in our example into ${basedir}/lib/libs. This folder has to be added to the user’s path environment variable.

Now that we have installed sikuli, let’s setup arquillian so that we can write our first unit test. How to setup arquillian is described for example here. As I don’t want to repeat everything, in the following you will find only the unit test class:

public class FilterTest {
    public static final String WEBAPP_SRC = "src/main/webapp";
    URL deploymentURL;
    private Screen screen;

    public void before() throws URISyntaxException, IOException {
        screen = new Screen();
        if (Desktop.isDesktopSupported()) {
        } else {

    public static WebArchive createDeployment() {
        return ShrinkWrap.create(WebArchive.class, "html5-sikuli-webapp.war")
                .addAsWebResource(new File(WEBAPP_SRC, "home.xhtml"))
                .addAsWebResource(new File(WEBAPP_SRC, "resources/css/style.css"), "resources/css/style.css")
                .addAsWebResource(new File(WEBAPP_SRC, "resources/images/rom.jpg"), "resources/images/rom.jpg")
                .addAsWebResource(new File(WEBAPP_SRC, "resources/js/html5Sikuli.js"), "resources/js/html5Sikuli.js")
                .addAsWebResource(new File(WEBAPP_SRC, "resources/js/jquery-2.0.3.js"), "resources/js/jquery-2.0.3.js")
                .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml")
                .setWebXML(new File(WEBAPP_SRC, "WEB-INF/web.xml"));

The createDeployment() method sets up the war archive, which is deployed by arquillian to JBoss AS 7.1.1.Final (see arquillian.xml file). In our @Before method we use the SDK class Desktop to open the default browser and point it to the deployment URL. Here we also create an instance of sikuli’s class Screen. This class provides all methods needed to perform the interaction with our application. Let’s look at this in more detail:

public void testGrayScale() throws FindFailed {
    screen.find(getPattern("grayscaleImage.png", 0.9f));
    screen.click(getPattern("originalImage.png", 0.9f));

private Pattern getPattern(String path, float similarity) {
    Pattern p = new Pattern(getFullPath(path));
    return p.similar(similarity);

private String getFullPath(String path) {
    return "src/test/resources/img/" + path;

As sikuli is based on image processing, we can define where to click and what to verify with screenshots we have taken before. In this simple example I have stored all screenshots as png files within the src/test/resources/img folder of our project. More advanced projects may need a more sophisticated folder hierarchy. As you can see, we first wait for the application to show up. Once sikuli has found the first screenshot, we verify that the button “Undo” is disabled. This is done by calling the method find() with an image of the disabled button. Now we can click on the button “Grayscale” (again specified by an image of the button) and then verify that the grayscale version of the image is found on the screen.

Sikuli does not only compare both images pixel by pixel, but if you like it computes the similarity of the found screen region with the requested region. This helps when you need to be more tolerant (e.g. if you want to test the application in different browsers and these render the buttons slightly different). The default value for the similarity attribute is 0.7f, but if you increase it to 1.0f you have a simple pixel by pixel comparison.

But this is not all. With sikuli you can do nearly all things you could do as human interactor:

  • Type characters using screen.type()
  • Double click with screen.doubleClick()
  • Perform drag and drop operations with screen.dragDrop()
  • Use the mouse wheel

Conclusion: Sikuli is a powerful and easy to use tool to perform integration tests for web applications that heavily rely on HTML5’s canvas object. The same is of course true for standard fat client applications (Swing, JavaFX). Together with arquillian you can setup comprehensive test suites that cover a lot of “real” use cases.