Archive | November 2013

Building and testing a websocket server with undertow

The upcoming version of JBoss Application Server will no longer use Tomcat as integrated webserver, but will replace it with undertow. The architecture of undertow is based on handlers that can be added dynamically via a Builder API to the server. This approach is similar to the way of constructing a webserver in Node.js. It allows developers to embed the undertow webserver easily into their applications. As the addition of features is done via the Builder API, one can only add the features that are really required in one’s application. Beyond that undertow supports WebSockets and the Servlet API in version 3.1. It can be run as blocking or non-blocking server and it is said, that first tests have proven that undertow is the fastest webserver written in Java.

As all of this sounds very promising, so let’s try to set up a simple websocket server. As usual we start by creating a simple java project and add the undertow maven dependency:

<dependency>
  <groupId>io.undertow</groupId>
  <artifactId>undertow-core</artifactId>
  <version>1.0.0.Beta20</version>
</dependency>

With undertow’s Builder API our buildAndStartServer() method looks like this:

public void buildAndStartServer(int port, String host) {
    server = Undertow.builder()
            .addListener(port, host)
            .setHandler(getWebSocketHandler())
            .build();
    server.start();
}

We just add a listener that specifies the port and host to listen for incoming connections and afterwards add a websocket handler. As the websocket handler code is a little bit more comprehensive, I have put it into its own method:

private PathHandler getWebSocketHandler() {
    return path().addPath("/websocket", websocket(new WebSocketConnectionCallback() {
        @Override
        public void onConnect(WebSocketHttpExchange exchange, WebSocketChannel channel) {
            channel.getReceiveSetter().set(new AbstractReceiveListener() {
                @Override
                protected void onFullTextMessage(WebSocketChannel channel, BufferedTextMessage message) {
                    String data = message.getData();
                    lastReceivedMessage = data;
                    LOGGER.info("Received data: "+data);
                    WebSockets.sendText(data, channel, null);
                }
            });
            channel.resumeReceives();
        }
    }))
    .addPath("/", resource(new ClassPathResourceManager(WebSocketServer.class.getClassLoader(), WebSocketServer.class.getPackage()))
            .addWelcomeFiles("index.html"));
}

Let’s go line by line through this code snippet. First of all we add a new path: /websocket. The second argument of the addPath() methods lets us specify what kind of protocol we want to use for this path. In our case we create a new WebSocket. The anonymous implementation has a onConnect() method in which we set an implementation of AbstractReceiveListener. Here we have a convenient method onFullTextMessage() that is called when a client has sent us a text message. A call of getData() fetches the actual message we have received. In this simple example we just echo this string back to client to validate that the roundtrip from the client to server and back works.

To perform some simple manual tests we also add a second resource under the path / which serves some static HTML and JavaScript files. The directory that contains these files is given as an instance of ClassPathResourceManager. The call of addWelcomeFiles() tells undertow which file to server when the client asks for the path /.

The index.html looks like this:

</pre>
<html>
<head><title>Web Socket Test</title></head>
<body>
  <script src="jquery-2.0.3.min.js"></script>
  <script src="jquery.gracefulWebSocket.js"></script>
  <script src="websocket.js"></script>
  <form onsubmit="return false;">
    <input type="text" name="message" value="Hello, World!"/>
    <input type="button" value="Send Web Socket Data" onclick="send(this.form.message.value)"/>
  </form>
  <div id="output"></div>
</body>
</html>
<pre>

Our JavaScript code is swapped out to the websocket.js file. We use jquery and the jquery-Plugin gracefulWebSocket to ease the client side development:

var ws = $.gracefulWebSocket("ws://127.0.0.1:8080/websocket");
ws.onmessage = function(event) {
    var messageFromServer = event.data;
    $('#output').append('Received: '+messageFromServer+'');
}

function send(message) {
    ws.send(message);
}

After having created a WebSocket object by calling $.gracefulWebSocket() we can register a callback function for incoming messages. In this method we only append the message string to the DOM of the page. The send() method is just a call to gracefulWebSocket’s send() method.

When we now start our application and open the URL http://127.0.0.1:8080/ in our webbrowser we see the following page:
undertow-websocket
Entering some string and hitting the “Send Web Socket Data” button sends the message to the server, which in response echos it back to the client.

Now that we know that everything works as expected, we want to protect our code against regression with a junit test case. As a websocket client I have chosen the library jetty-websocket:

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-websocket</artifactId>
    <version>8.1.0.RC5</version>
    <scope>test</scope>
</dependency>

In the test case we build and start the websocket server to open a new connection to the websocket port. The WebSocket implementation of jetty-websocket allows us to implement two callback methods for the open and close events. Within the open callback we send the test message to the client. The rest of the code waits for the connection to be established, closes it and asserts that the server has received the message:

@Test
public void testStartAndBuild() throws Exception {
    subject = new WebSocketServer();
    subject.buildAndStartServer(8080, "127.0.0.1");
    WebSocketClient client = new WebSocketClient();
    Future connectionFuture = client.open(new URI("ws://localhost:8080/websocket"), new WebSocket() {
        @Override
        public void onOpen(Connection connection) {
            LOGGER.info("onOpen");
            try {
                connection.sendMessage("TestMessage");
            } catch (IOException e) {
                LOGGER.error("Failed to send message: "+e.getMessage(), e);
            }
        }
        @Override
        public void onClose(int i, String s) {
            LOGGER.info("onClose");
        }
    });
    WebSocket.Connection connection = connectionFuture.get(2, TimeUnit.SECONDS);
    assertThat(connection, is(notNullValue()));
    connection.close();
    subject.stopServer();
    Thread.sleep(1000);
    assertThat(subject.lastReceivedMessage, is("TestMessage"));
}

As usual you can find the source code on github.

Conclusion: Undertow’s Builder API makes it easy to construct a websocket server and in general an embedded webserver that fits your needs. This also eases automatic testing as you do not need any specific maven plugin that starts and stops your server before and after your integration tests. Beyond that the jquery plugin jquery-graceful-websocket lets you send and receive messages over websockets with only a few lines of code.

Analyze package dependencies with structure101

One key to a stable application is a well-structured codebase. We know that we should build as many black boxes as possible, because as soon as one black box is finished, we no longer have to think about its interior. You just use the code you or another team member has written through a well-defined interface. This gives you the possibility to concentrate on the next feature you want to add.

When we think about black boxes we often have classes or whole jar packages in mind. Classes should of course be black boxes, no discussion about that. The same is true for jar packages. But in-between classes and jar packages there is another level of structure, which is often not seen this directly as a black box: packages.

Packages are often second class citizens and their interrelationship is not analyzed this thoroughly. But there is a great tool for such analysis: Structure101. It in general helps you to monitor and verify the dependency structures and complexity of your project by the means of well-organized diagrams.

So let’s start with a sample project. I have taken one of my own projects for this: japicmp is a tool to compute the differences between the API of two jar archives in means of what methods and classes have changed. structure101 has a great composition view, which shows you the dependencies between the packages of a project. This is how it looks for the current version of japicmp:

Image

 

Cleary we can see for example that the cli package, which is responsible for the command-line parsing, uses the exception as well as the config package and is used itself by the main package, where the main() method resides. With the cli package everything seems to be OK. But what about the three packages cmp, util and model. The difference computation between the classes and methods, i.e. the business logic, resides in the package cmp. Hence it should use the model as well as the util package. But these two packages should not have any backward dependencies. This problem is also shown in the matrix view:

Image

 

When we take a closer look at the tangle between these three packages, we see that the class AccessModifier, which is located in the cmp package, is used from the util package:

Image

 

Beyond that, this class is also used in the model. This clearly indicates that the class should rather stay in the model package as in the cmp package. This seems to make sense, as the access modifiers of a class or method are part of the model of a jar archive and do not belong into the business logic. If we move this class to the model package, we get the following result:

Image

This looks much better. We do not have any tangles within the package structure. The nice layout also shows clearly that the whole application depends on the model, as the package is located at the bottom of the diagram. The business logic, which resides within cmp, is called from the main package and uses util, config and the model, as it should be. The same is true for the output package, where the implementations for the cli and xml output reside. This package uses the config as well as the model, once it is computed.

Conclusion: Packages should not be second class citizens but help you to structure your application such that it is easy to overview the code and separate functionality. Tools like structure101 help you to analyze the dependencies between packages and therefore let packages be an important level between the jar on the one side and the class on the other side.

Testing HTML5 canvas applications with sikuli and arquillian

HTML5 introduces a great new element that can be used to draw arbitrary content on a pane: the canvas element. What has been a standard feature for fat client applications for decades is now introduced to the world of web applications. Web developers no longer need to use proprietary plugins to draw images or charts in their applications.

But when it comes for testing, this new feature imposes new challenges to the web development community. How to test that the canvas element is in an appropriate state at some point in time? Standard technologies like selenium focus on the markup that is generated by the web server and not on the pixels drawn on the canvas.

More promising in this field are technologies that use image processing to verify that an application renders its data correctly. One of these frameworks is sikuli. Sikuli is an open-source research project that was started at the MIT and is now maintained by Raimund Hocke.

To give a more practical introduction, let’s assume we have a simple web application that uses the HTML5 canvas element to implement some simple image processing functionality like a grayscale, a brighten and a threshold filter as well as an undo button (the code for this application can be found as usual on github):
html5-canvas-screenshot

The installation of sikuli is (of course) platform dependent. The installer, which can be downloaded from the sikuli download page, is a Java Swing application that asks you about your typical usage pattern. As we do not want to use the python IDE, we choose option 4 from the list of options. The actual jar file is then downloaded and prepared for our OS. After the installation process has finished, we find an OS dependent jar file within the installation directory. As our example project uses maven as build system, we have to introduce a system scope dependency after we have copied the library into the lib folder:

<dependency>
    <groupId>org.sikuli</groupId>
    <artifactId>sikuli</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>${basedir}/lib/sikuli-java.jar</systemPath>
</dependency>

When sikuli is used for the first time, it extracts some native libraries into a new folder, here in our example into ${basedir}/lib/libs. This folder has to be added to the user’s path environment variable.

Now that we have installed sikuli, let’s setup arquillian so that we can write our first unit test. How to setup arquillian is described for example here. As I don’t want to repeat everything, in the following you will find only the unit test class:

@RunWith(Arquillian.class)
public class FilterTest {
    public static final String WEBAPP_SRC = "src/main/webapp";
    @ArquillianResource
    URL deploymentURL;
    private Screen screen;

    @Before
    public void before() throws URISyntaxException, IOException {
        screen = new Screen();
        if (Desktop.isDesktopSupported()) {
            Desktop.getDesktop().browse(deploymentURL.toURI());
        } else {
            fail();
        }
    }

    @Deployment
    public static WebArchive createDeployment() {
        return ShrinkWrap.create(WebArchive.class, "html5-sikuli-webapp.war")
                .addClasses(HomeBackingBean.class)
                .addAsWebResource(new File(WEBAPP_SRC, "home.xhtml"))
                .addAsWebResource(new File(WEBAPP_SRC, "resources/css/style.css"), "resources/css/style.css")
                .addAsWebResource(new File(WEBAPP_SRC, "resources/images/rom.jpg"), "resources/images/rom.jpg")
                .addAsWebResource(new File(WEBAPP_SRC, "resources/js/html5Sikuli.js"), "resources/js/html5Sikuli.js")
                .addAsWebResource(new File(WEBAPP_SRC, "resources/js/jquery-2.0.3.js"), "resources/js/jquery-2.0.3.js")
                .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml")
                .setWebXML(new File(WEBAPP_SRC, "WEB-INF/web.xml"));
    }

The createDeployment() method sets up the war archive, which is deployed by arquillian to JBoss AS 7.1.1.Final (see arquillian.xml file). In our @Before method we use the SDK class Desktop to open the default browser and point it to the deployment URL. Here we also create an instance of sikuli’s class Screen. This class provides all methods needed to perform the interaction with our application. Let’s look at this in more detail:

@Test
@RunAsClient
public void testGrayScale() throws FindFailed {
    screen.wait(getFullPath("originalImage.png"));
    screen.find(getFullPath("btnUndo_disabled.png"));
    screen.click(getFullPath("btnGrayscale.png"));
    screen.find(getPattern("grayscaleImage.png", 0.9f));
    screen.click(getFullPath("btnUndo_enabled.png"));
    screen.click(getPattern("originalImage.png", 0.9f));
}

private Pattern getPattern(String path, float similarity) {
    Pattern p = new Pattern(getFullPath(path));
    return p.similar(similarity);
}

private String getFullPath(String path) {
    return "src/test/resources/img/" + path;
}

As sikuli is based on image processing, we can define where to click and what to verify with screenshots we have taken before. In this simple example I have stored all screenshots as png files within the src/test/resources/img folder of our project. More advanced projects may need a more sophisticated folder hierarchy. As you can see, we first wait for the application to show up. Once sikuli has found the first screenshot, we verify that the button “Undo” is disabled. This is done by calling the method find() with an image of the disabled button. Now we can click on the button “Grayscale” (again specified by an image of the button) and then verify that the grayscale version of the image is found on the screen.

Sikuli does not only compare both images pixel by pixel, but if you like it computes the similarity of the found screen region with the requested region. This helps when you need to be more tolerant (e.g. if you want to test the application in different browsers and these render the buttons slightly different). The default value for the similarity attribute is 0.7f, but if you increase it to 1.0f you have a simple pixel by pixel comparison.

But this is not all. With sikuli you can do nearly all things you could do as human interactor:

  • Type characters using screen.type()
  • Double click with screen.doubleClick()
  • Perform drag and drop operations with screen.dragDrop()
  • Use the mouse wheel

Conclusion: Sikuli is a powerful and easy to use tool to perform integration tests for web applications that heavily rely on HTML5’s canvas object. The same is of course true for standard fat client applications (Swing, JavaFX). Together with arquillian you can setup comprehensive test suites that cover a lot of “real” use cases.