Green Moon Software, LLC

Asynchronous Functional Testing

I previously wrote about writing sql tests. Another difficult-to-test scenario are asynchronous processes. If a method-under-test has threading, getting the timing right for the testcase asserts is difficult. Typically, the method will return control to the test class prior to competing the logic that is being tested. The test class then immediately starts processing the asserts. These tests are flaky at best or completely broken at worst.

One way to avoid testing asynchronous scenarios is to simply test the units individually. I am going to take a moment to discuss my view on unit tests.

I am personally not against small unit tests, but my view of them is different than some developers. For me, automated tests are about boosting my confidence in the codebase. However, unit tests are only a small percentage of that confidence. I have encountered many, many bugs in code that was thoroughly unit tested. In my experience, a large number of bugs appear in the communication between the individual units. I usually find there are not enough “integration” or “functional” tests. I personally prefer testing my application at the interface-level; not the implementation. Unit tests that rely heavily on mocks or other tricks are a big concern for me.

So with that in mind, let’s discuss how a technique to test asynchronous process. Please review the following code.

This is a simple use case, but is easy to understand. The method under test accepts a string and a callback function. The threaded logic simply uppers the string and calls the callback with the new value. The Thread.sleep(500) simulates a process taking a little time to complete.

The result of running the above is as follows:

Try running that in GroovyConsole. If you were to comment on the Thread.start{}, the assert would pass and “Success” would print. That’s great, but we need it to pass in a multi-threaded implementation. We can take care of that with a little test helper. Review the AsyncAssert class in the following example.

The run method in AsyncAssert executes the asserts and catches all exceptions. It checks if the timeout has lapsed in the exception handler. If not, then the asserts are ran again. If the timeout has passed, it will throw the exception (typically an AssertionException of some sort). If the asserts pass without an exception, the loop breaks out and the test passes.

So yes, asynchrouns tests will run a little slower than your typical unit tests. For me, the piece of mind that the process works is worth a couple extra seconds. How do some of you test your asynchronous processes? Let me know in the comments.

comments powered by Disqus