richards

Getting started with functional unit testing of Aikau

Blog Post created by richards Employee on Oct 30, 2014

Introduction



With the Aikau framework for Share construction we have created a large number of widgets with some underlying code from Dojo’s UI Library, Dijit. It makes sense to automatically test the Aikau widgets in isolation and therefore we have created a test framework using Intern (http://theintern.io/) to do just that. The tests run quite fast and with predictable results. It is now easy to see if a recent addition or update to an existing widget has caused a bug or regression.



To perform tests we have written a number of test web scripts that contain widget models. The widget models are usually quite simple but occasionally have a more complex structure to expose edge case behaviours. Some test web scripts also contain intentionally broken models to test the catching of errors. For testing purposes the test web scripts are run using Jetty as we do not need any of the more complex overhead from Share.

Process Overview



To run a suite of tests we do the following:



  1. Launch an instance of an Ubuntu virtual machine using Vagrant

    The virtual machine provides a set of test browsers, drivers for those browsers and a running instance of Selenium that Intern can target


  2. Run up the test web scripts using Jetty

    We do not need all of Share so a basic Jetty server is fine here


  3. Run tests against the test web scripts using Intern

    Each of the tests will be run according to the Intern configuration and against the browsers defined and requested therein


Note: One of the tools we install when setting up testing is Grunt (http://gruntjs.com/). Much of the process of running tests is handled for us in commands we have already added using Grunt. In fact, once we have set up the testing framework for the first time, we really only need two commands to start a test run - firstly launch the virtual machine and then secondly run the tests.

Step 1 - Prerequisites



The testing framework for Aikau makes use of a number of technologies and you will need the following free downloads to proceed. Please install with the default settings:

I am assuming that you can use a command line and have a vague idea of what a virtual machine and Vagrant are. If not, please read about it first: http://www.vagrantup.com/

Step 2 - Installation



Having downloaded and installed a current version of the Alfresco code base, open a command prompt and navigate to the following Alfresco directory:

{alfresco}/code/root/projects/slingshot


Run the following command to install Node dependencies:

>> npm install


If you’re interested to know what is being installed you can look at file package.json in the directory above which contains a list of the dependencies. Once you have installed all of the components required for the test framework, you should be able to launch a virtual machine with vagrant for the first time. The first time you run this process it may be slow as it has a lot to download:

>> g vup


Note: This command is ‘g’ for grunt and ‘vup’ for vagrant up.



When the ‘g vup’ command has completed, if it has been successful you should be able to observe the Selenium instance running here:

http://192.168.56.4:4444/wd/hub/static/resource/hub.html



The selenium console should look something like this:



selenium

Step 3 - Running the suite of tests



With an instance of Selenium available on the virtual machine, you should now be able to run Intern by issuing the following command:

>> g test


Note: This command is ‘g’ for grunt and ‘test’ for run intern test suite (virtual machine).



This command first checks if the test application server is running and launches it if not. Once it is happy that the server has started completely it will proceed to launch the Intern test runner. You should see the test suite run through a large number of tests (about 110 tests run twice for two different browsers at time of writing) and log them to the console. Hopefully they will all pass.



This is the sort of output you should expect to see once the initialisation steps have been performed:

...

>> Starting 'AccessibilityMenuTest' on chrome

>> Test page for 'AccessibilityMenuTest' loaded successfully

>> AccessibilityMenuTest: Find the menu element

>> AccessibilityMenuTest: Find the heading text

>> AccessibilityMenuTest: Find the menu items

>> AccessibilityMenuTest: Find the first target

>> AccessibilityMenuTest: Find the second target

>> AccessibilityMenuTest: Find the first menu link - which links the first target

>> AccessibilityMenuTest: Hit the browser with a sequence of different accesskey combinations and the letter 's' for a nav skip

>> Starting 'SemanticWrapperMixinTest' on chrome

>> Test page for 'SemanticWrapperMixinTest' loaded successfully

>> SemanticWrapperMixinTest: Check NO_WRAPPER dom is correct

>> SemanticWrapperMixinTest: Check GOOD_WRAPPER dom is correct

>> SemanticWrapperMixinTest: Check BAD_WRAPPER dom is correct

>> SemanticWrapperMixinTest: Check LEFT_AND_RIGHT_WRAPPER dom is correct

>> Starting 'Pie Chart Test' on chrome

>> Test page for 'Pie Chart Test' loaded successfully

>> Starting 'PublishPayloadMixinTest' on chrome

>> Test page for 'PublishPayloadMixinTest' loaded successfully

...



Note: When a test run is complete the Jetty server is left running. If there are any failures it is possible to immediately investigate if something catastrophic has occurred by simply observing the test web script in a browser.

Useful commands



There are several grunt commands that can be used with Vagrant, the test application server and Intern. Here are the ones we've already seen and a few more, all of which should be run from the directory shown above:



















































































CommandFunction
g vupShort for ‘vagrant up’ this will launch a virtual machine instance with Vagrant
g vdownShort for ‘vagrant down’ this will stop a running instance of a Vagrant virtual machine
g vcleanShort for ‘vagrant clean’ this will delete an existing instance of a Vagrant virtual machine
g testRun up an instance of the test application server and run the Intern test suite against it
g shell:startTestAppStart the test application server
g shell:stopTestAppStop the test application server if it is running
g ntShort for ‘new test’ this command will restart the Vagrant virtual machine, restart the test application server and finally run the Intern test suite against them
g utdShort for ‘update test deployment’ this command will bring down the test application server, rebuild slingshot and then relaunch the test application server with any file modifications that have been made


Adding a test



If you wanted to add a test of your own there are three steps to the process:



  1. Create a test web script


  2. Create an Intern test


  3. Add the test to the Intern Suites.js file


Let's investigate those steps individually:

Create a test web script



Test web scripts all live in this location or a sub-directory of it:

{alfresco}/code/root/projects/slingshot/tests/testApp/WEB-INF/classes/alfresco/site-webscripts/alfresco


Each web script requires three files - a JavaScript file, a Freemarker template file and an xml description file. There are examples in the directory that can be looked at and copied. As with web scripts in the main applications the format of the file names is important. With a correctly written test web script you should be able to view the test model in action at a URL such as:

http://localhost:8089/aikau/page/tp/ws/AccessibilityMenu


This example test web script has in it's model an AccessibilityMenu widget. It isn't very pretty as rendered here but it isn't supposed to be.

Create an Intern test



The actual test files are created here or in a sub-directory of it:

{alfresco}/code/root/projects/slingshot/tests/alfresco


Intern tests are written in JavaScript using a promise-based dependency called Leadfoot which is provided by SitePen, the company who wrote Intern itself. You can read the Leadfoot documentation here. Strategies for writing Selenium tests are complex and I'm not going to investigate them here. Needless-to-say, one emulates the behaviour of an individual using a browser to interrogate the test web script as rendered.



The specific way in which an Intern test addresses a test web script can be seen if any of the existing tests is viewed. Pay close attention to this part of most tests which is the point at which the test web script is loaded:

...

var browser = this.remote;

var testname = 'AccessibilityMenuTest';

return TestCommon.loadTestWebScript(this.remote, '/AccessibilityMenu', testname)

...



Add the test to the Suites.js file



Tests that should be run are all listed in this file in the array 'baseFunctionalSuites':

{alfresco}/code/root/projects/slingshot/tests/config/Suites.js


Next steps



The testing framework supports testing against a local instance of Selenium rather than a VM instance as described above. It is also possible to run a coverage report on the test framework to indicate the amount of the code base being interrogated. The details of these alternative scenarios will follow in another post.

Attachments

Outcomes