Benchmark Framework 2.0: Developing Tests

Document created by derek Employee on Mar 17, 2015Last modified by frank.becker on Apr 24, 2017
Version 2Show Document
  • View in full screen mode

Please see also alfresco-benchmark/Alfresco Benchmark Toolkit 2.0 - Load and Scalability testing - Sep 2015.pdf at master · AlfrescoBenc… for a detailed introduction!


Building and Running Tests


The sample application is packaged as a Maven archetype.
Create a working skeleton project:

 mvn org.apache.maven.plugins:maven-archetype-plugin:2.4:generate -DarchetypeCatalog=
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : org.alfresco:alfresco-benchmark-sample-archetype
Choose archetype:
1: -> org.alfresco:alfresco-benchmark-sample-archetype (-)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Choose org.alfresco:alfresco-benchmark-sample-archetype version:
1: 2.0.0
6: 2.0.5
Choose a number: 6: <ENTER>
Define value for property 'groupId': : com.example
Define value for property 'artifactId': : mytest
Define value for property 'version':  1.0-SNAPSHOT: :
Define value for property 'package':  com.example: :
[INFO] Using property: junit-version = 4.11
[INFO] Using property: server-version = 2.0.5 (Whatever was chosen)
Confirm properties configuration:
groupId: com.example
artifactId: mytest
version: 1.0-SNAPSHOT
package: com.example
junit-version: 4.11
server-version: 2.0.5 (or what was chosen)
  Y: : <ENTER>
cd mytest
mvn clean install


The test code uses an embedded MongoDB instance for the duration of the test.  Much of the debug output is associated with that and will not occur when running against a pre-installed instance.


Running using the UI


In production, the test will be deployed once and controlled by a central Alfresco Benchmark Server UI.  For local testing, start the server application in a dedicated Tomcat instance using Maven.
The next steps assume that a MongoDB instance is available for testing.

 cd ..
git clone
cd alfresco-benchmark
cd server
mvn tomcat7:run


The server will be running in a Tomcat 7 instance of port 9080.

Make sure that the test, when run in its own dedicated Tomcat, does not clash with the server Tomcat port, edit the test's 'pom.xml file and assign a unique Tomcat port for local testing:



This is just to avoid collisions when multiple dedicated Tomcat instances are run locally.

Ensure you have MongoDB running locally (or on some readily-available development server).


Depending on the test, additional software may be required.


Run the server in a dedicated Tomcat instance:

 cd ..
cd ..
cd mytest
mvn tomcat7:run


Connect to the server using



Create a Test and set the MongoDB test database host property as well as any other defaults required.
Create Test Runs and execute.




The sample application is designed to allow experimentation with execution before moving on to actually doing useful load testing.  For more complete and complex examples of tests maintained by Alfresco, see some of the tests used by Alfresco for load testing and benchmarking.
Note: Tests starting with bm- use an old framework and should be ignored.




A benchmark is all about events. All timings provided in the test results are about how long it took to process an event. Events can be handled with success or failure.




EventProcessors do the real test work. If there is an action that should be timed, you should write it inside an EventProcessor. The results of these actions will be stored in the MongoDB and in the CSV and Excel output of your test.

When a test is started, a 'start' event is launched. This event should always be processed by an EventProcessor.




EventProcessors return eventresults. This allows you to specify if the processing of the event succeeded or failed. It also allows you to store additional data in MongoDB, to be retrieved afterwards.
An EventResult also contains a list of next events. This allows you to trigger next EventProcessors or EventProducers.




EventProducers also take incoming events, but they should not do any real test work. EventProducers are not timed, and no results are recorded. They can be used to redirect an event to another event with a delay, or anything else related to the test setup, not to the test work.




Back: Benchmark Framework 2.0


Next: Deploying Tests