AnsweredAssumed Answered

Activiti on Websphere & Spring AsyncJobExcutor

Question asked by ggr on Sep 20, 2016
Context : Make long story short, it has been decided in my company to use Activiti, we have strong knowledge in EMC Workflow Engine. We have deployed it in a hurry in production on IBM Websphere Application Server.
Not enough load/perfomance tests, the inevitable happens : the server crashed.

After some investigations, blog posts & forums readings we found a solutions.

[h2]1. Work Manager et Timer Scheduler[/h2]

Managing your own thread on a WAS is never a good idea. Even if the new class ManagedAsyncJobExecutor use a threads pool of the container this is not suitable for the WAS.
It use work manager - a pool of threads - and managed it by itself.

I re-wrote the existing DefaultAsyncJobExecutor using the spring class org.springframework.core.task.AsyncTaskExecutor, it works like the java java.util.concurrent.ExecutorService.
This interface is mainly implemented by two classes
  • one for a Servlet Container : org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor
  • one for a Applicaton Container : org.springframework.scheduling.commonj.WorkManagerTaskExecutor (needs the jndi name of the work manager)
The AbstractAsyncJobExecutor starts two threads two retrieve jobs to treat. Regarding the code, this is just a runnable snippet that is scheduled.

I rewrite the two classes AcquireAsyncJobsDueRunnable & AcquireTimerJobsRunnable, the common code (99%) has been mutualized in AbstractAcquireJobsRunnable.
I use the time scheduler to avoid Thread.sleep(). I use the Spring implementation because of the WAS, but a standard java one does exist java.util.concurrent.ScheduledExecutorService.
I think this is better to use this than a thread.

The philosophy is still the same :
  • The Async executor starts schedule of execution of the AcquireAsyncJobsDueRunnable & AcquireTimerJobsRunnable
  • When the scheduler is executed, two tasks are executed by the AsyncTaskExecutor - one for each runnable
  • They check the database to retrieve jobs and put runnable task for each job in the AsyncTaskExecutor.
  • if there is no more job to treat or if the queue of the AsyncTaskExecutor is full, the AbstractAcquireJobsRunnable schedule its next execution.
I'm a little bit annoyed with the term AsyncJobExecutor because it does more than just executing job. It rerieves jobs to treat and then only executes them.
It is more a AsyncJobExecutorManager or something like that.

One more strange thing is that AcquireAsyncJobsDueCmd constructor requires a AsyncJobExecutor object. In fact, it only needs lockTimeInMillis and maxJobsPerAcquistions params.

Globally, the rewriting of the classes is more OOP / Spring compliant.

[h2]2. Transaction Manager[/h2]

We use the inappropriate transactionManager : org.springframework.jdbc.datasource.DataSourceTransactionManager.
For WAS, you have to use : org.springframework.transaction.jta.WebSphereUowTransactionManager

[h2]3. Websphere Application Server Config[/h2]

Define a workmanager :
  • name : workmanager/listener
  • Work request queue size : 0
  • Work request queue full action : FAIL
  • Alarm threads : 15
  • Minimum threads : 10
  • Max threads : 200
  • Threads priority : 5
Define a scheduler :
  • name : timer/listener

[h2]4. Activiti 6 and message queue based async executor[/h2]

The mechanism explained below is applicable for Activiti 6 OOTB.

Following this post, we can adapt the mechanism like this based on the producer / consumer pattern.

  • Use a scheduler to schedule tasks to retrieve jobs
  • job are put in a queue (java, jms, whatever.)
  • use a second scheduler to read this queue and put the job in the workmanager
I dont know if it will be more efficient, but it a regulate the load of the executor

Feel Free to ask more details of the solutions.

Thanks for providing such a great product.