AnsweredAssumed Answered

Lots of timer tasks, slow lock acquisition times

Question asked by kirkb75 on Jun 23, 2014
Latest reply on Jul 22, 2014 by jbarrez
Id like to know if this approach is feasible and I'll try to keep this first post simple.

Requirements call for recurring processes which wait and trigger at configurable intervals (weekly, monthly, yearly, etc). 10M+ active processes exist, with 50-100k becoming candidates any given day, each having the same ISO timestamp (current timestamps in legacy system are day precision).

I had some questions about load so I wrote a couple simple tests which yielded these results, which surprised me. This was done using a quad-core i7 with 16GB ram and MySQL 5.6, using simple process with a timer task that calculates the next date and loops and then waits and so on. The quality on this bites but it was the first hosting site I found :)

http://postimg.org/image/vj748msh5/

Candidates  Total Records   Lock Acquisition Time
50k               100k                     ~330ms
50k              1M                           ~4s
50k              10M                   ~2m

Candidates are process instances that have a trigger time that is less than the current time. Total records include active candidates, plus process instances with trigger times in the future. I wrapped the AcquireJobsCmd execution with a timer to obtain these numbers.

I notice no indexes exist on the ACT_RU_JOB table, by adding some indexes on DUEDATE_ and LOCK_EXP_TIME_ I was able to speed up the query dramatically, obtaining ~14s for a 10M record table. But, this may have other undesirable side effects and this is still much slower than I would expect to obtain a lock to execute a process…so this was just an experiment.

In theory, should Activiti be able to handle this better? Or is the approach flawed in the first place? Other options exist, but it's very tempting to have one product do all the queuing and locking work for us ;-) especially since it seems to be integral to the process flow.

Thanks in advance for any input!


Outcomes