Skip navigation
All Places > Alfresco Process Services & Activiti (BPM) > Blog > 2017 > September
2017

I'm proud to announce that we are now officially in Maven Central and Docker Hub. Our first Early Access release 7-201709-EA is out for the community to start experimenting. But what does this mean? Why Early Access? When is the next one coming? This blog post tries to clarify what is included in this Early Access release and when is the next release coming.

Plan, Build and Release Fast

In order to provide short iteration cycles we are planning to publish one of these Early Release every month (we will aim to the last Thursday of each month). In a componentized world, each of the modules will evolve separately and we want to make sure that we build, test and release each of these modules every month. We are making a lot of changes to the Activiti project structure and we introduced Activiti Cloud, for this reason we want to make sure that early adopters looking for cloud native components can get what they are looking for. 

Our release plan is directed by our Roadmap planning, what you can find here: Activiti Roadmap. A new update on this roadmap is coming next Monday (October Update). 

We will continue to improve our release process and also our projects structures to be aligned with the architectural decisions that we are making to enable Activiti to run natively on cloud providers. 

What's included in 7-EA201709?

This release included our first iteration to improve how we handle project dependencies to make our frameworks, services and starters easy to consume. This will also allows us to improve our modularity without affecting people implementations. 

The following two BOM (Bill Of Materials *-dependencies) artefacts were included in this release:

  • org.activiti:activiti-dependencies:7-EA201709
  • org.activiti.cloud:activiti-cloud-dependencies:7-EA201709

This allows you to include <dependencyManagement> tags in your projects to allow these artefacts manage the correct version for both activiti and activiti-cloud artefacts.

 

<dependencyManagement> 
  <dependencies>
    <dependency>
      <groupId>org.activiti</groupId>
      <artifactId>activiti-dependencies</artifactId>
      <version>${version.activiti}</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

 

This release also included our new set of services that provides the core set of functionality for our Cloud Starters. These services now include: 

  • Audit Service
  • Query Service
  • Events enabled endpoints and Event Emmitters
  • Runtime Bundle Services (Stateless & Inmutable Process Engine) 
  • Identity Services and integrations

Cloud Starters

If you are used to work with Spring Boot our Cloud Starters provides a quick way to bootstrap different components in a straight forward way. All these starters are powered with Spring Cloud integration to make sure that they can be configured to leverage the ecosystem that surround them. For example, if there is a Service Registry, these services will be automatically registered to it. If there is an API gateway, these services will be automatically exposed and routes will be created to access them via the Gateway. 

You can find our Cloud Starters here: https://github.com/Activiti/activiti-cloud-starters

Our Cloud Starters are powered by Spring Boot and the following Spring Cloud projects:

  • Spring Cloud Streams
  • Spring Cloud Registry
  • Spring Cloud Gateway
  • Spring Cloud Config (work in progress)
  • Spring Cloud Service Connectors (work in progress)
  • Spring Cloud Contracts (work in progress)
  • Spring Cloud Kubernetes (work in progress)

Docker Images

Using the starters we have created a set of Docker Images that you can quickly bootstrap the whole infrastructure for you. You can use Docker Compose or Kubernetes to run all these services. Helm Charts are in the process for Kubernetes and we, as developers and as the team in charge of the project, have chosen to use MiniKube for our everyday development environment.

Screen Shot 2017-09-29 at 09.12.52 

All our Docker images can be configured using Environment Variables to allow different environments, set ups and flexibility on the deployment phase. 

Git Book

We are constantly working in our new Git Book to explain how all these projects and components work together. We understand that Cloud Environments are much more complicated than using a Java Framework and the amount of technology that we will be using in the future might sounds overwhelming. For that reason, our Git Book tries to tackle the frequently asked questions about our architectural decisions and technology choices. This Git Book is going to be a common place for several pieces of documentation related with all our frameworks, services, cloud starters and how all these pieces fits together. 

You can access our Git Book here: https://www.gitbook.com/book/activiti/activiti-7-developers-guide/details

Examples & Cloud Examples

We are actively working in examples for our projects that you can find here:

We will welcome community feedback and help to improve these examples, which are probably the best way to get involved to the project and learn about the new features.

 

Next Steps

There will be some more repository refactoring during the next month to make sure that each project is independent from each other and we can evolve it separately. We are waiting to review and merge some pull requests before moving these projects to separate repositories. These refactorings will not affect your projects if you start using the BOMs introduced earlier. 

New Cloud Starters will be created and new Docker Images will be published related to:

  • Notification Service
  • Application Service
  • Repository Service 

 

 

Stay tuned!

Questions, Feedback, Comments?? Find us in Gitter: https://gitter.im/Activiti/Activiti7

Last week (18/9/17 - 24/9/17) we spend most of our time hardening our examples and our release process. @ryandawsonuk spent most of his time fighting with scripts for making releases as simple and flexible as possible. @erdemedeiros reviewed and merged @daisuke-yoshimoto Pull Request about supporting Mongo DB as an alternative Audit Service Module.

 

I’ve (@salaboy) spent some time reviewing @igdianov Pull Request for supporting extra GraphQL endpoints in our Query Service Module. This is a great addition and we are still reviewing how is the best way to add these functionality as an optional dependency to our Query Services. I will be interested in hearing from our community how feel about GraphQL. After some tests we can clearly see how UI components can leverage all the power of GraphQL endpoints.

 

This week we will be hardening our Kubernetes deployments while we finish some bits that we want to include in our first Maven Central release. I encourage the community to start looking at our GitBook which is slowly shaping up.

We will be also doing some planning for next month based on our progress, feel free to jump into our Gitter channel if you are interested in learning more about our plans.

Last week (11/9/17 - 18/9/17) we (@ryandawsonuk and myself) made huge progress validating our new cloud architecture with our partners from Pivotal. We attended Pivotal Partner’s day and discussed with their engineers our plans to run Activiti Cloud on top of Kubernetes and leverage their APIs and integrations. As result, we will start testing with Kubo, their approach flavour for Kubernetes which provides integration with their existing CloudFoundry infrastructure. This opens the doors to a myriad of possible integrations with existing apps and their entire marketplace. You will see more about this soon. In the meantime you can give it a try to our Kubernetes Example here.

 

@daisuke-yoshimoto reviewed his PR for Mongo DB and @gmalanga is updating his PR based on @erdemedeiros feedback.

 

At the same tim, @igdianov is working in finishing our first approach to GraphQL so we are looking forward to his PR this week.

 

We are working hard to get our release process sorted out to make sure that you can consume all of our artifacts from Maven Central. This require us a huge effort in setting up several builds and CI pipelines to coordinate all the new repositories and artifacts, you should expect a blog post about this during the week as soon as we finalize some extra details.

 

We are also structuring our GitBook to make sure that we cover the most important topics about Activiti and Activiti Cloud. We will appreciate your feedback, comments and suggestions about which topics should be covered in such book.

For the last two weeks, we have spent most of our time making sure that we can deploy Activiti Cloud components to Kubernetes. We’ve now shown how we can deploy Activiti cloud in the following environments:

  • Docker Compose (Development)
  • MiniKube (Development Environment - but a little bit more real)
  • Kubernetes (hosted in AWS)

The results can be found inside our cloud examples repository, where you will find docker compose and kubernetes descriptors to start these environments (docker / kubernetes directories). The most interesting bit of this deployment is that is composed of separate descriptors, one for the “Infrastructure” and another for each “Runtime Bundle” that we want to deploy/publish. Cloud Connector deployments will come next.

 

This means that in contrast with “normal” kubernetes deployments which are “static”, this deployment needs to deal with new Activiti Cloud Applications (Runtime Bundles, Query, Audit and Cloud Connectors) that will be deployed on demand based on business requirements.

If you remember, from my previous blog post about Activiti Cloud, we have the following components in our environment:

From this diagram, we can see the static components (that can be scaled using a load balancer) and the dynamic Activiti Cloud Applications.

Activiti Cloud Applications and their components will depend on the infrastructure to be running.

For each of these components we have separate Docker Images and some of them might use separate storage (requiring more docker containers for those DBs).

Based on the previous diagram here is the list of our current set of containers, already published for the infrastructure bits in Docker Hub:

  • Activiti-Cloud-Gateway (Zuul implementation)
  • Activiti-Cloud-SSO-IDM (Keycloak implementation)
  • Activiti-Cloud-Registry (Eureka implementation)
  • RabbitMQ  and PostgreSQL are also started as part of the infrastructure

It is important to note that even if the deployment descriptor is “static”, each of these components can be scaled independently.

This infrastructure will enable our Activiti Cloud applications to use common services such as Security and Single Sign On, the Service Registry and the message brokers to produce and consume messages.

Inside this infrastructure we enable our Domain Experts to deploy one or more Activiti Cloud Applications that will contain their business processes. This approach enables the interested parties to have a high level of separation between applications and a self healing network (provided by Kubernetes) that will fix problems when they happen (and they always happen).

Activiti Cloud Applications running in this “static” infrastructure are considered dynamic because:

  • Each application will be different
  • We cannot estimate when these applications are going to be needed
  • We cannot estimate in advance the load of each application
  • We must be able to scale each application separately
  • Each application can have different configurations

The services provided for Activiti Cloud Applications are:

  • Activiti-Cloud-Query (JPA Implementation)
  • Activiti-Cloud-Audit (JPA Implementation)
  • Activiti-Cloud-Nofitications (under development)
  • Activiti-Cloud-Runtime-Bundle (Base Image for your runtime bundles)
  • Activiti-Cloud-Connector (Base Image for your cloud connectors)

Activiti Cloud Applications will use the available infrastructure services to handle Service Lookups, Identity lookups, Single Sign On, and the Message Brokers to emit and receive events from other components.

As mentioned before, each runtime bundle can be configured against different databases or share the same instance if needed. For each application, meaning multiple runtime bundles, you can allocate a set of services that will aggregate information about execution and interaction of these runtime bundles. Some examples of these services are:

  • Query Service: you might want to aggregate all the tasks and process instances from all the Runtime Bundles running inside an application.
  • Audit Service: you might want to aggregate all the audit information of all the runtime bundles
  • Notification Service (under development): you might want to push notifications to all your users related to the changes of state inside your runtime bundles.

Getting Started

The first decision that you will need to make is to choose between running all services with:

  • Docker Compose
  • Kubernetes

The main difference between these two options is the degree of similarities that you will have a with a real production environment in your local setup. I would recommend Minikube if you are planning to run in Kubernetes hosted in a cloud provider. Minikube runs inside a VM, causing the application to be running inside a different instance of an Operating System.

The Docker Compose approach is probably faster, due the fact that it doesn’t require a VM, but you will need to be careful with configurations. Remember that if you are planning to run your applications in a clustered environment, each of the services might run in a different node.  

No matter which option do you choose, from a high level perspective these are the steps that you will need to perform:

  1. Start Infrastructure
  2. New runtime bundle
    1. Configure it
    2. Build it
    3. Deploy it
    4. Repeat 2 for a new Runtime Bundle

Docker Compose

If you choose the Docker Compose approach, you need to install Docker for your OS and Docker Compose will be ready to use in your terminal. (https://docs.docker.com/engine/installation/)

Then you will need to clone the github.com/activiti/activiti-cloud-examples repository or copy the contents of the docker/infrastructure.yml file into your laptop and then run:

> docker-compose -f infrastructure-docker.yml up -d

Before you can start interacting with your services you need to add an entry to your "/etc/hosts" file so the SSO component can sign the verification tokens using the same internal name as the services which are behind the gateway. We have learnt a lot about Zuul (Gateway), Keycloak and how to do SSO with microservices, so expect a blog post about that shortly.

sudo vi /etc/hosts
# add for sso
127.0.0.1       activiti-cloud-sso-idm

This will start up all the infrastructure services + Audit and Query so you can focus on creating your runtime bundles.

Once you have all your infrastructure services started you can create a new Runtime Bundle Docker image with your business processes and required resources by using the project:

https://github.com/Activiti/activiti-cloud-examples/tree/master/docker-runtime-bundle

Which doesn’t require any Java or Maven and it uses our Activiti-Cloud-Runtime-Bundle Docker image as base to just include your business processes included inside the /processes/ directory. In order to build your Runtime Bundle you just build it as any docker image, by running:

docker build -t rb-my-app .

Inside the docker-runtime-bundle directory, which contains the Dockerfile specification to build the image.

If you require more Java magic and customizations, you can use:

https://github.com/Activiti/activiti-cloud-examples/tree/master/maven-runtime-bundle which is a simple project to create runtime bundle Docker Images by using the Fabric8 Docker Maven Plugin and our Activiti-Cloud-Runtime-Starter project. You can build the docker image by running:

mvn clean install

The obvious advantage of using the Java/Maven approach is that you can include unit tests for your processes and customizations.

You need to make sure to tag your docker image accordingly (https://github.com/Activiti/activiti-cloud-examples/blob/master/maven-runtime-bundle/pom.xml#L100 or when building the docker image manually with -t ) so then you can reference it in the rb-docker-compose.yml -> https://github.com/Activiti/activiti-cloud-examples/blob/master/docker/rb-docker-compose.yml#L7

Once you have your Runtime Bundle Docker Image ready you should be able to start it with:

docker-compose -f rb-docker-compose.yml up -d

 

At this point you have the infrastructure and one runtime bundle ready to be used.

To shut everything down run:

docker-compose -f rb-docker-compose.yml down

docker-compose -f infrastructure-docker.yml down

MiniKube / Kubernetes

With Kubernetes we will achieve a more realistic environment to test our services. By using Minikube we will achieve a similar runtime environment as with docker compose, with the big difference that kubernetes will take care to self heal our services in the case of failures.

In order to get the minikube set up and running you need to first install minikube (https://github.com/kubernetes/minikube) and start the cluster with:

minikube start --memory 8000

You can change the memory allocation if you want to, Activiti Cloud with the current services is using around 3GB to bootstrap all the services.

You can access to the MiniKube dashboard running:

minikube dashboard

It should look like this:

And that will give you the MiniKube entry point IP address that you need to my to your /etc/hosts file

sudo vi /etc/hosts
# add for sso
127.0.0.1       activiti-cloud-sso-idm
192.168.64.4    activiti-cloud-sso-idm-kub

For my case the minikube IP address is: 192.168.64.4, but you should replace it for yours.

Once the cluster is ready you can start deploying services to it and you can do that by going to the /kubernetes/ directory inside the activiti-cloud-examples repository and run:

kubectl create -f infrastructure.yml

Look at the Docker Compose section to read more about how to build your Runtime Bundle Docker images, and when you have those ready you can do:

kubectl create -f runtime-bundle.yml

After running these commands you should see something like this in your Kubernetes Dashboard

Take a look at the runtime-bundle.yml (https://github.com/Activiti/activiti-cloud-examples/blob/master/kubernetes/runtime-bundle.yml#L17 ) file for customizations regarding your Runtime Bundle image name and how to configure a Database for it. Notice that we are creating a Single Pod with both Runtime Bundle + PostgreSQL, but this is not a restriction, you can change your deployments to suit your needs.

A couple of caveats regarding this deployment:

  • In minikube you can’t create service of type LoadBalancer, for that reason we are using NodePort. You will need to change this to LoadBalancer in two services: entrypoint and activiti-cloud-sso-idm-kub for a real Kubernetes environment. You will find comments explaining this in the infrastructure.yml deployments
  • In minikube we need to use the NodePorts specified in the infrastructure.yml in order to interact with the Gateway and SSO services. In a real Kubernetes environment you will need to use the Service specified ports intestead.

Now that you have everything up and running, you need to know how to shut it down:

kubectl delete -f runtime-bundle.yml

kubectl delete -f infrastructure.yml

minikube stop

Is it working?

How do you test all these services? You have two options for now, but more are coming, you can use (Chrome) Postman Collection to hit the REST endpoints of these services and check that components such as Query and Audit are receiving events from our Process Executions or you can use our Demo Client App (using Angular JS) that will perform the security Web Flow, redirecting you to Keycloak UI for Single Sign On to all of our services. This simple application allow you to test the different endpoints to check that everything is working there.

(Chrome) Postman Collection

Install Postman Chrome plugin, then download (if you cloned the github repo for the cloud examples you already have this in your local environment) the collection located here: https://github.com/Activiti/activiti-cloud-examples/blob/master/Activiti%20v7%20REST%20API.postman_collection.json

Import it into Postman and then you can execute different request against these services, mostly going through the API Gateway.

Depending on your environment setup you might need to change the IP addresses of the URLs in each request:

Notice that to perform any request you first need to get a Token, which will be added to all the following requests that you make against the services. The token is set to expire after 3 minutes, meaning that you might need to request a new token if you wait for longer than that between interactions.

Once you have the token you can interact with any the other services that you have deployed.

Notice also that based on the name of your runtime bundle the URL for such request might change. The API Gateway will use the name of your Runtime Bundle App to register a new route to it when it is deployed, you will need to adapt that accordingly.

Demo UI Client (Angular JS)

The other alternative, which looks more like building a client App, allows you to run an existing app that uses the Keycloak.js library to secure our rest calls and deal with requesting the token.

You can find this app in the demo-ui-client (https://github.com/Activiti/activiti-cloud-examples/tree/master/demo-ui-client ) directory of the activiti-cloud-examples repository.

The README file in that directory guides you through the steps to execute the application.

Once you started the server you can access to:

http://localhost:3000/#/main

In your browser and you should see something like this:

This is Keycloak kicking in, asking you for your credentials. You can use testuser/password to login and proceed to the application:

Again, here you will need to replace to your environment IP depending what setup are you testing. If you are running with Docker Compose (default URL to localhost) you can go ahead and execute some requests.

This should give you a high level idea about how to interact with these services and how to deploy new Runtime Bundles when you need them.

Sum up

On this very long blog post we have covered how to setup and run the Activiti Cloud Infrastructure and how to get one domain specific runtime bundle up and running. We will build up on the assumption that the infrastructure will provide a common set of services that each of the Activiti Cloud Applications will have available to use. We are also writing a GitBook to make sure that we document these components and these tutorials in a comprehensive way. We are just starting so we will appreciate any feedback or help that you might want to send our way.

Last week (4/9/17 - 10/9/17) we spend a lot of time refining our deployments mechanism to fit both Docker Compose and Kubernetes (and Minikube). We worked with @ryandawsonuk to polish our examples and make sure that you can bootstrap the infrastructure following a finite number of steps :) A blog post about this is coming today.

 

@daisuke-yoshimoto provided a PR for Mongo DB integration for the audit service and

@gmalanga provided a PR for ElasticSearch integration for the query module. Help us to review these PRs!

We got an update form @igdianov which is working our new GraphQL endpoints so we are looking forward to review all these contributions.

Besides that we started adding content in our GitBook: https://activiti.gitbooks.io/activiti-7-developers-guide/content/chapter1.html, feel free to send comments/issues and PR for that repository as well. This GitBook will have a section for each of our components and links to related resources.

This week we (@ryandawsonuk  and myself) are going to be a little bit off (UK working hours) the Gitter Channel, because we are in Seattle WA USA. If you are around and want to code/hangout drop us a message.

Last week (Week 29/8/17 - 3/9/17) was quiet, most of the team is on holidays, but we manage to make some progress with @daisuke-yoshimoto and @gmalanga, where two new implementations for the query(ElasticSearch) and audit(MongoDB) module are in the way and pull requests are going to be reviewed this week.

On my side (@salaboy) I manage to make some very good progress on the kubernetes deployments descriptors and I got our services working in minikube and a real kubernetes implementation with minimal effort. There will be a blog post about this in the next couple of days.

This week, the focus will be in documenting all the new modules, preparing for release to maven central and making sure that the community can run our cloud examples in less than 10 minutes (excluding download times :)

Stay tuned!

After finishing almost two months of work we wanted to update our Roadmap and announce a new Milestone for November. We have made huge progress on Docker, Security and the Kubernetes deployments are almost done. In the next following weeks we will do our first release to maven central (you can consume our monthly snapshots already from Alfresco Nexus repository). We are already releasing snapshots to Docker Hub as well, but we will start tagging these snapshots shortly.

Our previous August update (1/08/17)

Here is a review of our original Milestones

 

Milestone #0 - July 2017 - Ended

  • Clean up Activiti
  • Domain API + HAL API + Runtime Bundle
    • process definitions
    • process instances
    • tasks
    • XML/JSON/SVG for process definitions
  • Audit Service: Event Store for Audit Information
  • Identity Management and SSO (KeyCloak implementation)
  • First Release Process

Milestone #1 - August 2017 - Ended

  • Domain API + HAL API + Runtime Bundle
    • Improvements, refinements and additions
  • Query Service: Event Store for Runtime Information
    • Security Enabled
    • JPA - Reference Implementation
  • Infrastructure Enabled Services
    • Gateway (Zuul)
    • Application Registry (Eureka)
    • SSO & IDM (Keycloak default implementation)
    • All Services are Docker Enabled
    • All Services can be deployed into Kubernetes
  • Cloud Examples

August was all about enabling the Process Engine and other services to work in a cloud setup in collaboration with some infrastructural services. We created our Activiti Cloud Starters to make sure that working in these environments is easy and intuitive.

We also got our initial implementation of the Query Service, that will allow you to consume data about process executions without hitting (and affecting) any process engine runtime. You can find out some examples about how to filter data inside the (Chrome) Postman collection.

You can play with all these services by looking into our activiti-cloud-examples repository, where you will find a set of descriptors to start all our services using Docker Compose and Kubernetes plus a JavaScript application that demonstrate how you can interact with all the services provided. This simple example shows how the security layer (SSO)  kicks in when you want to interact with our services.

Milestone #2 - September 2017 - In Progress

  • Domain API + HAL API + Runtime Bundle
    • Integration Events & Cloud Connectors
  • Release to Maven Central
  • Distributed Notification Service (Design and Initial Implementation)
  • Infrastructure Enabled Services
    • Tracer (Zipkin)
    • ELK Stack support (optional)
  • Activiti Cloud Documentation
  • Validation Examples
    • AWS
    • CloudFoundry

September will be all about refining our current services to run in harmony to make sure that we use the right tool for the right infrastructure. The more environments we support (AWS, CloudFoundry, Kubernetes) the more pieces we need to replace and integrate with. We are looking into replacing Eureka when running on Kubernetes, based on the fact that Kubernetes already provide a service registry.

High priority this month will be the initial implementation of Integration Events producers and consumer (Cloud Connectors) which will enable us to remove the need for classpath extensions and improves the interoperability of our services.

We will aim to have a very simple implementation of our notification service  by the end of the month, to be able to demonstrate how our infrastructure allows you to build reactive and contextual applications.

Support for Zipkin  will be provided to monitor and troubleshoot service to service interactions.

Milestone #3 - October 2017

  • Application Context Service - initial version
    • Provide basic case management constructs
    • Publish/Deploy Runtime Bundle Service
  • Distributed Model Repository Service (Design and Initial Implementation)

Milestone #4 - November 2017

  • New Decision Runtime Design and Initial Implementation
  • Process Engine Clean-ups and refactoring
    • BPMN2 Extensions
    • History Service
    • Job Executor
    • Timers
    • Emails Service

As always, if you want to participate on the development process of some of these components get in touch. We are willing to collaborate with the open community and mentor people that want to learn about the project.

We look forward all community involvement, from comments, concerns, help with documentation, tests and component implementations.

You can always find the more up to date Roadmap in our Github Wiki.

Remember, we are 24/7 in our Gitter Channel, feel free to join us there.

Stay tuned!