Alfresco Monitoring with Prometheus and Grafana

[karthick]

Overview

There are many ways in which the activities and operations performed in Alfresco Content Services can be measured and montiored. In this post, we are going to take a look at montioring Alfresco using monitoring systems such as Prometheus & Grafana. Alfresco repository now exposes an endpoint /alfresco/s/prometheus that is compatible with the Prometheus scarping end point. Data such as JVM Metrics, Database metrics & REST API metrics are currently captured and exposed using the above mentioned endpoint. Prometheus data source can be integrated with the Grafana application where the Alfresco real time activities can be viewed and monitored in a more user-friendly manner.

Configure/Enable Prometheus endpoint metrics in ACS

Though the Prometheus endpoint is available by default in all ACS6.1.x deployments, in order to enable and capture the metrics following properties must be added in ACS alfresco-global.properties file. By default metrics.enabled is set to false, so accessing the Prometheus endpoint/webscript without these changes will return HTTP 404 error.

metrics.enabled=true
metrics.dbMetricsReporter.enabled=true
metrics.dbMetricsReporter.query.enabled=true
metrics.dbMetricsReporter.query.statements.enabled=true
metrics.jvmMetricsReporter.enabled=true
metrics.restMetricsReporter.enabled=true
metrics.restMetricsReporter.path.enabled=true

Once enabled, accessing the endpoint URL will display JVM metrics, DB metrics & REST API metrics related to Alfresco.

# HELP system_cpu_usage The "recent cpu usage" for the whole system
# TYPE system_cpu_usage gauge
system_cpu_usage 0.16914462251399884
# HELP jvm_threads_peak The peak live thread count since the Java virtual machine started or peak was reset
# TYPE jvm_threads_peak gauge
jvm_threads_peak 169.0
# HELP jvm_threads_daemon The current number of live daemon threads
# TYPE jvm_threads_daemon gauge
jvm_threads_daemon 132.0

Install & Configure Prometheus

Depending on the operating system you use, Prometheus can be installed using Distributed installer files, from Source or using Docker. Reference - https://prometheus.io/docs/prometheus/latest/installation/ & https://prometheus.io/docs/prometheus/latest/getting_started/

In this case I am using Mac, so I used Homebrew to install Prometheus.

brew install prometheus

Before starting up the Prometheus instance, you would need to configure the Alfresco metrics path and the appropriate Target URL (ACS Host URL) in prometheus YAML config file (prometheus.yml). You can get a copy of the default YAML file here - YAML Config

Key properties to note are 1) metrics_path & 2) targets in the scrape_configs section. Update the target property based on your ACS instance.

Example:

metrics_path: /alfresco/s/prometheus

targets: ['localhost:8080']

Save the file and then copy the file to /etc/prometheus directory. Once done run the following command to start Prometheus

prometheus --config.file "/etc/prometheus/prometheus.yml"

Monitor the logs and once the service is started, navigate to http://localhost:9090/config (listens to port 9090 by default). The config page should reflect the metrics_path & targets value that was done earlier.

Targets page - http://localhost:9090/targets will display the Alfresco Prometheus endpoint URL. Verify that it is correctly set such as - http://localhost:8080/alfresco/s/prometheus

Install Grafana

brew install grafana

Start Grafana using brew services start grafana and monitor the startup logs. Navigate to Grafana dashboard on http://localhost:3000/

In order to display and monitor the Alfresco metrics in the Grafana Dashboard, following steps are necessary.

1) Create a Promethues data source by clicking on the "Create your first data source" link in the home page. Select Prometheus and Enter http://localhost:9090 in the URL field. If "Basic Authenticaton" is selected, uncheck it and save the data source.

2) Create/Import dashboards to Grafana app and monitor the real-time metrics in Dashboard view. Sample dashboard JSON files are available in - https://github.com/Alfresco/acs-packaging/tree/master/docs/micrometer. Three different JSON files covering DB Metrics, JVM Metrics & REST API metrics are available in GitHub. While Importing the JSON file make sure the correct Promethues data source is selected.

Once the above steps are done, real-time metrics can be seen and monitored in the Grafana dashboards.

References

Thanks to all the below references 

https://github.com/Alfresco/acs-packaging/tree/master/docs/micrometer

https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus.yml

https://www.prometheusbook.com/MonitoringWithPrometheus_sample.pdf

[astrachan · Accepted solution]

Great article Karthick - just to perhaps help those who (like me initially) are struggling to figure out what to put into the Prometheus configuration (.yml) file, here is what I have:

- job_name: 'acs'

  metrics_path: /alfresco/s/prometheus

  static_configs:

  - targets:

    - localhost:8080

Additionally, to create a  datasource in Grafana via a RESTful call for this example
POST http://localhost:3000/api/datasources

BODY (json)
{
"name": "DS_PROMETHEUS",
"type": "prometheus",
"url": "http://localhost:9090",
"access": "proxy",
"org_id": 1,
"is_default": true,
"version": 1,
"editable": true
}

[karthick]

Thanks Alex for the additional information. It will be really helpful for the users who are trying this out.

[EddieMay]

Hi @astrachan 

Thanks for the extra info - really helpful!

Cheers,

[zayzou]

Hi,greet article

does Prometheus   integrated in community edition please ?

[EddieMay]

Hi @zayzou 

You might find the recent Alfresco Tech Talk Live on this subject useful.

HTH,

[grathod]

Hi Karthick,

Thanks for the details, I followed your article

For my case I installed both "prometheus and grafana" on ubuntu-22 host.

kept firewall off in both.

so Prometheus and Grafana on one VM host: http://192.168.29.187:9090/

alfresco on other VM host: http://192.168.29.152:8080/alfresco/

step 1) installed prometheus successfully on port 9090

set below values in alfresco-global.properties file

metrics.enabled=true
metrics.dbMetricsReporter.enabled=true
metrics.dbMetricsReporter.query.enabled=true
metrics.dbMetricsReporter.query.statements.enabled=true
metrics.jvmMetricsReporter.enabled=true
metrics.restMetricsReporter.enabled=true
metrics.restMetricsReporter.path.enabled=true

and my for prometheus 

prometheus.yml

- job_name: 'prometheus-alfresco'
  metrics_path: /alfresco/s/prometheus
  static_configs:
    - targets:
       - 192.168.29.152:8080

Now when I check at prometheus dashboard targets

http://192.168.29.187:9090/targets

it gives me endpoint for alfresco but showing down always, even alfresco is running on 8080 with added properties.

End point looks like

 I am new to both systems, so don't know , how do I get alfresco stats plus infrastructure stats into grafana visual dashboard.

And how do I import or see in Grafana, that also don't know

I want to monitor CPU / RAM / Network / Alfresco Load / JVM etc.. 

I installed node exporter as well. but not sure how do I use it here, please help.

Thanks,

Ghanshyam R

[angelborroy]

This feature is only available for Alfresco Enterprise.