Skip navigation
All Places > Alfresco Content Services (ECM) > Blog > 2019 > May
2019

Project Objective

(Link to the project in Github)

Note: the most up to date information about the project can be found in the github page (link above).

 

In this blog we cover the deployment of ACS (Alfresco Content Services) Enterprise 6.1 on Azure using VMs. The deployment consists of two mayor steps:

  1. Creating Azure Managed Images containing a base installation of the corresponding Alfresco component (Alfresco Content Services, Alfresco Search Services...).

  2. Building an Azure infrastructure (Load balancers, VMs, networks, public ips, etc) and deploying Alfresco Enterprise 6.1 to it.

 

The final ACS architecture looks like this:

Alfresco in Azure architecture

Disclaimer

 

The tools and code used in this blog to deploy ACS Enterprise 6.1 are not officially supported by Alfresco. They are used as a POC to show you how you can use OpenSource tools to deploy and configure resources and applications to Microsoft Azure in an automated way.

 

The software used to build and deploy ACS Enterprise 6.1 is available in a public repository in GitHub.

 

Software requirements

The following software is required to create and deploy resources in Azure:

(Mac Os X versions)

 

  • Packer  1.4.0 - used to automate the creation of Azure Images.

  • Terraform 0.11.13-  used to create and update infrastructure resources.

  • Ansible 2.7.6 - Ansible is an IT automation tool used to deploy and configure systems.
  • OpenSSL 1.0.2f - used to decrypt credentials stored in a file.
  • Azure-cli 2.0.55 - used to create and check resources in Azure.

 

Make sure these tools have been installed on the computer you are using to build and deploy ACS Enterprise 6.1.

 

How to run the project

 

The project contains a bash script called run-azure.sh. The script will do the following:

  1. Create the managed Images in Azure.
  2. Build the infrastructure and deploy and configure Alfresco Enterprise 6.1.

 

However, before running the script, the Azure and Nexus credentials should be stored encrypted in a file so the script can use the credentials. 

 

Instructions about how to store the credentials and run the project are described below.

 

1. Authenticating to Azure

 

Azure-cli, Packer and Terraform need to authenticate to Azure in order to create resources. Since we are creating a large number of infrastructure resources, the user we authenticate with to Azure, needs to have administrator access.

 

In order to authenticate in Azure we need the following information:

  • Subscription ID
  • Tenant ID
  • Client ID
  • Client Secret

 

The following links provide information to get the above values:

 

Once you gathered all the credentials you should store them encrypted in a file so the values can be accessed by the run-azure.sh script:

$ cat > ~/.az <<EOF
{
"client_id": "XXXXXX",
"client_secret": "XXXXXXXX",
"subscription_id": "XXXXXX",
"tenant_id": "XXXXXXXX"
}
EOF
$ openssl enc -aes-256-cbc -in ~/.az -out ~/.az.dat

$ chmod 400 ~/.az.dat
$ rm ~/.az

 

2. Authenticating to Nexus

 

Nexus is a repository manager used by Alfresco to publish software artifacts. Ansible will connect to Nexus repository to download the necessary software to install ACS Enterprise 6.1.
Alfresco Enterprise customers have Nexus credentials as part of their license subscription. Please refer to your CRM if you don't know or have your Nexus credentials.
In order to store your Nexus credentials in an encrypted file to use them in this project do the following:

$ echo "NEXUS_USER:NEXUS_PASSWORD" > ~/.nexus
$ openssl enc -aes-256-cbc -in ~/.nexus -out ~/.nexus.dat

$ chmod 400 ~/.nexus.dat
$ rm ~/.nexus

 

3. Set the correct values in run-azure script

 

There are some variables that should be configured before running the script:

 

  • NEXUS_FILE - location of the encrypted file containing the Nexus credentials. 
  • AZURE_FILE - location of the encrypted file containing the Azure credentials
  • DOMAIN_NAME_LABEL - domain name that will be used in the url to access the infrastructure. The format will be {DOMAIN_NAME_LABEL}-{UNIQ_ID}.{LOCATION}.cloudapp.azure.com. Example: 

    alfresco61-demo-02e16d30.uksouth.cloudapp.azure.com

  • RESOURCE_GROUP_NAME_IMG - name of the Azure resource group where the managed images will be created.
  • RESOURCE_GROUP_NAME_DEPLOY - name of the Azure resource group where the infrastructure will be deployed.
  • LOCATION - name of the location used in Azure for the project. To get a list of locations use the following command:
az account list-locations


4. Download the Digital Workspace WAR file:

 

Download the "Digital workspace" war file from here and save it as packer-files/downloaded/digital-workspace.war (this file is not yet available in Nexus for an automated download).

 

5. Run the project script

 

Make sure you have execution permission in the script and then execute it:

$ ./run-azure.sh

~~~~~~~~~~~~~~~~~~~~~

M A I N - M E N U

~~~~~~~~~~~~~~~~~~~~~

 

1. Run the entire project (create Azure Images, build infrastructure and deploy Alfresco Enterprise 6.1)

2. Create Azure Images

3. Build Azure infrastructure and deploy Alfresco Enterprise 6.1

4. Show infrastructure IPs and DNS names

 

Enter choice [ 1 - 4] 

Select option 1 if you want to run the entire project or option 2 followed by option 3 if you want to run it on stages. The last option will show the IPs and DNS names generated in case you need them again (the information is shown as well in options 1 and 3).

 

What happens when running the project

 

Creating managed Images in Azure

 

The first stage of the project consists in creating the following managed images:

 

  1. Alfresco Frontend Image - containing Alfresco Repository, Share, ADW (Alfresco Digital Workspace) and Apache Zeppelin.
  2. Insight Engine Image - containing Alfresco Search Services.
  3. Transform Service Image - containing the new Alfresco Transform Service.
  4. ActiveMQ Image - containing Apache ActiveMQ.
  5. DB Image - containing a base installation of MySQL Server.

 

The run-azure.sh script calls packer to create every image individually. A CentOS 7.6 distribution image is used by packer as a baseline, and after uploading the required files (from packer-files folder), ansible is called for provisioning the image. In the provisioning phase, ansible downloads from Nexus and other repositories the necessary "artifacts" and installs them.

 

As the run-azure script runs you can see what is is doing during its execution...

 

#############################################################

Creating Images

#############################################################

.....

#####################################

Creating image AlfrescoFrontendImage

#####################################

 

azure-arm output will be in this color.

 

==> azure-arm: Running builder ...

==> azure-arm: Getting tokens using client secret

    azure-arm: Creating Azure Resource Manager (ARM) client ...

....

==> azure-arm: Waiting for SSH to become available...

==> azure-arm: Connected to SSH!

==> azure-arm: Provisioning with shell script: /var/folders/z3/qv3400px0ys93js187p3fwwh0000gp/T/packer-shell046700839

.....

==> azure-arm: Uploading packer-files/downloaded/digital-workspace.war => /var/tmp/ansible/downloaded/digital-workspace.war

digital-workspace.war 2.81 MiB / 2.81 MiB [===============================================================================================================================================] 100.00% 0s

....

==> azure-arm: Provisioning with Ansible...

....

And the time elapsed for every image created:

##################

16 minutes and 37 seconds elapsed for AlfrescoFrontendImage image creation.

##################

Building Azure infrastructure and deploying Alfresco

 

Once the images are created it's time to build the infrastructure using Terraform. The file cluster.tf contains the terraform configuration for the Azure and ACS deployments. Additionally cluster_variables.tf contains some specific values (mostly private static ips) required in the deployment.

 

The output of the run-azure script when creating the infrastructure looks like this:

#############################################################

Deploying bastion

#############################################################

 

An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  + create

 

Terraform will perform the following actions:

 

  + azurerm_network_interface.bastionnic

      id:                                                                    <computed>

      applied_dns_servers.#:                                                 <computed>

      dns_servers.#:                                                         <computed>

      enable_accelerated_networking:                                         "false"

      enable_ip_forwarding:                                                  "false"

      internal_dns_name_label:                                               <computed>

      internal_fqdn:                                                         <computed>

   ...

 

azurerm_virtual_machine.bastionvm: Provisioning with 'ansible'...

azurerm_virtual_machine.bastionvm (ansible): Connecting to remote host via SSH...

azurerm_virtual_machine.bastionvm (ansible):   Host: 51.XXX.XXX.10

azurerm_virtual_machine.bastionvm (ansible):   User: azureuser

azurerm_virtual_machine.bastionvm (ansible):   Password: false

azurerm_virtual_machine.bastionvm (ansible):   Private key: true

azurerm_virtual_machine.bastionvm (ansible):   SSH Agent: true

azurerm_virtual_machine.bastionvm (ansible):   Checking Host Key: false

azurerm_virtual_machine.bastionvm (ansible): Connected!

 

And the time elapsed for deploying the infrastructure:

##################

17 minutes and 5 seconds elapsed for entire infrastructure deployment.

##################

Connecting to the infrastructure 

 

This is the screen shown when the infrastructure has been created (or when selecting the option 4 in the menu):

 

#############################################################

DNS NAMES AND IPS GENERATED

#############################################################

 

VM created on resource group "Alfresco_deployment"

 

Alfresco Content Services 6.1 (share) -----------> http://alfresco61-demo-4XXXXXX4.uksouth.cloudapp.azure.com/share

Alfresco Content Services 6.1 (admin console) ---> http://alfresco61-demo-4XXXXXX4.uksouth.cloudapp.azure.com/alfresco/s/enterprise/admin/admin-systemsummary

Digital Workspace -------------------------------> http://alfresco61-demo-4XXXXXX4.uksouth.cloudapp.azure.com/digital-workspace

Alfresco Insight Engine 1.0.0 -------------------> http://alfresco61-demo-4XXXXXX4.uksouth.cloudapp.azure.com/zeppelin

 

Via Bastion urls:

-----------------

Alfresco Search Services 1.2.0 ------------------> http://51.XX.XX.81:8983

 

IPs:

----

Application gateway IP: 52.XX.XX.250

Bastion ip (ssh): 51.XX.XX.81

 

Through the bastion node it's possible to connect to the VMs inside the private network. As follows there's a table containing all the endpoints reachable from the bastion IP:

 

EndpointNAT PortReal portVM
SSH2222bastion
ActiveMQ console81618161activemq-0
ActiveMQ console81628161activemq-1
Solr admin console89838983insight-0
Solr admin console89848983insight-1
Libreoffice transformer80908090transformation-0

PDF-renderer

transformer

80918091transformation-0
Imagemagick transformer80928092transformation-0
Tika transformer80938093transformation-0
Shared File Store (console)80948094transformation-0
Transform router80958095transformation-0
Libreoffice transformer81908090transformation-1

PDF-renderer

transformer

81918091transformation-1
Imagemagick transformer81928092transformation-1
Tika transformer81938093transformation-1
Shared File Store (console)81948094transformation-1
Transform router81958095transformation-1
SSH220022activemq-0
SSH220122activemq-0
SSH220222db
SSH220322frontend-0
SSH220422frontend-1
SSH220522insight-0
SSH220622insight-1
SSH220722transformation-0
SSH220822transformation-1
JMX5050050500frontend-0
JMX5050150500frontend-1

 

To do list

 

  • Secure the infrastructure using https and fine tune other security aspects.
  • Capability to integrate custom amps when creating the infrastructure.
  • Use the customer Alfresco license when deploying Alfresco. (Currently the license installed is a trial license of 2 days however a new license can be uploaded through the admin console).
  • Use autoscaling for the VMs
  • Use Azure SQL DB instead of a dedicated VM with MySQL.
  • Many other things ;-)

 

 

Project Objective

In this blog we cover the deployment of ACS (Alfresco Content Services) 6.1 Enterprise on AWS. The deployment consists of two mayor steps:

  1. Building AWS AMIs (Amazon Machine Images) containing a base installation of ACS 6.1 Enterprise.
  2. Building AWS infrastructure (VPC, ELB, RDS, etc) and deploying ACS 6.1 Enterprise to it.

 

Please make sure your Alfresco license subscription entitles you to install and run ACS 6.1 Enterprise and Alfresco Search Services with Insight Engine.

 

The final ACS architecture looks like this:

 

Disclaimer

The tools and code used in this blog to deploy ACS 6.1Enterprise are not officially supported by Alfresco. They are used as a POC to show you how you can use OpenSource tools to deploy and configure resources and applications to AWS in an automated way.

 

The software used to build and deploy ACS 6.1 Enterprise is available in a public repository in GitHub.

 

Software requirements

The following software is required to create and deploy resources in AWS:

  • Packer - used to automate the creation of AMIs.
  • Terraform - used to create and update infrastructure resources.
  •  Ansible - Ansible is an IT automation tool used to deploy and configure systems.
  • AWS CLI - The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services.

 

Make sure these tools have been installed on the computer you are using to build and deploy ACS 6.1Enterprise.

 

Authenticating to AWS

Both Packer and Terraform need to authenticate to AWS in order to create resources. Since we are creating a large number of infrastructure resources, the user we authenticate with to AWS, needs to have administrator access.

There are multiple ways to configure AWS credentials for Packer and Terraform, the following links will show you how to do it:

 

Note: Whatever method you use make sure you keep your AWS credentials private and secure at all times.


Authenticating to Nexus

Nexus is a repository manager used by Alfresco to publish software artifacts. Packer will connect to Nexus repository to download the necessary software to install ACS 6.1 Enterprise.

Alfresco Enterprise customers have Nexus credentials as part of their license subscription. Please refer to your CRM if you don't know or have your Nexus credentials.

 

Building AWS AMIs

The first step to deploy ACS 6.1 Enterprise is to build two types of AMIs:

  1. Repository AMI - containing Alfresco Repository, Share and ADW (Alfresco Digital Workspace)
  2. Search Services AMI - containing Alfresco Repository and Search Services with Insight Engine.

 

Repository AMI

For this process we use Packer and Ansible. We first export the "nexus_user" and "nexus_password" environment variables containing credentials to access the Nexus repository. These are stored in the ~/.nexus-cfg file contains the following.

export nexus_user=xxxxxxxxx
export nexus_password=xxxxxxxxx

 

Note that the .nexus-cfg file is in the user home folder, keep this file and its contents private and secured at all times.

 

If you want to include custom amps add them to the amps and amps_share folder and they will be deployed to the AMI.

For custom jar files add them to the modules/platform and modules/share folders.

 

If you want to deploy ADW (Alfresco Digital Workspace) place the digital-workspace.war file in the acs-61-files/downloaded folder.

 

We can now execute packer by calling the build_61_AMI.sh script.

cd acs-61-repo-aws-packer
./build_61_AMI.sh

This shell script will load the nexus environment variables and call packer build using a template file for the provisioning of the AMI and a variables file containing deployment specific information such as your default VPC Id, the AWS region, etc.

Make sure you change the value of the vpc_id variable to use your default VPC Id.

 

Search Services AMI

As on the previous section, we use Packer and Ansible to create a Search Services AMI.

 

Make sure you change the value of the vpc_id variable to your default VPC Id before running the build_61_AMI.sh script.

cd acs-61-repo-aws-packer
./build_61_AMI.sh

 

As the script runs you can see what is is doing during its execution...

▶ ./build_61_AMI.sh
amazon-ebs output will be in this color.

==> amazon-ebs: Prevalidating AMI Name: acs-61-repo-1557828971
amazon-ebs: Found Image ID: ami-00846a67
==> amazon-ebs: Creating temporary keypair: packer_5cda956b-bd62-1d09-cef2-639152741025
==> amazon-ebs: Creating temporary security group for this instance: packer_5cda956b-345b-2321-afd5-40b1b06a6bc1
==> amazon-ebs: Authorizing access to port 22 from 0.0.0.0/0 in the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Instance ID: i-0f80505eb56dccbb7
==> amazon-ebs: Waiting for instance (i-0f80505eb56dccbb7) to become ready...

 

On completion the script will output the AMI id of the newly created AMI. Keep track of both AMI Ids, as we will need to use them in the Terraform script next.

Build 'amazon-ebs' finished.

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
eu-west-2: ami-08fd6196500dbcb01

 

Building the AWS Infrastructure and Deploying ACS 6.1 Enterprise

Now that we have created both the Repository and the Search Services AMIs we can start building the AWS infrastructure and deploy ACS 6.1 Enterprise.

In the acs-61-aws-terraform folder we have the terraform.tfvars containing configuration specific for the AWS and ACS deployments.

 

Some of the variables that will need to be updated are:

  • resource-prefix - this is used to name all resources created with some initials to identify the resources belonging to this deployment.
  • aws-region
  • aws-availability-zones
  • vpc-cidr
  • autoscaling-group-key-name
  • s3-bucket-location

 

and of course we need to set the auto scaling image id with the newly generated AMIs

  • autoscaling-repo-group-image-id
  • autoscaling-solr-group-image-id

 

Once the configuration has been set we are ready to start building the solution. We first need initialize terraform with the "terraform init" command:

 

▶ terraform init
Initializing modules...
- module.vpc
- module.rds
- module.alfresco-repo
- module.alfresco-solr
- module.bastion
- module.alb
- module.internal-nlb
- module.activemq

Initializing provider plugins...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.aws: version = "~> 2.10"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

 

We can now issue the apply command to start the build. Upon completion (it will take around 15 minutes) we will get notification of the URLs available to connect to Alfresco.

Apply complete! Resources: 51 added, 0 changed, 0 destroyed.

Outputs:

Alfresco Digital Workspace = http://acs-61-alb-1503842282.eu-west-2.elb.amazonaws.com/digital-workspace
Alfresco Share = http://acs-61-alb-1503842282.eu-west-2.elb.amazonaws.com/share
Alfresco Solr = http://acs-61-alb-1503842282.eu-west-2.elb.amazonaws.com/solr
Alfresco Zeppelin = http://acs-61-alb-1503842282.eu-west-2.elb.amazonaws.com/zeppelin
RDS Endpoint = acs-61-db.cmftgvbzqrto.eu-west-2.rds.amazonaws.com:3306
VPC ID = vpc-006f0c6354656e96d5c

 

To destroy the resources issue a "terraform destroy" command.

 

Terraform will perform the following actions:
.......

Plan: 0 to add, 0 to change, 51 to destroy.

Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.

 

To do list

There are a couple of things to add to this project:

  • CI/CD scripts - I have already implemented this and will publish it on a different blog.
  • On the Search Services instances we should download a backup of the Solr indexes when starting a new instance instead of building the indexes from scratch.

Filter Blog

By date: By tag: