Migrating from Search Services to Search Enterprise

[angelborroy]

From ACS Enterprise 7.1 two kind of Search Engines are available:

• Search Services 1.x and 2.x, that is relaying on a customization of Apache SOLR 6.6
• Search Enterprise 3.x and 4.x, that is a standard client for Elasticsearch and OpenSearch

This blog post provides a step by step tutorial to upgrade an existing ACS running with Search Services (Solr) to a new ACS running with Search Enterprise (OpenSearch). Note that deploying the product in production environments would require additional configuration.

Docker Images from quay.io are used, since this product is only available for Alfresco Enterprise customers. In addition, Alfresco Nexus credentials are required. If you are Enterprise Customer or Partner but you are still experimenting problems to download Docker Images or download artifacts from Nexus, contact Alfresco Hyland Support in order to get required credentials and permissions.

Support resources are available in https://github.com/aborroy/search-services-to-search-enterprise

Steps of the process

1. Prepare the source ACS 23.1 deployment with Search Services (Solr)
2. Prepare the Reindexing App for Search Enterprise (OpenSearch)
3. Prepare the target ACS 23.1 deployment with Search Enterprise (OpenSearch)
4. Run the Reindexing App for Search Enterprise (OpenSearch)

Prepare source ACS 23.1

Custom Content Models can be defined in Alfresco using different options:

• Upload a XML file with a model definition to Repository > Data Dictionary > Models folder in the repository. This node must have Model Active property enabled for model that is used.
• Create a new content model using the Model Manager tool available in Share web application. The action Activate should be enabled for the model to be used. Definition of the new model is also stored in the same Repository folder, but applying a hidden aspect for the XML definition files.
• Apply an Alfresco addon that includes XML files with a content model definition

Since the Content Store will be re-used by target ACS, that includes all the Repository files, only content model definitions from Alfresco addons must be identified. Collect all the JAR or AMP files to apply them to target ACS.

Reindexing application requires a specific file, commonly named as reindex.prefixes-file.json, that includes a mapping of namespaces and prefixes of the content model definitions. This file can be obtained applying the Alfresco Repository addon provided in https://github.com/AlfrescoLabs/model-ns-prefix-mapping. Deploy the addon and invoke following URL to save the result as reindex.prefixes-file.json

http://localhost:8080/alfresco/s/model/ns-prefix-map

This file will be used later, when running the Reindexing process.

Check out https://github.com/aborroy/search-services-to-search-enterprise#1-prepare-source-acs-231 to understand how these actions can be applied on a demo environment.

Prepare Reindexing App

Reindexing App is a regular Spring Boot application that can be downloaded from Alfresco Nexus using customer credentials. For instance, to download version 4.0.0 use the following command:

$ curl --user $NEXUS_USER:$NEXUS_PASS \
https://nexus.alfresco.com/nexus/service/local/repositories/enterprise-releases/content/org/alfresco/alfresco-elasticsearch-reindexing/4.0.0/alfresco-elasticsearch-reindexing-4.0.0-app.jar \
-o alfresco-elasticsearch-reindexing-4.0.0-app.jar

Copy previous reindex.prefixes-file.json file to the same folder.

Check out https://github.com/aborroy/search-services-to-search-enterprise#2-prepare-reindexing-app to understand how these actions can be applied on a demo environment.

Prepare target ACS 23.1

Reindexing App requires following services in target ACS 23.1 to be accessible from execution environment:

• Database via JDBC protocol, like jdbcostgresql://localhost:5432/alfresco
• ActiveMQ via TCP protocol, like tcp://localhost:61616
• Transform Service via HTTP protocol, like http://localhost:8095/transform/config
• OpenSearch via HTTP protocol, like http://localhost:9200

Remember to deploy all collected JAR or AMP files containing content model definitions from source ACS.

Once this target ACS is up & ready (with the services and the addons applied), alfresco index must be created in OpenSearch. To trigger this creation process, access to ADW and click "Search" button. To verify the sucess of the operation, get the mapping of the alfresco index in OpenSearch from a URL similar to the following one:

http://localhost:9200

The result must be similar to this one:

{"alfresco":
  {"mappings":
    {"dynamic":"false","properties":
      {"ALIVE":{"type":"boolean"},"ANAME":{"type":"keyword"},"...": {"...":"..."} 
      }
    }
  }
}

Check out https://github.com/aborroy/search-services-to-search-enterprise#3-prepare-target-acs-231 to understand how these actions can be applied on a demo environment.

Run Reindexing App

Before running Reindexing App (with target ACS 23.1 up & running), identify the maximum number Id in ALF_NODE table.

For instance, for a PostgreSQL database, use following query:

# psql --u alfresco
psql (14.4 (Debian 14.4-1.pgdg110+1))
Type "help" for help.

alfresco=# select max(id) from alf_node;
 max
-----
 896
(1 row)

This result will define the limit for the reindexing process (896)

At this point, Reindeing App can be run to populate OpenSearch index in target ACS

java -jar alfresco-elasticsearch-reindexing-4.0.0-app.jar \
--alfresco.reindex.jobName=reindexByIds \
--alfresco.reindex.pagesize=100 \
--alfresco.reindex.batchSize=100 \
--alfresco.reindex.fromId=1 \
--alfresco.reindex.toId=896 \
--alfresco.reindex.concurrentProcessors=2 \
--alfresco.accepted-content-media-types-cache.base-url=http://localhost:8095/transform/config \
--spring.activemq.broker-url="tcp://localhost:61616?jms.useAsyncSend=true" \
--spring.elasticsearch.rest.uris=http://localhost:9200 \
--spring.datasource.url=jdbc:postgresql://localhost:5432/alfresco \
--alfresco.reindex.prefixes-file=file:reindex.prefixes-file.json
...
Step: [reindexByIdsStep] executed in 2s185ms
Total number of indexed nodes (includes retried items) 786
Total number of failed nodes (includes retried items) 0
Total number of updated nodes with Path (includes retried items) 785
Total number of nodes with failed Path update (includes retried items) 0 

Once the process is done, target ACS Repository contains an updated searching index and it's ready to be used.

When dealing with large repositories, reindexing process may be executed in parallel, so the task can be completed faster. This application uses standard Spring Batch configuration and details of this process can be obtained in Alfresco Docs.

Check out https://github.com/aborroy/search-services-to-search-enterprise#4-run-reindexing-app to understand how these actions can be applied on a demo environment.

Video Recording