Alfresco Search Enterprise 3.x - Queries and Doubts

[jigirshah]

Hello Everyone,

I'm currently looking to setup Alfresco Search enterprise 3.x with my ACS 7.1.x version. I would like to index only Metadata properties, not content or path. While setting and configuring this, I come across few issues and doubts. Can anyone help on below queries? It would be much appreciated.

1. Reindex app and Live index app will work without transformation server and shared file store? It seems to be not working without these components. If I want only metadata based search, there is no need to have tansformation server or shared file store, right?

2. Only elasticearch-metadata app will work without runnin any other jars? It seems to be not indexing documents if I run only metadata app jar. Can anyone confirm?

3. Instead of running Elastic search server, can we use AWS Open Search? If I integrate OpenSearch, there is no need to run Live and Re-indexing app? AWS OpenSearch is having all required components?

4. I want to index all existing documents from repository. While running re-indexing app jar, facing below errors. Any idea how to resolve?

INFO 23252 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=reindexByIds]] launched with the following parameters: [{run.id=1}]
INFO 23252 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [validateDbSchemaStep]
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionMinor
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of licenseMode
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionProperties
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of isIndexed
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionEdition
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of name
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionSchema
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionBuild
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of keyStore
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionLabel
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of authorizationChecksum
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionRevision
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of versionMajor
ERROR 23252 --- [xTaskExecutor-4] o.a.r.processors.AlfrescoNodeProcessor   : impossible to get prefixed name of isContentIndexed
ERROR 23252 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step validateDbSchemaStep in job reindexByIds java.lang.NumberFormatException: null
        at java.base/java.lang.Integer.parseInt(Integer.java:614) ~[na:na]
        at java.base/java.lang.Integer.parseInt(Integer.java:770) ~[na:na]
        at org.alfresco.reindexing.beans.RepositoryDescriptor.getSchema(RepositoryDescriptor.java:95) ~[classes!/:3.2.0]
        at org.alfresco.reindexing.writers.RepositoryDescriptorItemWriter.write(RepositoryDescriptorItemWriter.java:79) ~[classes!/:3.2.0]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:193) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:159) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:294) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:217) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:77) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:407) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.3.23.jar!/:5.3.23]
        at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82) ~[spring-batch-core-4.3.7.jar!/:4.3.7]
        at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262) ~[spring-batch-infrastructure-4.3.7.jar!/:4.3.7]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

[angelborroy]

1. If you don't want to index content, Transformation Service & Shared File Store are not required.

2. You need, at least, two components for live indexing: live-indexing-mediation & live-indexing-metadata. If you want to index also PATH information, add live-indexing-path

3. OpenSearch and Elasticsearch work in the same way. You need to use the live-indexing-* apps with both.

4. It looks like field "sys:versionSchema" has not a number in you Alfresco DB

[jigirshah]

Thanks for the response angelborroy.

Can you please help me with my below points on your reply?

1. If I do not run or integrate Transformation service & shared file store, re-indexing and live-indexing apps are keep on giving ERROR as shwon below. And no re-indexing happening. Checked from Alfresco admin console and number remains same. However, for new uploaded documents, indexing are being created and searchable with metadata. Hence, can you please check or let me know what could be wrong? 

From Live Indexing app:
ERROR 32168 --- [ent@19e0dffe-75] o.a.c.m.AcceptedContentMediaTypesCache   : Can't retrieve content media types: Connection refused: no further information; nested exception is java.net.ConnectException: Connection refused: no further information

From Re-indexing app:
ERROR 33432 --- [ctor-http-nio-4] o.a.c.m.AcceptedContentMediaTypesCache   : Can't retrieve content media types: Connection refused: no further information: localhost/127.0.0.1:8090; nested exception is io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: localhost/127.0.0.1:8090

2. I will try by running these 2 jars and see if live indexing for metadata is working or not.

3. Ok. So, OpenSearch can be configured to have managed service and no need to maintain infra I believe.

4. How can "sys:versionSchema" be not a number? I haven't set anything explicitely. Can you please let me know how to resolve or get to know which node is creating issue?

[angelborroy]

1. Did you follow this instructions?

https://docs.alfresco.com/search-enterprise/latest/admin/#indexing-only-metadata

4. No idea on how to get deeper into the issue. Probably, I'd try to log the response to find out what the repository is returning as "sys:versionSchema"

[jigirshah]

Hi @angelborroy ,

Yes, I'm setting the parameter alfresco.reindex.contentIndexingEnabled to false. But, still, I'm getting same error and Re-indexing is not progressing. Adding logs below. Let me know if anything is wrong.

2023-02-06 15:51:41.007  INFO 83024 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 9191 (http)
2023-02-06 15:51:41.039  INFO 83024 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2023-02-06 15:51:41.040  INFO 83024 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.65]
2023-02-06 15:51:41.414  INFO 83024 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2023-02-06 15:51:41.417  INFO 83024 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 9422 ms
2023-02-06 15:51:42.218  INFO 83024 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2023-02-06 15:51:43.477  INFO 83024 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2023-02-06 15:51:43.976  INFO 83024 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-02-06 15:51:44.152  INFO 83024 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.6.11.Final
2023-02-06 15:51:44.581  INFO 83024 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2023-02-06 15:51:44.890  INFO 83024 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQLDialect
2023-02-06 15:51:45.865  WARN 83024 --- [           main] org.hibernate.mapping.RootClass          : HHH000038: Composite-id class does not override equals(): org.alfresco.reindexing.model.AccessControlEntryKey
2023-02-06 15:51:45.866  WARN 83024 --- [           main] org.hibernate.mapping.RootClass          : HHH000039: Composite-id class does not override hashCode(): org.alfresco.reindexing.model.AccessControlEntryKey
2023-02-06 15:51:45.871  WARN 83024 --- [           main] org.hibernate.mapping.RootClass          : HHH000038: Composite-id class does not override equals(): org.alfresco.reindexing.model.PropertyKey
2023-02-06 15:51:45.873  WARN 83024 --- [           main] org.hibernate.mapping.RootClass          : HHH000039: Composite-id class does not override hashCode(): org.alfresco.reindexing.model.PropertyKey
2023-02-06 15:51:46.991  INFO 83024 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-02-06 15:51:47.013  INFO 83024 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-02-06 15:51:48.691  INFO 83024 --- [           main] bConfig$$EnhancerBySpringCGLIB$$53566385 : Reindexing by ID bounds: 0 - 10000
2023-02-06 15:51:48.726  INFO 83024 --- [           main] a.r.p.ReindexEventFieldsRemoverProcessor : These fields will be removed:: [cmis:changeToken, alfcmis:nodeRef, cmis:isImmutable, cmis:isLatestVersion, cmis:isMajorVersion, cmis:isLatestMajorVersion, cmis:isVersionSeriesCheckedOut, cmis:versionSeriesCheckedOutBy, cmis:versionSeriesCheckedOutId, cmis:checkinComment, cmis:contentStreamId, cmis:isPrivateWorkingCopy, cmis:allowedChildObjectTypeIds, cmis:sourceId, cmis:targetId, cmis:policyText, trx:password, pub:publishingEventPayload]
2023-02-06 15:51:50.518  INFO 83024 --- [           main] bConfig$$EnhancerBySpringCGLIB$$53566385 : Retry enabled: max retry count set to 3
2023-02-06 15:51:50.519  INFO 83024 --- [           main] bConfig$$EnhancerBySpringCGLIB$$53566385 : Backoff policy set to 1000 msecs as initial delay, multiplied by 2 for each retry, for a maximum delay of 30000 msecs
2023-02-06 15:51:51.179  WARN 83024 --- [           main] o.s.b.c.c.a.DefaultBatchConfigurer       : No transaction manager was provided, using a DataSourceTransactionManager
2023-02-06 15:51:51.208  INFO 83024 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: HSQL
2023-02-06 15:51:51.258  INFO 83024 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : No TaskExecutor has been set, defaulting to synchronous executor.
2023-02-06 15:51:51.275  INFO 83024 --- [           main] bConfig$$EnhancerBySpringCGLIB$$d2209e32 : Reindexing by date bounds: -2208988800000 - 1924991940000
2023-02-06 15:51:51.280  INFO 83024 --- [           main] bConfig$$EnhancerBySpringCGLIB$$d2209e32 : Retry enabled: max retry count set to 3
2023-02-06 15:51:51.281  INFO 83024 --- [           main] bConfig$$EnhancerBySpringCGLIB$$d2209e32 : Backoff policy set to 1000 msecs as initial delay, multiplied by 2 for each retry, for a maximum delay of 30000 msecs
2023-02-06 15:51:51.419  WARN 83024 --- [           main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2023-02-06 15:51:55.409  INFO 83024 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2023-02-06 15:51:56.111  INFO 83024 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2023-02-06 15:51:56.113  INFO 83024 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.errorChannel' has 1 subscriber(s).
2023-02-06 15:51:56.116  INFO 83024 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started bean '_org.springframework.integration.errorLogger'
2023-02-06 15:51:56.148  INFO 83024 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 9191 (http) with context path ''
2023-02-06 15:51:56.440  INFO 83024 --- [           main] o.a.c.impl.engine.AbstractCamelContext   : Routes startup (total:0 started:0)
2023-02-06 15:51:56.441  INFO 83024 --- [           main] o.a.c.impl.engine.AbstractCamelContext   : Apache Camel 3.15.0 (camel-1) started in 503ms (build:249ms init:204ms start:50ms)
2023-02-06 15:51:56.456  INFO 83024 --- [           main] org.alfresco.reindexing.ReindexingApp    : Started ReindexingApp in 26.383 seconds (JVM running for 29.058)
2023-02-06 15:52:04.471 ERROR 83024 --- [ctor-http-nio-2] o.a.c.m.AcceptedContentMediaTypesCache   : Can't retrieve content media types: Connection refused: no further information: localhost/127.0.0.1:8090; nested exception is io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: localhost/127.0.0.1:8090
2023-02-06 15:52:07.299 ERROR 83024 --- [ctor-http-nio-3] o.a.c.m.AcceptedContentMediaTypesCache   : Can't retrieve content media types: Connection refused: no further information: localhost/127.0.0.1:8090; nested exception is io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: localhost/127.0.0.1:8090

[angelborroy]

I tried this one on my side with ESC 3.2.0:

java -jar alfresco-elasticsearch-reindexing-3.2.0-app.jar \
    --alfresco.reindex.contentIndexingEnabled=false

And it worked as expected. Metadata was indexed successfully using ACS 7.3.0.

What version are you using?

[jigirshah]

Hi @angelborroy ,

I'm using ACS Enterprise - 7.1.0 (r0a8ca6f2-b3963)

As per Supported Platforms in Search Enterprise Documentation, it is supported version. Can you please let me know if not? Or can you test with this version if possible?

[asirika]

@jigirshah ,

I am not sure whether you already found the resolution by now if not, please note that if you append

--alfresco.accepted-content-media-types-cache.enabled=false to the reindexing command you can get rid of the Error message : Can't retrieve content media types: Connection refused: localhost/127.0.0.1:8090;