Is there a consistent documentation to setup search services and keystores ? Or source for 2.0.5 ?

[mikef38]

Hello

I am trying to install Alfresco (latest version) and of course the search services.

For Alfresco, for an unknown reason I could make it work, mostly. Excepted for the connection with the search services. But I have the same problem in both components (Repo and Search) so if I can understand one, I might have clue to understand the other.

The problem is that they complain : "password can't be null".

So I will try first with the search services, because it is a standalone application, easier to start and stop hundreds of times.

In spite of cleverly inconsistent documentation, I could, I believe, setup the search services, I created the keystores with the tool downloaded from https://github.com/Alfresco/alfresco-ssl-generator , provided the information of the keystores (location, password, type) in the numerous places it was required, before that there were other errors such as unsafe location of the keystore resource (although I put it where I was told to do), but now Solr starts, excepted for the "password can't be null" error.

For instance, I ran the cool command, found somewhere in the documentation :

./solr/bin/solr start -a  "-Dcreate.alfresco.defaults=alfresco,archive
 -Dsolr.ssl.checkPeerName=false
 -Dsolr.allow.unsafe.resourceloading=true
 -Dssl-keystore.password=PASSWORD-KS
 -Dssl-keystore.aliases=ssl-alfresco-ca,ssl-repo-client
 -Dssl-keystore.ssl-alfresco-ca.password=PASSWORD-KS
 -Dssl-keystore.ssl-repo-client.password=PASSWORD-KS
 -Dssl-truststore.password=PASSWORD-TRS
 -Dssl-truststore.aliases=ssl-alfresco-ca,ssl-repo,ssl-repo-client
 -Dssl-truststore.ssl-alfresco-ca.password=PASSWORD-TRS
 -Dssl-truststore.ssl-repo.password=PASSWORD-TRS
 -Dssl-truststore.ssl-repo-client.password=PASSWORD-TRS" -f

where PASSWORD-KS is the password for the keystore, and PASSWORD-TRS is the password for the truststore.

At this point I have an error: 

javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

Java is Java 11, because I tried with 16 and it doesn't work because half dozen start parameters are unknown.

If I start solr with "solr start", I get the null password error.

So now I will try to get the source of my version and try to find what exactly I should do in order to give the password or make it work.

I had downloaded the version 2.0.5. 

I go to the source repo (https://github.com/Alfresco/SearchServices/tree/2.0.0), no 2.0.5 version. There are tags for 2.0.0, 2.0.1, 2.0.2, 2.0.3, and pre-releases of 2.1.0.

Ok, let's go back to the beginning, download it again and check what is the latest version or an older one.

I go to the Alfresco download page for community edition. There I have "7.3 Community install". Ok. 

Down the page I can find: 3. Non-containerized manual deployment. So far so good. I was there already.

And: Alfresco Search Services 2.0.3.zip. Hum ? 2.0.3 ?

But the link is: https://download.alfresco.com/cloudfront/release/community/SearchServices/2.0.5/alfresco-search-serv.... And the README file says it's 2.0.5. That's why I believed I installed the version 2.0.5.

Then where is the source for this version ?

So to resume

• if someone can give the step-by-step procedure to setup this and have it work without any "null password" error, I'd be glad to follow it.
• Else if I have to find where is the mess, where are the sources in sync with the version available for download ?

Thank you.

[angelborroy]

Search Services GitHub project is a mirror of a private project. You can find details in https://hub.alfresco.com/t5/alfresco-content-services-blog/community-mirroring-with-git/ba-p/298628. If you want to use the Community version and build everything from scratch, I recommend you to use 2.0.3: https://github.com/alfresco/SearchServices/tree/2.0.3

Despite the official documentation, available in https://docs.alfresco.com/search-services/latest/install/options/

You have many different resources:

• Alfresco Docker Installer - https://github.com/alfresco/alfresco-docker-installer
• Sample projects - https://github.com/aborroy/alfresco-solr-docker-mtls
• Blog posts from the Community - https://hub.alfresco.com/t5/alfresco-content-services-blog/a-simplified-guide-to-enable-acs-mutual-t...
• Step by step installation walkthrougts - https://javaworld-abhinav.blogspot.com/2021/06/setup-acs70-ass201-and-transformation-service.html

I hope that is useful for you.

[angelborroy]

There is even a YouTube video describing the different communication modes between SOLR and Repo:

https://www.youtube.com/watch?v=BSNJz6nd8Xg

[mikef38]

Hello

Thank for the video but I am so far at the middle trying to start Alfresco in mtls mode of course.

I didn't answer yet because I have tried to run Alfresco zillions of times.

The keystore tool creates without extra options (other than the passwords) the following aliases, that are not those you use (extract of keytool output) :

ssl.keystore:
ssl.alfresco.ca, 7 févr. 2023, trustedCertEntry, 
ssl.repo, 7 févr. 2023, PrivateKeyEntry, 

ssl.truststore:
alfresco.ca, 7 févr. 2023, trustedCertEntry, 
ssl.repo.client, 7 févr. 2023, trustedCertEntry, 

My start options as given by Tomcat in the list of arguments (funny, the passwords are displayed by Tomcat in its log), I replaced the end of actual values with stars:

 -Dssl-keystore.password=ks+******
 -Dssl-keystore.aliases=ssl.alfresco.ca,ssl.repo
 -Dssl-keystore.ssl.alfresco.ca.password=ks+******
 -Dssl-keystore.ssl.repo.password=ks+******
 -Dssl-truststore.password=trs+******
 -Dssl-truststore.aliases=alfresco.ca,ssl.repo.client
 -Dssl-truststore.alfresco.ca.password=trs+******
 -Dssl-truststore.ssl.repo-client.password=trs+******

Result at this point after numerous other fails:

org.alfresco.error.AlfrescoRuntimeException: 01080000 Keystores are invalid
...
Caused by: org.alfresco.encryption.MissingKeyException: Key metadata is missing from keystore /opt/Alfresco/alf_data_7/keystore/keystore
	at org.alfresco.encryption.AlfrescoKeyStoreImpl.validateKeys(AlfrescoKeyStoreImpl.java:910)
	at org.alfresco.encryption.AlfrescoKeyStoreImpl.validateKeys(AlfrescoKeyStoreImpl.java:188)
	at org.alfresco.encryption.KeyStoreChecker.validateKeyStores(KeyStoreChecker.java:49)
	at org.alfresco.encryption.EncryptionChecker$1.execute(EncryptionChecker.java:73)

[mikef38]

Alfresco is started (again), this time without the errors about "null passwords".

But Solr still doesn't work. The keystores generated by ssl-tool are loaded without any error, at last.

But it gives: org.alfresco.error.AlfrescoRuntimeException: 01100000 api/solr/modelsdiff return status:403

org.alfresco.error.AlfrescoRuntimeException: 01100000 api/solr/modelsdiff return status:403
	at org.alfresco.solr.client.SOLRAPIClient.callRepository(SOLRAPIClient.java:1598)
	at org.alfresco.solr.client.SOLRAPIClient.getModelsDiff(SOLRAPIClient.java:1105)
	at org.alfresco.solr.tracker.ModelTracker.trackModelsImpl(ModelTracker.java:313)
	at org.alfresco.solr.tracker.ModelTracker.trackModels(ModelTracker.java:275)
	at org.alfresco.solr.tracker.ModelTracker.ensureFirstModelSync(ModelTracker.java:297)
	at org.alfresco.solr.lifecycle.SolrCoreLoadListener.createModelTracker(SolrCoreLoadListener.java:345)
	at org.alfresco.solr.lifecycle.SolrCoreLoadListener.newSearcher(SolrCoreLoadListener.java:139)
	at org.apache.solr.core.SolrCore.lambda$getSearcher$15(SolrCore.java:2249)

 Probably I missed some mandatory step, but which one ?

[mikef38]

So it looks like it just does not work.

All is finally configured propertly, I checked that if I introduce mistakes one by one in ports, passwords, etc., I get the expected error so I know that ly settings are used and valid.

But still error 403, Alfresco doesn't want solr access.

Then I tried to change the secureComms mode to secret, and miracle, not more error 403, Solr started indexing.

The problem is that Solr is in HTTPS mode and apparently Alfresco does not like that since when I try a query I obtain:

Caused by: org.apache.commons.httpclient.ProtocolException: The server localhost failed to respond with a valid HTTP response
	at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1987)
	at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
	at org.alfresco.httpclient.RequestHeadersHttpClient.executeMethod(RequestHeadersHttpClient.java:90)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
	at org.alfresco.httpclient.RequestHeadersHttpClient.executeMethod(RequestHeadersHttpClient.java:75)
	at org.alfresco.repo.search.impl.solr.AbstractSolrQueryHTTPClient.postQuery(AbstractSolrQueryHTTPClient.java:100)
	at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.postSolrQuery(SolrQueryHTTPClient.java:1112)
	at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.executeQuery(SolrQueryHTTPClient.java:579)

So I would have to throw away all the SSL configuration I took days to fix and go back to plain HTTP + a ridiculous "secret" setting.

Thank you very much for all the days lost for nothing, expected get a non-working **bleep** that I will have to throw away.

Fortunately I am still ROFL when I see the password, stored in the startup scripts, passed as command arguments (I followed the documentation) and displayed by tomcat at startup, or available for any tier application who just have to setenv() to get them, this because "storing passwords in a property file is not secure".

[mikef38]

So overnight I had another surprise, at 1 AM Solr tried to do a "snapshot", whatever it is, but failed because some directory in solrhome does not exist. I googled this error and what I got was about someone who installed some site creation module in Alfresco and didn't put admin in some group. But obviously it's a problem with solr settings or environment... Next step will probably be a sequel of Alice in Wonderland, they got all what it takes to do it.

[angelborroy]

Additional resource:

https://www.slideshare.net/angelborroy/alfresco-certificates

[mikef38]

Hello,

I posted a long answer with the results and all hours ago, and it was here, I even checked that a long log was collapsed by the "spoiler" button, but now this answer no more there !!! Is it a joke ?

Ok, short version then.

I downloaded the search-services version 2.0.3, did the same setup, activated the SSL log, and apparently the communication is OK.

Spoiler

"ClientHello": {
  "client version"      : "TLSv1.2",
  "random"              : "D7 CC 7B 96 E7 AC B9 B9 3B 28 14 59 3E 26 F0 A8 AE 64 07 41 00 F3 B6 78 BE 3B 84 56 23 C3 58 89",
  "session id"          : "6F 13 F1 16 6C CC 9C 90 BA 7D D0 3F 8E D4 41 5B 13 8D 56 21 47 9C EA ED 30 AA 91 01 5C BF 95 24",
  "cipher suites"       : "[TLS_AES_256_GCM_SHA384(0x1302), TLS_AES_128_GCM_SHA256(0x1301), TLS_CHACHA20_POLY1305_SHA256(0x1303), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA9), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA8), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCAA), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_EMPTY_RENEGOTIATION_INFO_SCSV(0x00FF)]",
  "compression methods" : "00",
  "extensions"          : [
    "status_request (5)": {
      "certificate status type": ocsp
      "OCSP status request": {
        "responder_id": <empty>
        "request extensions": {
          <empty>
        }
      }
    },
    "supported_groups (10)": {
      "versions": [x25519, secp256r1, secp384r1, secp521r1, x448, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
    },
    "ec_point_formats (11)": {
      "formats": [uncompressed]
    },
    "signature_algorithms (13)": {
      "signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, rsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1]
    },
    "signature_algorithms_cert (50)": {
      "signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, rsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1]
    },
    "status_request_v2 (17)": {
      "cert status request": {
        "certificate status type": ocsp_multi
        "OCSP status request": {
          "responder_id": <empty>
          "request extensions": {
            <empty>
          }
        }
      }
    },
    "extended_master_secret (23)": {
      <empty>
    },
    "supported_versions (43)": {
      "versions": [TLSv1.3, TLSv1.2]
    },
    "psk_key_exchange_modes (45)": {
      "ke_modes": [psk_dhe_ke]
    },
    "key_share (51)": {
      "client_shares": [  
        {
          "named group": x25519
          "key_exchange": {
            0000: 56 4D 53 DE A4 3A CA 4C   C7 82 89 4E 6E C5 B3 93  VMS..:.L...Nn...
            0010: E8 51 AB 80 07 BB 60 41   FD 08 01 AD 67 A8 88 7A  .Q....`A....g..z
          }
        },
      ]
    }
  ]
}
)
javax.net.ssl|DEBUG|4C|searcherExecutor-8-thread-1-processing-x:alfresco|2023-02-15 16:29:25.048 UTC|ServerHello.java:867|Consuming ServerHello handshake message (
"ServerHello": {
  "server version"      : "TLSv1.2",
  "random"              : "A2 C6 D9 CF F1 EA 18 9E 24 D6 A0 26 4C CD 84 B3 F3 34 1E C7 34 12 B5 C6 97 02 3A AA 51 FE 76 B4",
  "session id"          : "6F 13 F1 16 6C CC 9C 90 BA 7D D0 3F 8E D4 41 5B 13 8D 56 21 47 9C EA ED 30 AA 91 01 5C BF 95 24",
  "cipher suite"        : "TLS_AES_256_GCM_SHA384(0x1302)",
  "compression methods" : "00",
  "extensions"          : [
    "supported_versions (43)": {
      "selected version": [TLSv1.3]
    },
    "key_share (51)": {
      "server_share": {
        "named group": x25519
        "key_exchange": {
          0000: C8 08 EC C2 D9 90 8F F0   BB 60 E0 97 65 E3 B8 A4  .........`..e...
          0010: E0 8A 55 CB 4E B1 24 4A   5A 5C 3A AD D9 8A B4 09  ..U.N.$JZ\:.....
        }
      },
    }
  ]
}
)

The problem is still: error 403 when solr tries to get data from Alfresco.

2023-02-14 17:07:30.751 ERROR (searcherExecutor-8-thread-1-processing-x:archive) [   x:archive] o.a.s.t.ModelTracker Model tracking failed for core: archive
org.alfresco.error.AlfrescoRuntimeException: 01140000 api/solr/modelsdiff return status:403
	at org.alfresco.solr.client.SOLRAPIClient.callRepository(SOLRAPIClient.java:1598)
	at org.alfresco.solr.client.SOLRAPIClient.getModelsDiff(SOLRAPIClient.java:1105)

I understand that Alfresco refuses to reply to Solr even in MTLS mode ? Why ?

The first script (or so?) called by Solr is aclChangeSets.get, there is a comment in the script explaining why there is no authentication:

<!-- Solr api is protected by a solr-specific authentication mechanism -->
<authentication>none</authentication>

So, how and where is done this "solr-specific" authetication, that will block Solr in MTLS mode ?

This, in order to understand what I don't do properly.

Thank you.

[mikef38]

Hello

So I answered by myself to the red question with the help of Eclipse and the code, the policy is enforced in X509ServletFilterBase, it depends on the mode:

- in "secret" mode it's about checking the secret in a specific HTTP header (lol).

- in "https" mode the filter checks if there is a certificate in an attribute of the request(javax.servlet.request.X509Certificate). And then verify if it is valid.

But here there is none. certs = null.

So what should I do in order to have this certificate in the request from Solr ?

Thank you.