AnsweredAssumed Answered

Desktop sync (0.9.5 and 0.9.6) fails due to very large result from lucene query

Question asked by bopolissimus on Aug 15, 2013
Latest reply on Aug 15, 2013 by bopolissimus
Hi all,

Context:
  Server Alfresco Community 4.2.c on Ubuntu Precise using lucene indexing
  Desktop sync client 0.9.5 and 0.9.6 on Windows 7
 
  The alfresco content is approx 21GB on disk.  There are more than 7000 users and more than 7000
  groups, both imported from LDAP.

  This alfresco instance has been upgraded from Alfresco community 3.4 to 4.2.c

I run desktop sync (with wireshark on the server sniffing the wire) and my wireshark filter

   "tcp.port == 8080 and http"

so I only see the HTTP requests and replies (no TCP syn/ack/fin, etc).

Most requests to /alfresco/api succeed.  The ones that fail are sending this query:

  +ALL:"*" +@sys\:archivedDate:[2013\-08\-14T17:53:52.0649 TO 2013\-08\-14T17:53:37]

and they're failing with this error:

  org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 10000

(and tomcat is returning a 500 status)

I can see the matching exceptions in alfresco.log.

I've enabled debug logging of lucene FTS queries with:

  log4j.logger.org.alfresco.repo.search.impl.lucene.LuceneQueryParser=debug

and I can see the reply going by (in the log!, not in wireshark).  When I cut the reply out of the alfresco.log I get a file that is 127645 bytes wide.

Is this an issue with soap requests only?  When I run exactly the same query in the browser (/alfresco/faces/jsp/admin/node-browser.jsp with search language lucene) I don't get an error (although I get Results (0 rows)) and the query runs in Time ms:83.

The corresponding exception in alfresco is:


org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 10000
   at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:163)
   at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:154)
   at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:63)
   at org.apache.lucene.search.WildcardQuery.rewrite(WildcardQuery.java:54)
   at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383)
   at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383)
   at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:162)
   at org.apache.lucene.search.Query.weight(Query.java:94)
   at org.apache.lucene.search.Hits.<init>(Hits.java:76)
   at org.apache.lucene.search.Searcher.search(Searcher.java:50)
   at org.apache.lucene.search.Searcher.search(Searcher.java:40)
   at org.alfresco.repo.search.impl.lucene.LuceneAlfrescoLuceneQueryLanguage.executeQuery(LuceneAlfrescoLuceneQueryLanguage.java:218)
   at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:278)
   at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
   at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
   at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
   at sun.reflect.GeneratedMethodAccessor457.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.alfresco.repo.management.subsystems.SubsystemProxyFactory$1.invoke(SubsystemProxyFactory.java:72)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at com.sun.proxy.$Proxy12.query(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor457.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
   at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:46)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:159)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at com.sun.proxy.$Proxy12.query(Unknown Source)
   at org.alfresco.repo.webservice.repository.SearchQuery.execute(SearchQuery.java:98)
   at org.alfresco.repo.webservice.repository.SearchQuery.execute(SearchQuery.java:50)
   at org.alfresco.repo.webservice.AbstractQuery.execute(AbstractQuery.java:50)
   at org.alfresco.repo.webservice.AbstractQuerySession.getQueryResults(AbstractQuerySession.java:105)
   at org.alfresco.repo.webservice.repository.RepositoryQuerySession.getNextResults(RepositoryQuerySession.java:58)
   at org.alfresco.repo.webservice.repository.RepositoryWebService$2.execute(RepositoryWebService.java:143)
   at org.alfresco.repo.webservice.repository.RepositoryWebService$2.execute(RepositoryWebService.java:134)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:433)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:323)
   at org.alfresco.repo.webservice.repository.RepositoryWebService.executeQuery(RepositoryWebService.java:161)
   at org.alfresco.repo.webservice.repository.RepositoryWebService.query(RepositoryWebService.java:189)
   at sun.reflect.GeneratedMethodAccessor470.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
   at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
   at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
   at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
   at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
   at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
   at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
   at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
   at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
   at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:61)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
   at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
   at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
   at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
   at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:1771)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:722)

Outcomes