AnsweredAssumed Answered

Alfresc indexing slow due to transformation

Question asked by dmorozov on May 12, 2011
Latest reply on Mar 12, 2012 by ebogaard
Hello,
I have been fighting last week with Alfresco going terribly slow because (I think) of Tika transformations happening in background.
Please provide an advice how to solve this issue.

We have Alfresco 3.4.d installed on Ubuntu 64 bit server.
RAM: 16G
CPU: 4
JVM settings: -Djava.awt.headless=true -server -Xss1M -Xms1G -Xmx4G -XX:NewSize=1G -XX:MaxPermSize=512M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=80 -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:+UseTLAB
Database -> MySQL on separate server
Content repository size: 23G
Server: Apache tomcat 6.0.32

Alfresco starting with memory about 1.5G and after some time memory usage jumped up to 4.6G
This seems okay while it has good throughout. No slowness, now errors.

But after some time it go really slow and then hang. Even if nobody use the site for some time. I don't know what cause the issue but here is what I have:
1. Linux top shows the average CPU utilization is about 25% (can assume that one of CPUs loaded for ~100% ???)
2. memory dump (kill -3 PID) shows always the same picture. The only really interesting thread that always showed while slowness is Tika transformer for Excel files started from full text search job:

"DefaultScheduler_Worker-2" prio=10 tid=0x0000000041280800 nid=0x7891 runnable [0x00007fe37f4f2000]
   java.lang.Thread.State: RUNNABLE
        at org.apache.xmlbeans.impl.store.Locale.count(Locale.java:2049)
        at org.apache.xmlbeans.impl.store.Xobj.count_elements(Xobj.java:2050)
        at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTSheetDataImpl.sizeOfRowArray(Unknown Source)
        - locked <0x00007fe3e3352a58> (a org.apache.xmlbeans.impl.store.Locale)
        at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTSheetDataImpl$1RowList.size(Unknown Source)
        at java.util.AbstractList$Itr.hasNext(AbstractList.java:339)
        at org.apache.poi.xssf.usermodel.XSSFSheet.initRows(XSSFSheet.java:177)
        at org.apache.poi.xssf.usermodel.XSSFSheet.read(XSSFSheet.java:147)
        at org.apache.poi.xssf.usermodel.XSSFSheet.onDocumentRead(XSSFSheet.java:134)
        at org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:234)
        at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:190)
        at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:182)
        at org.apache.poi.xssf.extractor.XSSFExcelExtractor.<init>(XSSFExcelExtractor.java:56)
        at org.apache.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactory.java:172)
        at org.apache.poi.extractor.ExtractorFactory.createExtractor(ExtractorFactory.java:152)
        at org.apache.tika.parser.microsoft.ooxml.OOXMLExtractorFactory.parse(OOXMLExtractorFactory.java:65)
        at org.apache.tika.parser.microsoft.ooxml.OOXMLParser.parse(OOXMLParser.java:68)
        at org.alfresco.repo.content.TikaOfficeDetectParser.parse(TikaOfficeDetectParser.java:78)
        at org.alfresco.repo.content.transform.TikaPoweredContentTransformer.transformInternal(TikaPoweredContentTransformer.java:185)
        at org.alfresco.repo.content.transform.AbstractContentTransformer2.transform(AbstractContentTransformer2.java:161)
        at org.alfresco.repo.content.transform.AbstractContentTransformer2.transform(AbstractContentTransformer2.java:137)
        at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.indexProperty(ADMLuceneIndexerImpl.java:944)
        at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocumentsImpl(ADMLuceneIndexerImpl.java:620)
        at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocuments(ADMLuceneIndexerImpl.java:585)
        at org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.updateFullTextSearch(ADMLuceneIndexerImpl.java:1580)
        at org.alfresco.repo.search.impl.lucene.fts.FullTextSearchIndexerImpl.index(FullTextSearchIndexerImpl.java:217)
        at sun.reflect.GeneratedMethodAccessor329.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
        at $Proxy79.index(Unknown Source)
        at org.alfresco.repo.search.impl.lucene.fts.FTSIndexerJob.execute(FTSIndexerJob.java:46)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)

3. JProfiler showed memory allocation mostly caused by the same Tica transformer classes.
Most memory taken by xmlbeans, poi and openxmlformats packages and allocation tree showed the same transformation job.

4. Full re-indexing done without any issues.

Can anybody suggest what else I can do and what is the reason of all that?
Is it common to have Alfresco taking almost 5G of RAM?
How can I disable CONTENT indexing for Excel files (that doesn't make sense for me)?
I believe that users can upload pretty big Excel files into repository (say 3M-10M) can it cause the issue?

Any suggestions are appreciated.

Outcomes