AnsweredAssumed Answered

out of heap space when uploading large files

Question asked by alanbell on Dec 24, 2010
Latest reply on Aug 30, 2013 by victorba
I am using the python-cmis library to upload files to Alfresco community 3.4.b running on Ubuntu server.
where /mnt/import/bigfile is a 298MB file. The file can be imported successfully using the share interface or classic Alfresco web UI and small files of a few MB in size are uploaded by this code successfully.

the python code is as follows:

from cmislib.model import CmisClient
cmisClient = CmisClient('http://localhost:8080/alfresco/s/cmis', 'importuser','password')
repo = cmisClient.defaultRepository
ImportFolder = repo.getObjectByPath("/import")
instream=open("/mnt/import/bigfile","r")
d=ImportFolder.createDocument("bigfile",contentFile=instream)
and it falls over leaving this error:
Traceback (most recent call last):
  File "mapimport.py", line 12, in <module>
    d=ImportFolder.createDocument("bigfile",contentFile=instream)
  File "/usr/local/lib/python2.6/dist-packages/cmislib-0.3-py2.6.egg/cmislib/model.py", line 2700, in createDocument
    result = self._cmisClient.post(postUrl, xmlDoc.toxml(), ATOM_XML_ENTRY_TYPE)
  File "/usr/local/lib/python2.6/dist-packages/cmislib-0.3-py2.6.egg/cmislib/model.py", line 243, in post
    self._processCommonErrors(result)
  File "/usr/local/lib/python2.6/dist-packages/cmislib-0.3-py2.6.egg/cmislib/model.py", line 296, in _processCommonErrors
    raise RuntimeException(error.status)
cmislib.exceptions.RuntimeException: 500

and in catalina.out (first error from a clean restart of Alfresco)

INFO: Server startup in 76334 ms
23:38:52,675  ERROR [extensions.webscripts.AbstractRuntime] Exception from executeScript - redirecting to status template error: 11230001 Wrapped Exception (with status template): 11230016 Failed to execute script
'classpath*:alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js': 11230015 Java heap space
org.springframework.extensions.webscripts.WebScriptException: 11230001 Wrapped Exception (with status template): 11230016 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/cmis/childre
n.post.atom.js': 11230015 Java heap space
        at org.springframework.extensions.webscripts.AbstractWebScript.createStatusException(AbstractWebScript.java:749)
        at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:171)
        at org.alfresco.repo.web.scripts.RepositoryContainer$2.execute(RepositoryContainer.java:383)
        at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:377)
        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:436)
        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:466)
        at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:304)
        at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:333)
        at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:189)
        at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:619)
Caused by: org.alfresco.scripts.ScriptException: 11230016 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js': 11230015 Java heap space
        at org.alfresco.repo.jscript.RhinoScriptProcessor.execute(RhinoScriptProcessor.java:194)
        at org.alfresco.repo.processor.ScriptServiceImpl.executeScript(ScriptServiceImpl.java:282)
        at org.alfresco.repo.web.scripts.RepositoryScriptProcessor.executeScript(RepositoryScriptProcessor.java:102)
        at org.springframework.extensions.webscripts.AbstractWebScript.executeScript(AbstractWebScript.java:982)
        at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:86)
        … 24 more

Caused by: org.alfresco.error.AlfrescoRuntimeException: 11230015 Java heap space
        at org.alfresco.repo.jscript.RhinoScriptProcessor.executeScriptImpl(RhinoScriptProcessor.java:488)
        at org.alfresco.repo.jscript.RhinoScriptProcessor.execute(RhinoScriptProcessor.java:190)
        … 28 more
Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3209)
        at java.lang.String.<init>(String.java:215)
        at java.lang.StringBuffer.toString(StringBuffer.java:585)
        at com.ctc.wstx.util.TextBuffer.contentsAsString(TextBuffer.java:403)
        at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:821)
        at org.apache.axiom.om.impl.builder.StAXBuilder.createOMText(StAXBuilder.java:245)
        at org.apache.axiom.om.impl.builder.StAXBuilder.createOMText(StAXBuilder.java:216)
        at org.apache.abdera.parser.stax.FOMBuilder.applyTextFilter(FOMBuilder.java:158)
        at org.apache.abdera.parser.stax.FOMBuilder.next(FOMBuilder.java:206)
        at org.apache.axiom.om.impl.llom.OMElementImpl.getNextOMSibling(OMElementImpl.java:265)
        at org.apache.axiom.om.impl.traverse.OMChildrenQNameIterator.hasNext(OMChildrenQNameIterator.java:69)
        at org.apache.axiom.om.impl.llom.OMElementImpl.getFirstChildWithName(OMElementImpl.java:223)
        at org.apache.abdera.parser.stax.FOMExtensibleElement.getExtension(FOMExtensibleElement.java:91)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
        at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:243)
        at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:66)

        at org.mozilla.javascript.gen.c9._c0(file:/opt/alfresco-3.4.b/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js:541)
        at org.mozilla.javascript.gen.c9.call(file:/opt/alfresco-3.4.b/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js)
        at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
        at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2834)
        at org.mozilla.javascript.gen.c9.call(file:/opt/alfresco-3.4.b/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js)
        at org.mozilla.javascript.gen.c9.exec(file:/opt/alfresco-3.4.b/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/templates/webscripts/org/alfresco/cmis/children.post.atom.js)
        at org.alfresco.repo.jscript.RhinoScriptProcessor.executeScriptImpl(RhinoScriptProcessor.java:472)
        at org.alfresco.repo.jscript.RhinoScriptProcessor.execute(RhinoScriptProcessor.java:190)
        at org.alfresco.repo.processor.ScriptServiceImpl.executeScript(ScriptServiceImpl.java:282)
        at org.alfresco.repo.web.scripts.RepositoryScriptProcessor.executeScript(RepositoryScriptProcessor.java:102)
        at org.springframework.extensions.webscripts.AbstractWebScript.executeScript(AbstractWebScript.java:982)
        at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:86)
so the python code gets an error 500 from Alfresco CMIS, I don't think there is anything wrong on the python side as such.
I tried editing /opt/alfresco-3.4.b/tomcat/scripts/ctl.sh to give java more heap size

    export JAVA_OPTS="-XX:MaxPermSize=512m -Xms2G -Xmx3G -Dalfresco.home=/opt/alfresco-3.4.b -Dcom.sun.management.jmxremote"
but that made no visible difference to the outcome. The server has 8GB of ram assigned to it and is virtualised.

Can anyone else reproduce this, or successfully upload large files to CMIS?
Is there something I can do to increase the heap size or tune things so that it works?
Is there another way I should be doing this?

Outcomes