AnsweredAssumed Answered

Unable to import large amounts of files into alfresco.

Question asked by m_mind on Jul 7, 2009
Latest reply on Sep 8, 2009 by kevinr
We currently have about 32G of files which we currently have on a windows share. As we want to be able to index and search for files, we want to add these files into Alfresco.
However, when we copy the files to the CIFS interface or to the FTP server, after some time errors occur and the connection is closed.
We need to restart Alfresco in order to be able to use the Fileserver interface again.

The error I get when importing through the CIFS interface is:

09:18:41,168 ERROR [org.alfresco.util.transaction.SpringAwareUserTransaction.trace] UserTransaction being garbage collected without a commit() or rollback().
   Started at:
      org.alfresco.util.transaction.SpringAwareUserTransaction.begin(SpringAwareUserTransaction.java:389)
      org.alfresco.filesys.alfresco.AlfrescoDiskDriver.beginTransaction(AlfrescoDiskDriver.java:332)
      org.alfresco.filesys.alfresco.AlfrescoDiskDriver.beginWriteTransaction(AlfrescoDiskDriver.java:180)
      org.alfresco.filesys.repo.ContentDiskDriver.closeFile(ContentDiskDriver.java:1822)
      org.alfresco.jlan.smb.server.NTProtocolHandler.procCloseFile(NTProtocolHandler.java:1386)
      org.alfresco.jlan.smb.server.NTProtocolHandler.runProtocol(NTProtocolHandler.java:251)
      org.alfresco.jlan.smb.server.SMBSrvSession.runHandler(SMBSrvSession.java:1344)
      org.alfresco.jlan.smb.server.SMBSrvSession.processPacket(SMBSrvSession.java:1436)
      org.alfresco.jlan.smb.server.nio.NIOCIFSThreadRequest.runRequest(NIOCIFSThreadRequest.java:105)
      org.alfresco.jlan.server.thread.ThreadRequestPool$ThreadWorker.run(ThreadRequestPool.java:141)
      java.lang.Thread.run(Thread.java:619)
but also a longer exception:

11:25:53,320 ERROR [org.hibernate.AssertionFailure] an assertion failure occured (this may indicate a bug in Hibernate, but is more likely due to unsafe use o
f the session)
org.hibernate.AssertionFailure: null id in org.alfresco.repo.domain.hibernate.ChildAssocImpl entry (don't flush the Session after an exception occurs)
        at org.hibernate.event.def.DefaultFlushEntityEventListener.checkId(DefaultFlushEntityEventListener.java:55)
        at org.hibernate.event.def.DefaultFlushEntityEventListener.getValues(DefaultFlushEntityEventListener.java:164)
        at org.hibernate.event.def.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:120)
        at org.hibernate.event.def.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:196)
        at org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:76)
        at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:26)
        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
        at org.alfresco.repo.domain.hibernate.DirtySessionMethodInterceptor.flushSession(DirtySessionMethodInterceptor.java:317)
        at org.alfresco.repo.domain.hibernate.DirtySessionMethodInterceptor.flushSession(DirtySessionMethodInterceptor.java:282)
        at org.alfresco.repo.node.db.hibernate.HibernateNodeDaoServiceImpl.newChildAssoc(HibernateNodeDaoServiceImpl.java:1588)
        at sun.reflect.GeneratedMethodAccessor659.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:296)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:177)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at org.alfresco.repo.transaction.TransactionalDaoInterceptor.invoke(TransactionalDaoInterceptor.java:68)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.domain.hibernate.DirtySessionMethodInterceptor.invoke(DirtySessionMethodInterceptor.java:381)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy2.newChildAssoc(Unknown Source)
        at org.alfresco.repo.node.db.DbNodeServiceImpl.createNode(DbNodeServiceImpl.java:319)
        at sun.reflect.GeneratedMethodAccessor528.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:296)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:177)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at org.alfresco.repo.transaction.SingleEntryTransactionResourceInterceptor.invokeInternal(SingleEntryTransactionResourceInterceptor.java:169)
        at org.alfresco.repo.transaction.SingleEntryTransactionResourceInterceptor.invoke(SingleEntryTransactionResourceInterceptor.java:138)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy7.createNode(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor528.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:296)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:177)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at org.alfresco.repo.tenant.MultiTNodeServiceInterceptor.invoke(MultiTNodeServiceInterceptor.java:110)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy7.createNode(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor528.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.alfresco.repo.service.StoreRedirectorProxyFactory$RedirectorInvocationHandler.invoke(StoreRedirectorProxyFactory.java:221)
        at $Proxy8.createNode(Unknown Source)
        at org.alfresco.repo.node.MLPropertyInterceptor.invoke(MLPropertyInterceptor.java:279)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.node.NodeRefPropertyMethodInterceptor.invoke(NodeRefPropertyMethodInterceptor.java:188)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.node.NodeRefPropertyMethodInterceptor.invoke(NodeRefPropertyMethodInterceptor.java:188)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy7.createNode(Unknown Source)
        at org.alfresco.repo.model.filefolder.FileFolderServiceImpl.create(FileFolderServiceImpl.java:729)
        at sun.reflect.GeneratedMethodAccessor688.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:296)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:177)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at org.alfresco.repo.model.filefolder.TempFileMarkerInterceptor.invoke(TempFileMarkerInterceptor.java:83)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.model.ml.MLContentInterceptor.invoke(MLContentInterceptor.java:131)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.model.filefolder.MLTranslationInterceptor.invoke(MLTranslationInterceptor.java:210)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:49)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.alfresco.repo.audit.AuditComponentImpl.audit(AuditComponentImpl.java:275)
        at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:69)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy53.create(Unknown Source)
        at org.alfresco.filesys.repo.CifsHelper.createNode(CifsHelper.java:367)
        at org.alfresco.filesys.repo.ContentDiskDriver.createDirectory(ContentDiskDriver.java:1656)
        at org.alfresco.jlan.smb.server.CoreProtocolHandler.procCreateDirectory(CoreProtocolHandler.java:479)
        at org.alfresco.jlan.smb.server.CoreProtocolHandler.runProtocol(CoreProtocolHandler.java:3709)
        at org.alfresco.jlan.smb.server.NTProtocolHandler.runProtocol(NTProtocolHandler.java:359)
        at org.alfresco.jlan.smb.server.SMBSrvSession.runHandler(SMBSrvSession.java:1344)
        at org.alfresco.jlan.smb.server.SMBSrvSession.processPacket(SMBSrvSession.java:1436)
        at org.alfresco.jlan.smb.server.nio.NIOCIFSThreadRequest.runRequest(NIOCIFSThreadRequest.java:105)
        at org.alfresco.jlan.server.thread.ThreadRequestPool$ThreadWorker.run(ThreadRequestPool.java:141)
        at java.lang.Thread.run(Thread.java:619)

I've also tried to create a relatively small ZIP file (about 160MB) in order to import the files like this, but that also doesn't work.
Is there a way to import large amounts of files without crashing Alfresco?

Thanks in advance.
Peter

Outcomes