AnsweredAssumed Answered

In-Place bulk import issue

Question asked by hasan.bekci on Jan 25, 2017

Hello,

 

I'm trying to use the inplace bulk import tool to import a large quantity of documents (more than 1,000,000) in Alfresco.

 

The documents are already in the contentstore : /ccv/share/alf_data/alfespopIN/alf_data/contentstore/medusa so I can use the inplace bulk import.

 

The documents are sorted by years :

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/1900

...

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/2016

 

When I try to import the whole contentstore in once, i have the folowing error after a few hours :

 

java.lang.NullPointerException
         at java.util.Arrays$ArrayList.(Arrays.java:3357)
         at java.util.Arrays.asList(Arrays.java:3343)
         at org.alfresco.repo.bulkimport.AnalysedDirectory.(AnalysedDirectory.java:49)
         at org.alfresco.repo.bulkimport.impl.DirectoryAnalyserImpl.analyseDirectory(DirectoryAnalyserImpl.java:124)
         at org.alfresco.repo.bulkimport.impl.AbstractFilesystemTracker.getImportableItemsInDirectory(AbstractFilesystemTracker.java:60)
         at org.alfresco.repo.bulkimport.impl.StripingFilesystemTracker.getImportableItems(StripingFilesystemTracker.java:146)
         at org.alfresco.repo.bulkimport.impl.StripingFilesystemTracker$1.getNextWork(StripingFilesystemTracker.java:202)
         at org.alfresco.repo.batch.BatchProcessor$WorkProviderIterator.hasNext(BatchProcessor.java:579)
         at org.alfresco.repo.batch.BatchProcessor.process(BatchProcessor.java:387)
         at org.alfresco.repo.bulkimport.impl.StripingBulkFilesystemImporter.bulkImportImpl(StripingBulkFilesystemImporter.java:64)
         at org.alfresco.repo.bulkimport.impl.AbstractBulkFilesystemImporter$3.execute(AbstractBulkFilesystemImporter.java:395)
         at org.alfresco.repo.bulkimport.impl.AbstractBulkFilesystemImporter$3.execute(AbstractBulkFilesystemImporter.java:371)
         at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:388)
         at org.alfresco.repo.bulkimport.impl.AbstractBulkFilesystemImporter.bulkImport(AbstractBulkFilesystemImporter.java:370)
         at org.alfresco.repo.bulkimport.impl.AbstractBulkFilesystemImporter$2$1.doWork(AbstractBulkFilesystemImporter.java:351)
         at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:529)
         at org.alfresco.repo.bulkimport.impl.AbstractBulkFilesystemImporter$2.run(AbstractBulkFilesystemImporter.java:347)
         at java.lang.Thread.run(Thread.java:662)

 

Given this exception, I first thought that the problem was an empty directory, but there is no empty directories.

 

So then I started to do the import step by step to find the document, or the folder, that causes this error.

 

I separated the documents in two like this :

 

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/1900

...

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/1950

and

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/B/1951

...

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/B/2016

 

The import of /ccv/share/alf_data/alfespopIN/alf_data/contentstore/A also failed, with the same error.

 

After separating the documents in two again :

 

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/A1/1900

...

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/A1/1925

and

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/A2/1926

...

/ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/A2/1950

 

The import of /ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/A1 and the import of /ccv/share/alf_data/alfespopIN/alf_data/contentstore/A/A2 both worked...

 

In the other half of the documents, 1951 to 2016, there are more documents and I have to go deeper in the sorting in order to make the import work. For some years I have to do it year by year and the advantage of a bulk import is lost.

 

Have anyone already seen this error? Any help would be much appreciated.

 

The version of Alfresco is Alfresco Enterprise v4.1.2

 

Hasan.

Outcomes