AnsweredAssumed Answered

Moving large datasets around Alfresco

Question asked by bengrah on Sep 24, 2013
Latest reply on Sep 25, 2013 by bengrah
Hi all,

Got a question around moving large datasets around Alfresco. We're in the middle of migrating large datasets into our on-premise installation, these can be around 500Gb on average. We migrate the data in using the Bulk FS tool that comes with Alfresco and it works quite well for the most part.

Ocasionally we may import files into the wrong area, and we need to move the folders into the correct area after the data has been imported into Alfresco.

This is where we get stuck. For example, right now we've got an 18Gb folder we need to move. This is our experience through the various interfaces:

<ul>Explorer: Cutting and Pasting the folder from one place to another appears to work (the page appears to load as standard as it's executing the command) but does not give any indication of time it'll take to complete. After half an hour there is no evidence that the move has taken place (we had a different window open to the target directory)</ul>
<ul>Share: Using the Move to… command on the folder starts the move process, along with the Moving files… prompt. This prompt disappears after 1-2 minutes. There's no indication of a file being moved. As above, no data appeared in the target directory</ul>
<ul>CIFS: Far too slow, and occasionally we're getting timeouts here</ul>
<ul>WebDAV: Same as CIFS</ul>
<ul>FTP: Appears to make no progress after half an hour of waiting</ul>

We'll most likely just have to re-import the data back into Alfresco only this time in the correct location, but it's data migrations are not the only context that we encounter the above problems.

Does anyone else have similar experiences? Or better yet, seen any solutions / using any solutions?

I do appreciate I am talking about relatively big data sets here, so there's a certain degree of expectation.

Outcomes