AnsweredAssumed Answered

feedback upgrade 3.1.2 to 3.4.3 + Lucene reindex

Question asked by vgsiz on Sep 28, 2011
Hi,

just sharing some upgrade information that might be usefull for others and is difficult to find out elsewhere

kind regards

christian


version from Alfresco Enterprise 3.1.2 (390)
version migrated to Alfresco Enterprise 3.4.3 (450)
Oracle 10.2.0.4
Windows Server 2003 R2 Enterprise x64 Edition SP2
java 12Gb
hardware IBM System x3650, SAN N-series attached
1 CPU Intel Xeon L5420  2,5Ghz ( Quad core )

records of the biggest tables :
alf_node :       16 888 745
alf_node_properties :  162 540 297
alf_acl_member :    44 993 245
alf_node_aspects :   49 306 606
alf_access_control_list    7 955 157

documents in Alfresco    9 432 676
lucene index size    23Gb
database size used    70Gb
database stats gather   8hours35minutes

concurrent users :   15

upgrade
=======

+00h00 started
+00h35 group 1 ( see below ) of patches applied in table alf_applied_patch
+03h15 Applying patch 'patch.fixNameCrcValues-2' for 16892404 entries, stopped at 41%
+07h35 'patch.fixNameCrcValues-2' finished for 16892404 entries
       failed to update the CRC32 for 2 ID's due to a unique constraint
+07h36 FixUserQNamesPatch: Completed batch of 240 entries
+07h37 Authority Migration: Completed batch of 398 entries
+07h38 group 2 ( see below ) of patches applied in table alf_applied_patch
+07h38 starting Applying patch 'patch.convertContentUrls'
       (Converts pre-3.2 content URLs to use the alf_content_data table. 
       The conversion work can also be done on a schedule; please contact Alfresco Support for further details.).
       –> why is this displayed at the monitor and not in the upgrade or release notes file ????
+11h51 Patch patch.convertContentUrls - Finished processing ADM nodes up to ID 101,668,815. showing progress every 5%
+11h51 Finished processing AVM nodes up to ID 418.
+27h16 Scheduled 49,250 content URLs for deletion from store: FileContentStore
+27h17 group 3 ( see below ) of patches applied in table alf_applied_patch
+27h24 group 4 ( see below ) of patches applied in table alf_applied_patch
+32h13 group 5 ( see below ) of patches applied in table alf_applied_patch


Remarks upgrade :
=================
10 or 12Gb for java as well as a 64bit OS is necessary
-> tried with less memory for java (6Gb) and on a 32bit Windows on the test environment and crashed on patch.fixNameCrcValues-2 or patch.convertContentUrls
-> this should be mentionned in the release notes or the upgrade should not be possible

no indications on time or even the steps to go are given in any documentation not even from the Enterprise support helpdesk

shown in monitor : patch.convertContentUrls (Converts pre-3.2 content URLs to use the alf_content_data table. 
The conversion work can also be done on a schedule; please contact Alfresco Support for further details.).
-> takes 20hours to complete, should be mentionned in the upgrade / release notes that it can be scheduled after the upgrade, once the upgrade is started it's not usual bussiness to interrupt it

several new tables were created, alf_content_url had 9 432 676 inserts and alf_content_data had 9 385 655 inserts
-> should be mentionned that such big db actions are gonna take place

some db actions require big undo-space, check the db that the UNDO-tablespace is set to autoextent
( we needed +7Gb of extent tablespace, we doubled the UNDOTBS )

patch.fixNameCrcValues-2 creates a log file of over 3Gb

first remark while using ( 10users ) : getting the home page and performing a log in takes more than 3minutes
( statistics since the upgrade still needs to be executed )


reindex
=======
index.recovery.mode=FULL
2 694 350 transactions
+00h00 started
+02h52 10% complete
+05h45 20% complete
+09h02 30% complete
+27h53 40% complete
+50h22 50% complete
+78h22 60% complete
lost the log of the rest but finished between 100 and 120hours of processing

several warnings ( +1000 ) between 30 and 70% of the processing
23:42:38,328 WARN  [org.alfresco.repo.cache.TransactionalCache.org.alfresco.cache.contentDataTransactionalCache] Transactional update cache 'org.alfresco.cache.contentDataTransactionalCache' is full (1000).
23:43:07,218 WARN  [org.alfresco.repo.cache.TransactionalCache.org.alfresco.cache.node.aspectsTransactionalCache] Transactional update cache 'org.alfresco.cache.node.aspectsTransactionalCache' is full (10000).


Remarks reindexing :
====================
executed the same on a 32bit server with following error after 30% of processing and a lot of warnings :
ERROR [org.alfresco.repo.search.impl.lucene.index.IndexInfo] Index merger failed to recover - suspending java.lang.RuntimeException: Error during run with lock.
Caused by: java.io.IOException: Not enough quota is available to process this command






alf_applied_patch
=================

group 1
=======
patch.db-V3.2-Remove-AVM-Issuer         Manually executed script upgrade V3.2 to remove AVM Issuer
patch.db-V3.2-LockTables         Manually executed script upgrade V3.2: Lock Tables
patch.db-V3.2-ContentTables2         Manually executed script upgrade V3.2: Content Tables 2 (pre 3.2 Enterprise Final)
patch.db-V3.2-PropertyValueTables      Manually executed script upgrade V3.2: PropertyValue Tables
patch.db-V3.2-Upgrade-JBPM         Manually executed script upgrade V3.2 to jbpm version 3.3.1 usage
patch.db-V3.2-Child-Assoc-QName-CRC      Manually executed script upgrade V3.2 to Add qname_crc column to alf_child_assoc and change indexes
patch.db-V3.3-modify-index-permission_id   Remove context_id from the permission_id unique index (as it always contains null and therefore has no effect)
patch.db-V3.3-Remove-VersionCount      Manually executed script upgrade V3.3 to remove Version Count
patch.db-V3.2-ContentTables         Manually executed script upgrade V3.2: Content Tables
patch.db-V3.2-AuditTables         Manually executed script upgrade V3.2: Audit Tables
patch.db-V3.2-Modify-AVM-MimeType      Manually executed script upgrade V3.2 to modify AVM mimetype
patch.db-V3.3-Fix-AVM-Seqs         Manually executed script upgrade V3.3 to create AVM sequences
patch.db-V3.4-child-assoc-indexes      Manually executed script upgrade V3.4
patch.db-V3.4-authority-unique-idx      Manually executed script upgrade V3.4
patch.db-V3.4-property-unique-ctx-idx      Manually executed script upgrade V3.4
patch.db-V3.4-property-unique-ctx-value      Manually executed script upgrade V3.4
patch.db-V3.3-Node-Prop-Serializable      Manually executed script upgrade V3.3
patch.db-V3.3-Fix-Repo-Seqs         Manually executed script upgrade V3.3 to create repo sequences
patch.db-V3.4-JBPM-FK-indexes         Manually executed script upgrade to add FK indexes for JBPM
patch.db-V3.2-AddFKIndexes-2         Script fix for ALF-5396: Missing FK indexes on non-MySQL databases
patch.db-V3.4-VarcharFieldSizesQuadrupleIncreasing   Increasing VARCHAR field sizes quadruply for DB2 dialect V3.4
patch.db-V3.4-RemovingLinkValidationMetadata   Link Validation reports metadata removed V3.4
patch.db-V3.4-AVM-rename-dupes         Manually executed script upgrade V3.4
patch.db-V3.3-JBPM-Extra         Manually executed script upgrade V3.3 fix problems in JBPM tables
patch.migrateAttrTenants         Migrate old Tenant attributes
patch.thumbnailsAssocQName         Update the 'cm:thumbnails' association QName to 'rn:rendition'.


group 2
=======
patch.fixNameCrcValues-2         Fixes name and qname CRC32 values to match UTF-8 encoding.
patch.db-V3.4-UsageTables         Ensures that the database upgrade script has been run.
patch.db-V3.0-ActivityTables         Ensures that the database upgrade script has been run.
patch.zonedAuthorities            Adds the remodelled cm:authority container to the spaces store
patch.webSiteAddModerated         Changing Web Site visibility from a boolean to enum.
patch.invitationMigration         Migrate invitations from old invite service to invitation service
patch.mtShareExistingTenants         Update existing tenants for MT Share.
patch.redeployInvitationProcess         Re-deploy Invitation Process Definitions.
patch.fixUserQNames            Fixes user store qnames to improve native authentication performance
patch.authorityMigration         Copies any old authorities from the user store to the spaces store.
patch.personUsagePatch            Add person 'cm:sizeCurrent' property (if missing).
patch.authorityDefaultZonesPatch      Adds groups and people to the appropriate zones for wcm, share and everything else.
patch.imapFolders            Creates folders tree necessary for IMAP functionality
patch.redeployNominatedInvitationProcessWithPropsForShare   Redeploy nominated invitation workflow
patch.rendition.rendering_actions      Creates the Rendering Actions folder.
patch.emailInviteAndNotifyTemplatesFolder   Ensures the existence of the 'Email Invite Templates' and 'Email Notify Templates' folders.



group 3
=======
patch.convertContentUrls         Converts pre-3.2 content URLs to use the alf_content_data table.  The conversion work can also be done on a schedule; please contact Alfresco Support for further details.
patch.transferServiceFolder         Add transfer definitions folder to data dictionary.
patch.fixAuthoritiesCrcValues         Fixes authority CRC32 values to match UTF-8 encoding.
patch.updateMimetypes1            Fix mimetypes for Excel and Powerpoint.
patch.db-V3.2-AddFKIndexes         Ensures that the database upgrade script has been run.
patch.migrateAttrAVMLocks         Migrate old AVM Lock attributes
patch.migrateAttrPropBackedBeans      Migrate old Property-Backed Bean component attributes
patch.migrateAttrChainingURS         Migrate old Chaining User Registry Synchronizer attributes
patch.migrateAttrDelete            Delete old attributes (if any) after they have been migrated
patch.replication.replication_actions      Creates the Replication Actions folder.
patch.transfer.targetrulefolder         Creates the transfer target rule folder for the default transfer group.


group 4
=======
patch.transfer.targetrule         Creates the transfer target rule for the default transfer group.
patch.actions.scheduledfolder         Creates the scheduled actions folder in the Data Dictionary.
patch.updateMimetypes2            Fix mimetypes for Excel and Powerpoint.
patch.eliminateDuplicates         Fixes ALF-4203: Searches for AVM duplicate nodes and changes their name
patch.mtFixAdminExistingTenants         Fix bootstrapped creator/modifier

group 5
=======
patch.fixAclInheritance            Fixes any ACL inheritance issues.
patch.imap.messages.share         Adds email template for txt files to the Imap Configs/Templates
patch.imap.clear.old.messages         Remove old IMAP message templates

Outcomes