AnsweredAssumed Answered

lucene search returns less than max results

Question asked by tobikl1 on Oct 28, 2016
Latest reply on Nov 2, 2016 by tobikl1

Hi, i have a question regarding the lucene search and moving / renaming nodes.

 

We are moving some folders from one location to a backup location and then recreate the same folderstructure by copying from a Space template to the base folder again. we then move existing files from the old folder structure to this newly created one in a javascript webscript.

 

The problem we are now facing is that when we call search.query with a lucene search term and a configured maximum resultset of e.g. 10, at first we get the 10 results, but after calling this javascript webscript, even though we would expect to still get 10 maximum results afterwards, we only get 9 results afterwards.

 

So we have lots of folders with a specific folderstructure inside and rules applied to several of this folders. Since on our production server this has become outdated we are trying to rebuild this folderstructures from scrath, yet the original files contained in them should remain the same. For this we move all the files and folders from the base folder to a backup folder. Then we recreate the folderstructure by copying from a Space Templates Folder and afterwards move some existing files from the backup to the newly created subfolder.

 

E.g.:

 

We have the Folder "Apps/<App_Name>/Channels/General" containing a file "<App_Name>.xml"

 

In the script, we create a folder "Apps Backup/<App_Name>_backup_<timestamp>" and move the folder "Apps/<App_Name>/Channels" to the folder "Apps Backup/<App_Name>_backup_<timestamp>".

In the same script we then recreate the folder structure in "Apps/<App_Name>" by copying "Channels/General" from a Space Template Folder. Then we move the "<App_Name>.xml" file from the General Folder moved to the backup by now to the newly created General folder.

 

in "Apps" we have 120k+ Folders of the same structure, yet over the time we have added new features to our apps and therefor new rules got applied to the different folders inside. Since in javascript we do not have direct access to the ruleservice, we thought it would be a viable way, to update rules on the folders by just recreating them with the most recent rules configured on the space templates folders.

 

When starting a search for given xml files with search.query by searching for the path + specialized type of xml file and limiting the maximum results to 10, we get a resultset of 10 of this xml files as expected. but once we run the described script on one of the folders containing one of the xml files returned by this search, on the next search configured with 10 max results we only get 9 results afterwards, yet the search should still return 10 results always, since we have more than 120k of this xml files. i would suggest the lucene search messed something up and gets some kind of db-id afterwards which it cannot connect to a node id anymore and therefore just discards if of the search result leaving us with only 9 nodes returned?

 

I am not aware on how this could happen and hope you could give me some insight.

 

the query we use is

{ 
   "query": "PATH:\"/app:company_home/cm:Apps/cm:*/cm:Channels/cm:General/cm:*\" AND TYPE:\"sbd:sam\" AND ASPECT:\"praxis:baseSAM\" AND @praxis\\:visible:\"false\" AND @sbd\\:city:\"Köln\"",
   "sort": [   {
        "column": "name",
        "ascending": true
   }   ],
   "language": "lucene",
   "page": {
        "maxItems": 10,
        "skipCount": 0
}

Best regards,

 

Tobias Kleigrewe

Outcomes