AnsweredAssumed Answered

RepositoryServiceSoapBindingStub.fetchMore() method

Question asked by pitt1 on Mar 7, 2007
Latest reply on Apr 16, 2008 by kdejaeger
I would like to programmatically grab results from a lucene query in chunks, and furthermore would like to be able to make sure that I can grab all search results.  That is, if there are 2950 results and I want to retrieve them in batches of 100, I want to be able to make 30 successive calls to retrieve the next batch of results.

First I noticed that no matter what I did with web services (or even running the query in the node browser), search results were constrainted to 1000.  Support directed me to parameters in repository.properties that allow more than 1000 results to come back:

# The maximum time spent pruning results
system.acl.maxPermissionCheckTimeMillis=100000
# The maximum number of results to perform permission checks against
system.acl.maxPermissionChecks=5000

(Default values for these properties are 10000 and 1000, respectively).  After changing these, my web service query returned more than 1000 results.  While this is helpful, it does not solve the problem of controlling results returned via the API.

I noticed that there is a RepositoryServiceSoapBindingStub.fetchMore() method, which, I surmised, must allow batched fetching of results.  I tried the following code:



    QueryResult queryResult = repositoryService.query( store , query, false );
    String querySession = queryResult.getQuerySession();

    ResultSet resultSet = queryResult.getResultSet();
    ResultSetRow[] rows = resultSet.getRows();
   
    if (rows!= null) {
        System.out .println( "RESULTS: there are "+rows. length + " results");
        QueryResult q = repositoryService .fetchMore(querySession);
        ResultSet rs = q.getResultSet();
        ResultSetRow[] r = rs.getRows();
        System.out .println( "RESULTS: there are "+r. length + " results in the second batch");
}

This code was run with max permission checks set to 200, even though there were over 500 results to my query, so I knew that there should be more results to fetch.  Axis complained rather badly when I ran this code.

Client-side error:
ERROR: ; nested exception is:
org.xml.sax.SAXParseException : Premature end of file.
AxisFault
faultCode: { http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: org.xml.sax.SAXParseException : Premature end of file.
faultActor:
faultNode:
faultDetail:
{ http://xml.apache.org/axis/ }stackTrace:org.xml.sax.SAXParseException : Premature end of file.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(Unknown Source)
at org.apache.axis.encoding.DeserializationContext.parse( DeserializationContext.java:227 )
at org.apache.axis.SOAPPart.getAsSOAPEnvelope( SOAPPart.java:696 )
at org.apache.axis.Message.getSOAPEnvelope( Message.java:435 )
at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke( MustUnderstandChecker.java:62 )
at org.apache.axis.client.AxisClient.invoke( AxisClient.java:206 )
at org.apache.axis.client.Call.invokeEngine( Call.java:2784 )

Server-side error:
14:13:04,487 ERROR [org.apache.axis.Message] java.io.IOException:
AxisFault
faultCode: { http://schemas.xmlsoap.org/soap/envelope/}Server.generalException
faultSubcode:
faultString:
faultActor:
faultNode:
faultDetail:
        { http://xml.apache.org/axis/ }exceptionName:org.alfresco.repo.webservice.repository.RepositoryFault
        { http://xml.apache.org/axis/}stackTrace:
        at org.alfresco.repo.webservice.repository.RepositoryWebService.fetchMore(RepositoryWebService.java:465)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
        at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
        at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
        at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

So my questions are:

Is fetchMore() supposed to do what I want?  If so, what am I doing wrong?  If not, is there a way to set the batch size and max total number of returned results via web services?

Outcomes