HTTP 502 error is thrown when more than 100 HTTP(s) calls are made using Java-backed-Webscript API

[SG]

The HTTP 502 error is thrown when more than 100 HTTP(s) calls are made using Java-backed-Webscript API to read documents from the document library.

The code works fine if there are less number of documents. But when a large number of documents exist in the document library (meaning, hundreds of HTTP calls are made), it throws 502 error.

I have attempted by giving a sleep (e.g., sleep for 5 seconds) after a certain number of HTTP calls (say after 25 calls), but I still see 502 error. I have attempted with a longer time to sleep (longer than 5 seconds), and after a smaller number of document calls (say 5 seconds sleep after 2 documents), but I still get 502 error when a large number of documents exist in the document library, and the system is "flooded" with hundreds of HTTP calls.

I am using Java-backed-webscript API only and do not plan to change it to any other API.

The code extends from org.springframework.extensions.webscripts.AbstractWebScript, and I override the method execute(WebScriptRequest request, WebScriptResponse response)

Inside the execute method, I use the NodeService API, and it recursively iterates through the folders under the document library to get the document information (document/PDF and metadata of the document).

It is a GET HTTP(S) call.

Everything works fine if there are a less number of documents under the document library (say less than 100). But when there are more than 100 documents, the system cannot seem to handle being flooded with hundreds of HTTPS-GEt calls. Again, with sleep, it still throws the 502 exception.

I 502 exception is also thrown from the Spring API (which I do not manage/control) of org.springframework.extensions.webscripts.AbstractWebScript.

The exception received is:
IOException: Server returned HTTP response code: 502 for URL (simlar to): https://mydomain.org/alfresco/service/extract/abc

My questions:

1. Anything that you may suggest at the code level so that I can continue using the Java-backed-Webscript API for hundreds of HTTP calls? As you can see, the 502 error is thrown by Spring's AbstractWebScript, and I cannot put a try-catch to re-try.

2. Any configuration that you can suggest in the Alfresco server so that it will allow this? This code (amp file) is deployed directly in a single Alfresco server and the code (driver code) is also executed from inside the same server only, where the amp is deployed.

[afaust]

There is no code in Alfresco which actively reports a 502 response status as far as I know / can see from a code search. If a 502 is reported, it must come from a custom status response / exception within the custom web script, and only be reported as coming from the abstract web script class because that is logging / re-throwing the error. You should provide the full stack trace in such issues so others can better tell if your interpretation of the error is correct and potentially point you to a more specific cause / source.

[SG]

Hello AFaust,

It is indeed true (as you have also mentioned) that the 502 exception is getting thrown by the Spring-API by the org.springframework.extensions.webscripts.AbstractWebScript, which is extended by my code of Java-backed-Alfresco-webscript.

I am attempting to call the Web service URL deployed in Alfresco-Windows-machine as:
https://mydomain.org/alfresco/service/getdata/xyz

I get 502 exception ONLY-IF the server is flooded with hundreds of calls. It is a GET call to retrieve documents and metadata. If the documentlibrary has several (e.g. over 500 or so documents in the document library folder), then only it throws 502 exception. As I noted, I gave time-out (pause/sleep after a certain number of reads and tried various combinations for the value of sleep and the number of document reads), but it still throws 502 exception if the number of files in the document library is large.

As I also noted, the driver (which makes the HTTPS call) is executed in the same Alfresco-Windows machine where the Alfresco-amp file (URL) is deployed.

The execption stacktrace is:

java.io.IOException: Server returned HTTP response code: 502 for URL: https://mydomain.org/alfresco/service/getdata/xyz
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at MyDriver.java:63
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoa
der.java:58)

It is indeed true that the exception gets thrown from the line number 63 of MyDriver.java, where I make the HTTP(s) call to the URL of the Java-webscript deployed in Alfresco-Windows. 

The MyDriver.java code is similar to:

URL url = new URL("https://mydomain.org/alfresco/service/getdata/xyz");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod(GET_REQUEST_METHOD);
conn.setRequestProperty(ACCEPT_REQUEST_PROPERTY, APPLICATION_JSON);
BufferedReader br = new BufferedReader(new InputStreamReader((conn.getInputStream())));

The line number 63 (whic throws the 502 exception is thrown) is the line:

BufferedReader br = new BufferedReader(new InputStreamReader((conn.getInputStream())));

Can you suggest:

1. Any code change in the driver, or in the Alfresc-Java-backed webscript.

2. Any configuration change in Alfresco-Windows server? It is a single Alfresco-Tomcat instance (Enterprise version 5.1.x) that runs on a Windows Server 2012, R2, 64-bit operating system.