Provide timeout options when WLS is taking forever returning results.

[vkhanork]

Context:
In our production environment, due to some issue(s) at the WLS end, the web logic exporter is taking forever returning results for /metrics query.

Expectation:
The wls exporter should provide a timeout configuration option after which the exporter terminates gracefully, along with an exception.

Stacktrace:
<May 15, 2020 8:42:37,126 PM UTC> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "621" seconds working on the request "Http Request Information: weblogic.servlet.internal.ServletRequestImpl@2ad499d6[GET /wls-exporter/metrics] ", which is more than the configured time (StuckThreadMaxTime) of "600" seconds in "server-failure-trigger". Stack trace: java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.socketRead(SocketInputStream.java:116) java.net.SocketInputStream.read(SocketInputStream.java:171) java.net.SocketInputStream.read(SocketInputStream.java:141) org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) io.prometheus.wls.rest.WebClientImpl.getReply(WebClientImpl.java:75) io.prometheus.wls.rest.WebClientImpl.sendRequest(WebClientImpl.java:66) io.prometheus.wls.rest.WebClientImpl.doPostRequest(WebClientImpl.java:96) io.prometheus.wls.rest.ExporterServlet.getMetrics(ExporterServlet.java:89) io.prometheus.wls.rest.ExporterServlet.displayMetrics(ExporterServlet.java:67) io.prometheus.wls.rest.ExporterServlet.printMetrics(ExporterServlet.java:61) io.prometheus.wls.rest.ExporterServlet.displayMetrics(ExporterServlet.java:54)

[stephgeddis]

Hi vkhanork, I have been having similar issues to this when running off a local host weblogic instance. Did you ever find a way to resolve this?

[vkhanork]

@stephgeddis unfortunately no.

[ddsharpe]

The exporter should not default to waiting indefinitely on responses. Fix needed to add default timeout and enhancement for configurable timeout value.

[udayreddyd]

Hi Monica,
We are seeing similar issue in our production env. Could you please provide if there is any fix for this issue. Thank you.

Regards,
Uday