I often encounter this error in my logs: ClosedStreamException: received a RST_stREAM frame: CANCEL

[liuhao710118]

skywalking v9.7
elasticsearch：v8.14

I want to know what caused this error. I can't find any relevant information online, so I would like to inquire here

Here are the key parts of my log

2024-08-29 16:30:19,455 - com.linecorp.armeria.server.logging.LoggingService - 95 [armeria-eventloop-epoll-5-2] WARN  [] - [sreqId=b841785b, chanId=82ef2569, raddr=172.16.251.172:60706, laddr=172.16.251.172:12800][h2c://localhost.localdomain/internal/l7check#HEAD] Request: {startTime=2024-08-29T08:29:28.800Z(1724920168800483), length=0B, duration=50579ms(50579404976ns), scheme=none+h2c, name=HEAD, headers=[EOS, :method=HEAD, :authority=172.16.251.172:12800, :scheme=http, :path=/internal/l7check, if-none-match="healthy", prefer=wait=60, user-agent=armeria/1.25.0]}
2024-08-29 16:30:19,455 - com.linecorp.armeria.server.logging.LoggingService - 156 [armeria-eventloop-epoll-5-2] WARN  [] - [sreqId=b841785b, chanId=82ef2569, raddr=172.16.251.172:60706, laddr=172.16.251.172:12800][h2c://localhost.localdomain/internal/l7check#HEAD] Response: {startTime=2024-08-29T08:29:28.800Z(1724920168800742), length=0B, duration=50579ms(50579672744ns), totalDuration=50579ms(50579933069ns), cause=com.linecorp.armeria.common.stream.ClosedStreamException: received a RST_STREAM frame: CANCEL, headers=[:status=0]}
com.linecorp.armeria.common.stream.ClosedStreamException: received a RST_STREAM frame: CANCEL
	at com.linecorp.armeria.server.Http2RequestDecoder.onRstStreamRead(Http2RequestDecoder.java:396) [armeria-1.25.0.jar:?]
	at io.netty.handler.codec.http2.Http2FrameListenerDecorator.onRstStreamRead(Http2FrameListenerDecorator.java:59) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.Http2MaxRstFrameListener.onRstStreamRead(Http2MaxRstFrameListener.java:56) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.Http2FrameListenerDecorator.onRstStreamRead(Http2FrameListenerDecorator.java:59) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onRstStreamRead(DefaultHttp2ConnectionDecoder.java:442) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onRstStreamRead(Http2InboundFrameLogger.java:80) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readRstStreamFrame(DefaultHttp2FrameReader.java:509) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:259) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:159) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:173) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DecoratingHttp2ConnectionDecoder.decodeFrame(DecoratingHttp2ConnectionDecoder.java:63) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.DecoratingHttp2ConnectionDecoder.decodeFrame(DecoratingHttp2ConnectionDecoder.java:63) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:393) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:453) [netty-codec-http2-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) [netty-codec-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) [netty-codec-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) [netty-codec-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) [netty-handler-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) [netty-handler-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) [netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) [netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) [netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.100.Final.jar:4.1.100.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.100.Final.jar:4.1.100.Final]
	at java.lang.Thread.run(Thread.java:829) [?:?]

[wu-sheng]

This usually means connection disconnected from the client side somehow.

[liuhao710118]

I found that when I query indicator data for 7 days or more, this exception occurs, but when I query data for 3 days or less, this exception does not occur

When I query the indicator data interface "/graphql" for 7 days or more, the 503 service is unavailable

[wu-sheng]

I can't see much different. Maybe your database(I don't know which you chose) is slow or something.

[xbchen711]

I found that when I query indicator data for 7 days or more, this exception occurs, but when I query data for 3 days or less, this exception does not occur

When I query the indicator data interface "/graphql" for 7 days or more, the 503 service is unavailable

I meet the same error, resulting graphql 503 response, any ideas to resolve this problem?

[liuhao710118]

I haven't found a way to deal with this problem.