RST with not matching sequence number causes continuously sending ACK/RST

[mj-servus]

Describe the bug
The target and the host remain in a loop continuously sending RST and challenge ACK.

The reason is a RST packet from the HOST with the sequence number inside the current receive window, but not exactly matching the expected value.
In that case the target responds with an challenge ACK (https://datatracker.ietf.org/doc/html/rfc5961#section-3.2). The received frame is the base for the challenge ACK, so the sequence number sent by the host is used for the challenge ACK message. As the host already closed the connection the response is again a RST with the same sequence number.

The ping pong of the messages is done until an other message is sent to the host, including the expected sequence number. The RST response of the other message finally causes the connection to close.

Target

• FreeRTOS-Plus-TCP network stack version 4.2.0
• Development board: Proprietary board with STM32H7 CPU and RS9116 WLAN module.
• Instruction Set Architecture: cortex-m7
• Toolchain and version: GNU Arm Embedded Toolchain 10.3-2021.10

Host

• Host OS: Windows Server 2016 Datacenter
• Version: 1607

To Reproduce

• Establish a connection to the host
• Interrupt the data transfer for about 25s (WLAN disconnect)
• Cause the host to abort the connection (in our case implicit by the Websocket server)
• Continue the data transfer

Expected behavior
In case a RST frame with the sequence number inside the current receive window is received, one challenge ACK should be sent. In case the host responds again with an RST the connection should be closed.

Screenshots

Wireshark logs
The Wireshark logs where taken at the target.
Capture: RST_ACK.zip
Target: 10.1.126.248
Host: 10.1.126.11

Additional context
I did an experiment using the expected sequence number in the challenge ACK message. By doing so, the host responded with a RST with the expected sequence number and the connection was closed after the second RST.

[tony-josi-aws]

@mj-servus

Thanks for contributing to FreeRTOS+TCP by creating this bug report and sharing your findings.

It seems like when the challenge ACK is sent, the sequence number and the acknowledge number of the initial RST packet are swapped and are used in the challenge ACK and are not tracked anywhere.

When the new RST comes, its sequence number is compared with the existing RX.NXT instead of the acknowledgment number of the challenge ACK. I believe the RST after challenge ACK needs to be handled separately and should be compared against the acknowledgment number of challenge ACK to close the connection:

   The sequence
   number of the second RST is derived from the acknowledgment number of
   the incoming ACK.  This second RST, if it reaches the sender, will
   cause the connection to be aborted since the sequence number would
   now be an exact match.

Since you have done your experiment, do you wish to contribute your changes as a pull request?

[mj-servus]

I added the changes of my experiment as pull request: #1236

[tony-josi-aws]

Closing this issue as the PR is merged