Repository navigation
Conversation
A Linux peer ACKs every segment that carries PSH at once, so setting PSH on every data segment makes a sender with several segments queued get one ACK per segment. - ipconfigTCP_PSH_ON_LAST_SEGMENT_ONLY sets PSH only on the segment that carries the last queued byte, as other stacks do. - It is off by default, and nothing changes while it is off.
htibosch
left a comment
There was a problem hiding this comment.
@comotion , thank you for sharing this change.
You are right, the throughput increases dramatically when tcpTCP_FLAG_PSH is set when the last segment has been sent. It saves ACKs: less bandwidth needed.
We chose to set PSH in every packet that contains TX data.
I just ran iperf3, and tried it as: iperf3 -c 192.168.1.3 -4 --port 5001 --bytes 100M -R.
-R makes that the DUT will send packets to the client. The performance looked good.
Changing this behaviour means that the counterpart must be able to accommodate incoming packets that have not been consumed.
Although, the counterpart can also just ignore the flag and consume the data.
I just see a WireShark option, when you view a TCP packet:
[SEQ/ACK analysis]
[iRTT: 1.120900 milliseconds]
[Bytes in flight: 1460]
[Bytes sent since last PSH flag: 5840]
Should we limit what we send "Bytes sent since last PSH flag" by counting them and insert a PSH flag?
I want to play more with the change and see how it behaves before approving it.
Thanks,
Hein
|
Hi @htibosch and thanks for taking the time to review and test. You have a point that if an application keeps the send buffer busy, this change never sets a PSH and some receivers could hold the data until a PSH arrives. Linux reads the data straight away, Windows might wait for a PSH in some cases, I haven't tested. We can guard against this in the same way Linux does in |
A sender that keeps its queue full never empties it, so PSH would never be set. A receiver that waits for PSH before it hands data to the application then holds the data until its buffer fills. Linux sets PSH in the same case.
The proof now also runs with PSH set only on the last segment, so CI checks the code that the option compiles in.
|
@comotion I like how you used of the circular buffers: roughly it says: if( this is likely the last TX packet )
{
Set the PSH flag
}For my self and others, a short comment on TCP sliding windows and circular buffers. When implementing TCP with sliding windows, we didn't want to use pbuf, in stead each Ethernet packet is stored in a contiguous block, not in linked chunks. Each TCP socket may allocate two circular buffers, one for RX and one for TX. Their sizes are configured: Note that you can configure all properties with the socket option Back to the circular buffers: As soon as " Note this function: size_t uxStreamBufferGetSize( const StreamBuffer_t * const pxBuffer )
{
return uxStreamBufferDistance( pxBuffer, pxBuffer->uxTail, pxBuffer->uxHead );
}When the distance is 0, the buffer is empty const size_t uxLength = pxBuffer->LENGTH;
size_t uxCount = uxLength + uxHead - uxTail;
if( uxCount >= uxLength )
{
uxCount -= uxLength;
}
return uxCount;The lDataLen = ( int32_t ) ulTCPWindowTxGet( pxTCPWindow, pxSocket->u.xTCP.ulWindowSize, &lStreamPos );lDataLen : The number of bytes that can be sent if( lDataLen != 0L )
{
size_t uxDistance = uxStreamBufferDistance( txStream, ( size_t ) lStreamPos, txStream->uxHead );
if( uxDistance == ( size_t ) lDataLen )
{
pxProtocolHeaders->xTCPHeader.ucTCPFlags |= ( uint8_t ) tcpTCP_FLAG_PSH;
}
}Here an overview of the cyclic buffers and markers:
|

Set PSH only on the segment that empties the send queue
Description
A Linux peer ACKs every segment that carries PSH at once, so setting PSH on every data segment makes a sender with several segments queued get one ACK per segment. The stack has to process each ACK in order, which takes CPU time from sending, reducing throughput.
Test Steps
Measure transmit speeds for MSS, 16KB and 256KB transmits.
My measurements are on a 125MHz Nios-V with a msgDMA ethernet driver and a 10GbE PHY.
On TX, 16KB sends, this is 27% faster than baseline, and on 256KB sends, 28% faster.
Single-MSS sends are unchanged.
Checklist:
Related Issue
#1380
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.