Optimize TCP Channel allocations by using pooled IValueTaskSource for Operation type and CancellationTokenSource for timeout handling

[romanett]

As of net 8 we can use ValueTasks overload of the Socket async methods:

• Socket.ConnectAsync
• Socket.ReceiveAsync
• Socket.SendAsync

This would allow more efficient sending & receiving of messages with less allocations.

Send & Receive are in the hot path, so this will improve allocations for most usecases

[marcschier]

ConnectAsync is already being used, Send/Receive do not easily benefit from ValueTask as they are full duplex operations, however, the Operation class inside the channel would benefit from becoming a IValueTaskSource just like on the server, instead of using TaskCompletionSource, now that all operations are async.

[romanett]

@marcschier but dont we need an update to use the valueTask overload of connectasync and also propagate that?

Send/ receive Block CPU quite a bit, doesnt using async overloads help If the Socket eg is waiting for the network Stack or is this handled on a lower level and doesnt affect .Net threads?

Tbh I dont know how Socket works and which overloads are best to use.

[marcschier]

You are right, we currently ConnectAsync and use the Completion callback of the awaiter to trigger open/secure channel open essentially as Begin/End and then await the connect "operation". Same as what we do with SendRequestAsync. In the back of the socket implementation in .net it just uses the IO Completion to trigger a ValueTaskSource manual completion and what we should do is to do the same but wrap it one level higher at the Operation level, essentially what you did on the server. There is good code in Kestrel that we can in spirit align with.