[Feature]: Socket.recv_into: Is there any way to recv data directly to an existing buffer?

[noooop]

_recv_frame always creates a new Frame.

@cfunc
@inline
def _recv_frame(handle: p_void, flags: C.int = 0, track: bint = False) -> Frame:
    """Receive a message in a non-copying manner and return a Frame."""
    rc: C.int
    msg = zmq.Frame(track=track)
    cmsg: Frame = msg

    while True:
        with nogil:
            rc = zmq_msg_recv(address(cmsg.zmq_msg), handle, flags)
        try:
            _check_rc(rc)
        except InterruptedSystemCall:
            continue
        else:
            break
    return msg

This will result in one more copy of the data, which is not cool.

Is there any way to recv data directly to an existing buffer?

This way I can pass in the pre-allocated buffer instead of copying the data in the Frame to where it should go.

When using the C language API, you always need to prepare a buffer.

char buf [256];
nbytes = zmq_recv (socket, buf, 256, 0);
assert (nbytes != -1);

C language can always achieve all the things.

[minrk]

It is not yet implemented, but we could use the newer C API to implement a Socket.recv_into

[noooop]

It is not yet implemented, but we could use the newer C API to implement a Socket.recv_into

cool !

looking forward to this feature.

[noooop]

hi @minrk

There’s another new feature you might want to hear about:

recv_multipart always accept all data, which is not cool.

def recv_multipart(
      parts = [self.recv(flags, copy=copy, track=track)]
      # have first part already, only loop while more to receive
      while self.getsockopt(zmq.RCVMORE):
          part = self.recv(flags, copy=copy, track=track)
          parts.append(part)
      # cast List[Union] to Union[List]
      # how do we get mypy to recognize that return type is invariant on `copy`?
      return cast(Union[List[zmq.Frame], List[bytes]], parts)

If I can use the previous Frame or even the first Frame to determine whether the subsequent Frame is needed,

1. if subsequent Frame is needed，use Socket.recv_into to accept the date with the minimum overhead to where it should go.

2. if subsequent Frame is not needed，how about use a new interface to throw away this data with minimal overhead.

I know that when notifying the socket to read, the multipart is actually ready, but one less copy is always better

zguide chapter2

Some things to know about multipart messages:

• When you send a multipart message, the first part (and all following parts) are only actually sent on the wire when you send the final part.
• If you are using zmq_poll(), when you receive the first part of a message, all the rest has also arrived.
• You will receive all parts of a message, or none at all.
• Each part of a message is a separate zmq_msg item.
• You will receive all parts of a message whether or not you check the more property.
• On sending, ZeroMQ queues message frames in memory until the last is received, then sends them all.
• There is no way to cancel a partially sent message, except by closing the socket.