request receipt contains wrong size for response_transfer_size at final delivery

[evgmik]

Hi, first of all many thanks for amazing idea and implementation of Reticulum.

I was playing with idea of link.request as a base of inter node communication and I noticed that on delivery, i.e. when the progress reaches "1.0" the response_transfer_size contains wrong value which is roughly twice larger than response_size. While the report about size is wrong, the data itself seems to be correct.

Steps to reproduce:
This is happening when response does not fit to a single packet and request uses resource to communicate.
I was sending incompressible random binary blob (to avoid artifacts of compression) and monitoring request progress_callback and printing {receipt.response_transfer_size}/{receipt.response_size} notice the very last debug string where transferred amount double. However it is not always doubled.

[2025-02-23 22:02:29] [Debug] Request get a.10MB progress 0.9999115005088721: 10485856/10485800
[2025-02-23 22:02:29] [Debug] Request get a.10MB progress 0.9999557502544361: 10485856/10485800
[2025-02-23 22:02:29] [Debug] Request get a.10MB progress 1.0: 10485856/10485800
[2025-02-23 22:02:29] [Debug] Request get a.10MB progress 1.0: 20971712/10485800
[2025-02-23 22:02:29] [Info]  Request get a.10MB received

Similar result can be observed with Examples/Request.py where one of the texts string values is replaced with a longer random string: bytes(random.getrandbits(8) for _ in range(1000))

2nd minor bug. Request itself is not documented in the official API page :). I stumbled on it by an incident: 1st I noticed RequestReceipt in the API reference, then I start digging through examples. In my opinion Request is much easier to handle concept since it does automatic fallback to resource transfer for a bigger data and returns outcomes of requests automatically.

[markqvist]

Thanks for the issue report.

The request functionality is documented under the Link API in the API reference.

I'll have a look at why that transfer progress variable is incorrectly set in some cases. Rest assured this won't affect the data integrity, it is simply an internal variable to report transfer progress to the calling application. Data is always hashed and signed, and won't be accepted unless integrity is ensured.

[markqvist]

Also, the best way to get transfer progress, etc., is to supply the request() call with callback functions on creation, instead of manually checking the RequestReceipt object.

[evgmik]

The request functionality is documented under the Link API in the API reference.

I misspoke, what I ment that documentation for request is somewhat rudimental. For example data field is not described, and especially that data is not limited to a packet size for the request and response. Also, it would be nice to add a intended use in the caption like it is done for Resource and Channel. How about adding something like:

---

Sends a request to the remote peer.

Request can send an arbitrary size data to the destination over the link, the RequestReceipt response from the destination can also carry any data size. If the data does not fit in a packet the transmission falls back to the Resource data transfer method.

The request receiver has to set a callback for the intended path otherwise the request will be neglected.

---

Side note. It is a bit strange that constructor is called link.request and not to the RNS.request like it is done with RNS.Resource and RNS.Channel. Conceptually, it is also a data transfer mechanism with different capabilities.

[markqvist]

Side note. It is a bit strange that constructor is called link.request

No.

How are you going to create a request without an established link? It's a much simpler API to run the request call directly on the established link.

[evgmik]

Please do not treat my question as the request to redo the software, but more like please give me more info about how the pieces are held together.

I want to explain the source of my confusion. I believe that I read the API guide and all available docs. But once I started to implement my own code with data transfer, I run to the need to reverse engineer the examples' code. Reverse engineer in the sense that the examples themselves are very well written, but I missing the design motivation for the choices made. So I am trying to build a "model" in my head and pieces do not fully fit. This is why I asked my questions.

I think this is where I am missing a bigger picture. I noticed that for example RNS.Resource takes link as a parameter. I.e. it expect link to be established before a call to it is made. But request seems to be defined differently from either Resource or Channel. So there is an inconsistency between data transferring classes.

Should I think that request is a higher level (compound) primitive which does not belong to the base classes such as Resource and Channel?

[markqvist]

Thanks for the clarification, and I totally understand that. There's definitely room for improvement there! And I appreciate your comments because it makes it more clear where the conceptual communication can be failing.

What I really want to do here is create a much more conceptual overview and introduction to how the different primitives in Reticulum can interact and be used in systems design. It does not completely follow the conventional hierarchical approach that most contemporary software systems exist within, and as such needs a bit more of a high-level explanation to become truly "grokked" by those using it.

In the early 2000's, Palm Computing published a document called The Zen of Palm, which was a conceptual guide towards a more optimal framework for creating PalmOS applications. A resource inspired by this approach is what I've really been wanting to add to the Reticulum Manual for a long time, but time and priorities always seem to be the prima veritas, so to speak.