fix: perform streaming upload one chunk at a time

[kskalski]

Closes #404

---

This change is 

[Copilot]

Pull Request Overview

This PR refactors the streaming upload functionality to process chunks sequentially rather than concurrently, addressing issue #404. The change simplifies the multipart upload logic by removing the concurrent chunk processing and instead handling one chunk at a time.

Key changes:

• Replaces concurrent chunk processing with sequential processing using try_join to parallelize only the current chunk upload and next chunk reading
• Simplifies error handling by removing nested match statements for abort scenarios
• Eliminates the intermediate collection and transformation of ETags by building the parts vector directly

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The method call response.to_string()? appears incorrect. Based on the original code that used response_data.as_str()?, this should likely be response.as_str()?.to_string() to first convert to string slice then to owned String.

Suggested change

	etag: response.to_string()?,
	etag: response.as_str()?.to_string(),

[durch]

@kskalski appreciate your patience, will be release in 0.37.*, very nice work, thank you!

[durch]

Thank you for identifying this memory issue and providing a fix!

You're absolutely right that the current implementation reads ALL chunks into memory before starting uploads, which causes memory exhaustion for large files and connection timeouts.

Your sequential approach correctly fixes the memory issue. However, we're concerned about the performance impact of processing chunks one at a time, especially for large files or high-latency connections.

Better Solution We'll Implement: Bounded Parallelism

We'll implement a sliding window approach with bounded parallelism that:

• Maintains a small number of chunks in flight simultaneously (e.g., 3-5)
• As each chunk upload completes, reads and starts the next one
• Keeps memory usage bounded while preserving upload performance

This can be done using futures::stream::FuturesUnordered with a buffer limit or a semaphore-based approach.

Thank you again for identifying this issue and demonstrating the problem clearly. We'll build on your work to create a solution that addresses both the memory and performance concerns.