Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Add Flow#switchMap operator. #974

Closed
He-Pin opened this issue Jan 17, 2024 · 8 comments · Fixed by #1787
Closed

Feature request: Add Flow#switchMap operator. #974

He-Pin opened this issue Jan 17, 2024 · 8 comments · Fixed by #1787
Assignees
Labels
t:stream Pekko Streams
Milestone

Comments

@He-Pin
Copy link
Member

He-Pin commented Jan 17, 2024

Motivation:
I find that Pekko stream doen't has the switchmap operator, it's called as this in both fs2 and flux, I think we need to take the same name and same semantic.

Usercase:
A Stream of Post, eg twitter/X 's Post, and every Post has a Stream of Comments.

[Post 1] ---> [Comment 1] ... [Comment 2] ... [Comment 3] ...
.
.
.
[Post 2] ---> [Comment 1] ... [Comment 2] ... 
.
.
.
[Post 3] ---> [Comment 1] ... [Comment 2] ... [Comment 3] ... [Comment 4]..
.
.
.
\|/ User scrolling down.

Emits when: sub stream emits and downstream is available
Backpressures when: sub stream backpressure.
cancels when: down stream cancels
completes when: upstream completes and the current substream completes.

In some implementation, the upstream is never backpressured, I think that's not safe. I think the upstream should be backpressured if there is no downstream demands.

Result:
We have rich operators set.

@He-Pin He-Pin added the t:stream Pekko Streams label Jan 17, 2024
@He-Pin He-Pin added this to the 1.1.0 milestone Jan 17, 2024
@He-Pin He-Pin self-assigned this Jan 17, 2024
@He-Pin He-Pin changed the title Add Flow#switchMap operator. Feature request: Add Flow#switchMap operator. Jan 18, 2024
@He-Pin
Copy link
Member Author

He-Pin commented Jan 19, 2024

@pjfanning @mdedetrich what do you think about his one, I find it would be nice for live streaming.

@He-Pin He-Pin modified the milestones: 1.1.0, 2.0.x Jan 24, 2024
@He-Pin
Copy link
Member Author

He-Pin commented Jan 24, 2024

Implementation this shold not be that hard but seems no much interest, will defer to later version. it's a kind of stream of streams

@mdedetrich
Copy link
Contributor

I think we have enough on our plate right now with all of the new features, we can always add this into 1.1.0-M2 (if a new M2 is needed) or in 1.2.x.

@He-Pin
Copy link
Member Author

He-Pin commented Jan 24, 2024

I will back my hometown soon, if anyone submited a pr about this, then this can be added to 1.1.0-M2 I think, before that, it's safe for later version.

I was expecting we stating remove code in 1.2.x but not adding features.

I just update what I thought about this operator.

@He-Pin
Copy link
Member Author

He-Pin commented Jan 3, 2025

refs: #1660

@He-Pin
Copy link
Member Author

He-Pin commented Jan 7, 2025

@MartinHH
Copy link
Contributor

MartinHH commented Mar 2, 2025

I used to use this operator a lot when dealing with various applications related to monitoring live data and always missed it when working with akka streams in the past. I'm currently not working on any of such projects - if I were, I'd happily submit a PR (and might as well still do that if I find the time).

One thing though: from my experience (I might be biased due to the "live data" use cases I used it for), I'd strongly advocate for "Backpressures when: never." The way this operator works, only the very latest value from upstream should cause items to be emitted to downstream, so there is no sense in keeping anything but that one latest value (and the resulting substream) in the pipeline.

@He-Pin
Copy link
Member Author

He-Pin commented Mar 2, 2025

That's true; this kind of operator will never do any backpressure.
I was planning to add one, but I have no scenario in my $Work, if you submit one, I would be pleased to review it.

@He-Pin He-Pin linked a pull request Mar 7, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t:stream Pekko Streams
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants