Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow making conditional regions #392

Open
Kixiron opened this issue Jun 5, 2021 · 1 comment · May be fixed by #393
Open

Allow making conditional regions #392

Kixiron opened this issue Jun 5, 2021 · 1 comment · May be fixed by #393

Comments

@Kixiron
Copy link
Contributor

Kixiron commented Jun 5, 2021

Scope::region() and Scope::region_named() have intrinsic overhead since in order to report events about all sub-operators it has to observe the traffic that they send and receive. In order to combat this, timely should offer some sort of optional region that takes a boolean which determines whether or not the region acts like Scope::region()/Scope::region_named() or whether it does nothing and is a noop. Regions are incredibly helpful for debugging and visualizing dataflows, but in large codebases they can end up having non-trivial overhead.

Example Code

Since writing something like this is not only painful for developers (and not always possible because of the dreaded generics) but also hurts both code reuse and compile times

if regions_are_enabled {
    scope.region(|region| {
        // ...inner dataflow graph
    })
} else {
    // ...inner dataflow graph
}

Optional regions would allow writing this instead

scope.optional_region(regions_are_enabled, |region| {
    // ...inner dataflow graph
})

As per usual the normal Scope::optional_region() and Scope::optional_region_named() (or maybe Scope::optional_named_region()?) should be offered so that users can name the regions

@frankmcsherry
Copy link
Member

There is a related issue that might be good to think about at the same time, of "dynamic scope checking", where streams could check at runtime that they belong to the same scope before they are connected. This used to be largely enforced by types, as the scope type had a stack of lifetimes, but is a bit easier to get wrong with regions. It seems like it could be even more likely to get wrong, if we dynamically choose whether regions exist or not.

@Kixiron Kixiron linked a pull request Jun 9, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants