[WIP] Start prototyping support for opaque engine expressions #686
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Kernel does not provide any way for engines to work with expressions kernel does not fully understand. This creates unwelcome pressure to implement operations various engines need, directly in kernel, even tho kernel itself does not use them at all. A canonical example is
[NOT] IN
, which is highly complex to implement fully and correctly (see e.g. #652), and which was only added in the first place because engines wanted to use it for their own query processing.The solution is to define an opaque expression trait which engines can use to round-trip any expression they want through the kernel. Initially, kernel can support partition pruning over such expressions if the engine provides an appropriate implementation. Eventually, it might be possible to also express basic data skipping support as well -- but any such effort would be far out of scope for this initial exploration.
This PR affects the following public APIs
Adding more variants to the
Expression
enum, along with corresponding API support for manipulating it.How was this change tested?
TODO!