Skip to content

Conversation

@fsk119
Copy link
Member

@fsk119 fsk119 commented Oct 20, 2025

What is the purpose of the change

Add a rule to convert vector search call with constant input to a correlated vector search call.

Brief change log

  • Added rule ConstantVectorSearchCallToCorrelateRule

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot
Copy link
Collaborator

flinkbot commented Oct 20, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

.booleanType()
.defaultValue(false)
.withDescription(
"Whether to run an async search function or not. Default to false.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is ambiguous. Does it mean:
when false the function does not run or when false the function runs synchronously.

.intType()
.defaultValue(10)
.withDescription(
"The max number of async i/o operation that the async vector search can trigger.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: operation -> operations

+ "ALLOW_UNORDERED means the operator emits the result when "
+ "execution finishes. The planner will attempt to use "
+ "ALLOW_UNORDERED when it doesn't affect the correctness of "
+ "the results.\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be useful to explain more about the situations where the the correctness of the results would have been effected, so we can understand the scenarios in which this option is ignored.

GenericRowData argsData = new GenericRowData(args.length - 1);
for (int i = 1; i < args.length; ++i) {
argsData.setField(i, args[i]);
argsData.setField(i - 1, args[i]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this change mean that the previous code did not work? Is the symptom of this failure included in the Jira?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't added async vector search operator yet. 0.0

RowType outputType = (RowType) getOutputType();
StreamOperatorFactory<RowData> operatorFactory =
isAsyncEnabled
? createAsyncVectorSearchOperator()
Copy link
Contributor

@davidradl davidradl Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious what this is doing , there is one implementation that throws an Exception. I thought this change is adding options for the async search.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will introduce async operator in another PR.

@fsk119
Copy link
Member Author

fsk119 commented Oct 20, 2025

@davidradl Hi, please see the first PR #27122

@github-actions github-actions bot added the community-reviewed PR has been reviewed by the community. label Oct 20, 2025
@fsk119 fsk119 force-pushed the literal-rule branch 4 times, most recently from 4944f99 to 42e6e02 Compare October 22, 2025 11:10
@fsk119
Copy link
Member Author

fsk119 commented Oct 23, 2025

@flinkbot run azure

@fsk119 fsk119 merged commit c95979d into apache:master Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants