-
Notifications
You must be signed in to change notification settings - Fork 74
MLE-22706 Added encode/decode support for vectors #1789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for Base64 encoding and decoding of float vectors in the Java client to match the MarkLogic server’s vec:base64-encode
/vec:base64-decode
behavior and verifies interoperability via new tests.
- Introduces
VectorUtil.encode(float...)
andVectorUtil.decode(String)
with versioned, little-endian serialization. - Adds JUnit tests in
VectorUtilTest
to validate round-trip encoding/decoding using both Java client and server evaluations.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
File | Description |
---|---|
marklogic-client-api/src/main/java/com/marklogic/client/util/VectorUtil.java | Adds static methods for Base64 vector serialization with version handling. |
marklogic-client-api/src/test/java/com/marklogic/client/util/VectorUtilTest.java | Implements tests covering client-only, server-only, and cross-boundary encode/decode scenarios. |
Comments suppressed due to low confidence (2)
marklogic-client-api/src/test/java/com/marklogic/client/util/VectorUtilTest.java:15
- [nitpick] Test constants are generally defined as
static final
. Consider changing this toprivate static final float[] VECTOR
to follow standard constant conventions and avoid redundant instance fields.
private final float[] VECTOR = new float[]{3.14f, 1.59f, 2.65f};
marklogic-client-api/src/main/java/com/marklogic/client/util/VectorUtil.java:36
- Add a test case to cover this
IllegalArgumentException
path by decoding a Base64 string with a non-zero version to ensure the exception is thrown as expected.
throw new IllegalArgumentException("Unsupported vector version: " + version);
static String encode(float... vector) { | ||
final int dimensions = vector.length; | ||
ByteBuffer buffer = ByteBuffer.allocate(8 + 4 * dimensions); | ||
buffer.order(ByteOrder.LITTLE_ENDIAN); | ||
buffer.putInt(0); // version |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extract the hard-coded version 0
into a named constant (e.g. VERSION = 0
) to improve clarity and make future version changes easier.
static String encode(float... vector) { | |
final int dimensions = vector.length; | |
ByteBuffer buffer = ByteBuffer.allocate(8 + 4 * dimensions); | |
buffer.order(ByteOrder.LITTLE_ENDIAN); | |
buffer.putInt(0); // version | |
// Define the version constant | |
public static final int VERSION = 0; | |
static String encode(float... vector) { | |
final int dimensions = vector.length; | |
ByteBuffer buffer = ByteBuffer.allocate(8 + 4 * dimensions); | |
buffer.order(ByteOrder.LITTLE_ENDIAN); | |
buffer.putInt(VERSION); // version |
Copilot uses AI. Check for mistakes.
buffer.order(ByteOrder.LITTLE_ENDIAN); | ||
|
||
final int version = buffer.getInt(); | ||
if (version != 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use the same named constant instead of the literal 0
when checking the version to maintain consistency with the encoding logic.
if (version != 0) { | |
if (version != VECTOR_VERSION) { |
Copilot uses AI. Check for mistakes.
String encoded = VectorUtil.encode(VECTOR); | ||
assertEquals("AAAAAAMAAADD9UhAH4XLP5qZKUA=", encoded); | ||
|
||
ArrayNode decoded = (ArrayNode) Common.newEvalClient().newServerEval() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of casting from JsonNode
, consider using .evalAs(ArrayNode.class)
directly to improve type safety and remove the unchecked cast.
Copilot uses AI. Check for mistakes.
36d11d0
to
aede4bc
Compare
No description provided.