-
Notifications
You must be signed in to change notification settings - Fork 685
Update Rust client SDK for V2 WebSocket format #4257
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gefjon
wants to merge
20
commits into
jsdt/ws-v2
Choose a base branch
from
phoebe/rust-sdk-ws-v2
base: jsdt/ws-v2
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This compiles, but absolutely won't run, and is filled with a ton of TODOs.
Not actually update Rust codegen, manually rewrite quickstart-chat-rust template to make it all compile.
…-ws-v2 Complete with lots of merge conflicts, and also updating the Nix flake.
Pass the generated request_id into the CallReducer message instead of hardcoded 0, matching the procedure code pattern. Without this, reducer callbacks never fire because the server echoes back request_id 0 which doesn't match any stored callback.
- Replace on_<reducer>() callbacks with _then() invocation pattern - Replace Status::Failed/OutOfEnergy with Status::Err/Panic - Remove CallReducerFlags, set_reducer_flags, Event::UnknownTransaction - Remove ReducerEvent.caller_identity/caller_connection_id field access
It no longer does anything
Zero-sized rows are well-defined, and we have tests that rely on them working
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Update the Rust client SDK to use the new V2 WebSocket format, and present the V2 user-facing API.
Reducer events
Remove on-reducer callbacks
It's no longer possible to observe reducers called by other clients by registering callbacks with
ctx.reducers.on_{my_reducer}. We no longer code-generate those methods, or the associatedctx.reducers.remove_on_{my_reducer}. Internal plumbing for storing and invoking those callbacks is also removed.Add specific reducer invocation callbacks
In addition to the previous way to invoke reducers,
ctx.reducers.{my_reducer}(args...), we add a method that registers a callback to run after the reducer is finished. This method has the suffix_then, as inctx.reducers.{my_reducer}_then(args..., callback).The callback will accept two arguments:
ctx: &ReducerEventContext, the same context as was previously passed to on-reducer callbacks.status: Result<Result<(), String>, InternalError>, denoting the outcome of the reducer.Ok(Ok(())means the reducer committed. This corresponds toReducerOutcome::OkorReducerOutcome::Okmptyin the new WS format.Ok(Err(message))means the reducer returned an "expected" or "user" error. This corresponds toReducerOutcome ;: e rrin the new WS format.Err(internal_error)means something went wrong with host execution. This corresponds toReducerOutcome::InternalErrorin the new WS format.Internally, the SDK stores the callbacks in its
ReducerCallbacksmap. This is keyed onrequest_id: u32, a number that is generated for each reducer call (from anAtomicU32that we increment each time), and included in theClientMessage::CallReducerrequest. TheServerMessage::ReducerResultincludes the samerequest_id, so the SDK pops out of theReducerCallbacksand invokes the appropriate callback when processing that message.These new callbacks are very similar to the existing procedure callbacks.
The
Eventexposed to row callbacksRow callbacks caused by a reducer invoked by this client will see
Event::Reducer, the same as they would prior to this PR. These callbacks will be the result of aServerMessage::ReducerResultwithReducerOutcome::Ok. In order to expose the reducer name and arguments to this event, the client stores them in itsReducerCallbacksmap, alongside the callback for when the reducer is complete.Row callbacks caused by any other reducer, or any non-reducer transaction, are now indistinguishable to the client. These will see
Event::Transaction, which is renamed from the oldEvent::UnknownTransaction.Less metadata in
ReducerEventSome metadata is removed from
ReducerEvent, as the V2 WebSocket format no longer publishes it, even to the caller.CallReducerFlagsare removedAll machinery for setting, storing and applying call reducer flags is removed from the SDK, as the new WS format does not have any non-default flags.
Requesting rows in unsubscribe
When sending a
ClientMessage::Unsubscribe, we always request that the server include the matching rows in its responseServerMessage::UnsubscribeApplied. This saves us having to update the SDK to store query sets separately, at least for now. (We'll do that later.)Handling rows
The new SDK does some additional parsing to wrangle rows in the new WebSocket format into the same internal data structures as before, rather than re-writing the client cache. (We'll do that later.) Specifically, parsing of
DbUpdateis changed so that:TransactionUpdateinto the generatedDbUpdatetype, which requires an additional loop compared to the previous version, to cope with the new WS format's dividing updates by query set. We define a functiontransaction_update_iter_table_updateswhich encapsulates this nested loop in an iterator.QueryRowsinto the generatedDbUpdatetype, one for when they come from aSubscribeApplied, and the other when they come from anUnsubscribeApplied.QueryRowsfromSubscribeAppliedtranslate to aDbUpdateof all inserts, while one fromUnsubscribeAppliedwill be all deletes.Legacy subscriptions
"Legacy subscriptions" are removed. These were only used for
subscribe_to_all_tables, which as of now is stubbed. I will follow up with a change to re-implementsubscribe_to_all_tablesby code-generating a list of all known tables, and having it subscribe toselect * from {table}for every table in that list.Light mode removed
Light mode is no longer meaningful in the V2 WS format, so all code related to it is removed.
Internal changes
Renamed WS messages
The SDK's internal code is updated to account for various renames:
QueryId->QuerySetId,query_id->query_set_id.SubscribeMulti->Subscribe,UnsubscribeMulti->Unsubscribe.Incidental changes in this PR, not necessary for other client SDKs
Don't filter out empty ranges in
RowSizeHintThe Rust implementation of
RowSizeHintinBsatnRowListgot regressed in the base branch to not work with zero-sized rows. This change fixes that.API and ABI breaking changes
Boy howdy is it!
Expected complexity level and risk
3? Changes ended up being less complicated than I feared, but we do have some fiddly code here, and we have internal dependencies on the SDK.
Testing
fail_reducer, which depends on functionality that @jsdt hasn't yet implemented in the parent branch.subscribe_all_select_star, which depends on the aforementionedsubscribe_to_all_tables.