Skip to content

Conversation

@carte7000
Copy link
Contributor

@carte7000 carte7000 commented Oct 30, 2025

feat: add idempotency key support to prevent duplicate CCV verification processing

Updates aggregator database schema, API validation, and storage layer to use idempotency_key for deduplication instead of verification_timestamp. Modifies verifier pipeline to generate and propagate idempotency keys through the CCVDataWithIdempotencyKey paired struct from source readers to storage operations.

@carte7000 carte7000 changed the title feat: add idempotency key support to prevent duplicate CCV verificati… feat: add idempotency key support Oct 30, 2025
@carte7000 carte7000 force-pushed the simon/aggregator/idempotency-key branch from 4b3d0b1 to fa9f4fe Compare October 30, 2025 18:30
…on processing

Updates aggregator database schema, API validation, and storage layer to use idempotency_key for deduplication instead of verification_timestamp. Modifies verifier pipeline to generate and propagate idempotency keys through the CCVDataWithIdempotencyKey paired struct from source readers to storage operations.
@carte7000 carte7000 force-pushed the simon/aggregator/idempotency-key branch from fa9f4fe to 2719f1e Compare October 30, 2025 19:28
signature_s BYTEA NOT NULL DEFAULT '',
ccv_node_data BYTEA NOT NULL,
verification_timestamp BIGINT NOT NULL,
idempotency_key TEXT NOT NULL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for not using UUID? It's more efficient, less storage and perfectly fit for an idempotency key

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main reason was not to enforce any format on the client. When storing as text a verifier could use any value they see fit as their idempotency key.

It's a question of whether we want to be more flexible or optimize storage efficiency.

I like that we let the caller pick the idempotency value without any strong validation the aggregator side, but open to change it if we think it's better to enforce the UUID as idempotency key

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think it's not wise to let the client choose the idempotency key freely. It's error prone to do so IMO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and uuid are global enough, easy to use and battle tested already. On top of it the efficiency and the more performant storage. I think it's a clear choice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do the change just that I think it's not necessarily a clear cut decision, probably it's pretty low stakes in our context anyways:

https://sdk.amazonaws.com/swift/api/awsec2/1.5.74/documentation/awsec2/runinstancesinput/clienttoken

We don't have to do anything with the key so if the client wants to use timestamp as value they could. But I guess they could derive a UUID from any value they want.

@github-actions
Copy link

github-actions bot commented Nov 3, 2025

E2E Smoke Test Results

Test Case Status Duration
TestE2ESmoke/extra_args_v2/src->dst_msg_execution_eoa_receiver pass 11.22s
TestE2ESmoke/extra_args_v2/dst->src_msg_execution_eoa_receiver pass 10.22s
TestE2ESmoke/extra_args_v2/1337->3337_msg_execution_mock_receiver pass 10.72s
TestE2ESmoke/extra_args_v2 pass 32.16s
TestE2ESmoke/extra_args_v3/EOA_receiver_and_default_committee_verifier pass 9.72s
TestE2ESmoke/extra_args_v3/EOA_receiver_and_secondary_committee_verifier pass 10.22s
TestE2ESmoke/extra_args_v3/receiver_w/_secondary_verifier_required pass 9.72s
TestE2ESmoke/extra_args_v3/receiver_w/_secondary_required_and_tertiary_optional_threshold=1 pass 10.22s
TestE2ESmoke/extra_args_v3/receiver_w/_default_required,_secondary_and_tertiary_optional,_threshold=1,_message_specifies_all_three pass 9.72s
TestE2ESmoke/extra_args_v3/receiver_w/_default_required,_secondary_and_tertiary_optional,_threshold=1,_message_specifies_default_and_secondary pass 9.72s
TestE2ESmoke/extra_args_v3/receiver_w/_default_required,_secondary_and_tertiary_optional,_threshold=1,_message_specifies_default_and_tertiary pass 10.22s
TestE2ESmoke/extra_args_v3/EOA_receiver_and_default_committee_verifier#01 pass 9.72s
TestE2ESmoke/extra_args_v3/max_data_size pass 9.72s
TestE2ESmoke/extra_args_v3/EOA_receiver_and_default_committee_verifier_with_token_transfer pass 10.23s
TestE2ESmoke/extra_args_v3 pass 99.22s
TestE2ESmoke pass 131.59s

Full logs are available in the workflow artifacts.

@github-actions
Copy link

github-actions bot commented Nov 3, 2025

Code coverage report:

Package main simon/aggregator/idempotency-key
aggregator 47.89% 48.01%
cciptestinterfaces 100.00% 100.00%
ccv-evm 0.00% 0.00%
cmd 0.00% 0.00%
executor 36.32% 36.32%
indexer 25.37% 33.62%
integration 4.28% 4.27%
protocol 42.02% 42.02%
verifier 42.93% 43.03%

@carte7000 carte7000 marked this pull request as ready for review November 3, 2025 13:39
@carte7000 carte7000 requested review from a team and skudasov as code owners November 3, 2025 13:39
@carte7000 carte7000 merged commit faefe8b into main Nov 3, 2025
11 checks passed
@carte7000 carte7000 deleted the simon/aggregator/idempotency-key branch November 3, 2025 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants