Skip to content

feat: add idempotency key support #244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Nov 3, 2025
Merged

feat: add idempotency key support #244

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2719f1e
feat: add idempotency key support to prevent duplicate CCV verificati…
carte7000 Oct 30, 2025
466d421
Merge remote-tracking branch 'origin/main' into simon/aggregator/idem…
carte7000 Oct 31, 2025
ada9daa
Merge branch 'main' into simon/aggregator/idempotency-key
carte7000 Oct 31, 2025
59ee59a
Merge branch 'simon/aggregator/idempotency-key' of github.com:smartco…
carte7000 Oct 31, 2025
1538340
Fix merge issues
carte7000 Oct 31, 2025
809eda1
Address PR comments
carte7000 Nov 3, 2025
81ec00d
Merge remote-tracking branch 'origin/main' into simon/aggregator/idem…
carte7000 Nov 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion aggregator/migrations/postgres/00001_create_all_tables.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ CREATE TABLE IF NOT EXISTS commit_verification_records (
signature_s BYTEA NOT NULL DEFAULT '',
ccv_node_data BYTEA NOT NULL,
verification_timestamp BIGINT NOT NULL,
idempotency_key TEXT NOT NULL,
Copy link
Contributor

@asoliman92 asoliman92 Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for not using UUID? It's more efficient, less storage and perfectly fit for an idempotency key

Copy link
Contributor Author

@carte7000 carte7000 Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main reason was not to enforce any format on the client. When storing as text a verifier could use any value they see fit as their idempotency key.

It's a question of whether we want to be more flexible or optimize storage efficiency.

I like that we let the caller pick the idempotency value without any strong validation the aggregator side, but open to change it if we think it's better to enforce the UUID as idempotency key

Copy link
Contributor

@asoliman92 asoliman92 Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think it's not wise to let the client choose the idempotency key freely. It's error prone to do so IMO.

Copy link
Contributor

@asoliman92 asoliman92 Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and uuid are global enough, easy to use and battle tested already. On top of it the efficiency and the more performant storage. I think it's a clear choice.

Copy link
Contributor Author

@carte7000 carte7000 Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do the change just that I think it's not necessarily a clear cut decision, probably it's pretty low stakes in our context anyways:

https://sdk.amazonaws.com/swift/api/awsec2/1.5.74/documentation/awsec2/runinstancesinput/clienttoken

We don't have to do anything with the key so if the client wants to use timestamp as value they could. But I guess they could derive a UUID from any value they want.

created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT unique_verification UNIQUE (message_id, committee_id, signer_address, verification_timestamp)
CONSTRAINT unique_verification UNIQUE (message_id, committee_id, signer_address, idempotency_key)
);

CREATE TABLE IF NOT EXISTS commit_aggregated_reports (
Expand Down
3 changes: 2 additions & 1 deletion aggregator/pkg/handlers/validations.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ import (
func validateWriteRequest(req *pb.WriteCommitCCVNodeDataRequest) error {
err := validation.ValidateStruct(
req,
validation.Field(&req.CcvNodeData, validation.Required))
validation.Field(&req.CcvNodeData, validation.Required),
validation.Field(&req.IdempotencyKey, validation.Required, validation.Length(1, 255)))
if err != nil {
return err
}
Expand Down
1 change: 1 addition & 0 deletions aggregator/pkg/handlers/write_commit_ccv_data.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ func (h *WriteCommitCCVNodeDataHandler) Handle(ctx context.Context, req *pb.Writ
MessageWithCCVNodeData: *req.GetCcvNodeData(),
IdentifierSigner: signer,
CommitteeID: signer.CommitteeID,
IdempotencyKey: req.GetIdempotencyKey(),
}
err := h.storage.SaveCommitVerification(signerCtx, &record)
if err != nil {
Expand Down
3 changes: 2 additions & 1 deletion aggregator/pkg/model/commit_verification_record.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ func (c CommitVerificationRecordIdentifier) ToIdentifier() string {
type CommitVerificationRecord struct {
IdentifierSigner *IdentifierSigner
pb.MessageWithCCVNodeData
CommitteeID CommitteeID
CommitteeID CommitteeID
IdempotencyKey string
}

// GetID retrieves the unique identifier for the commit verification record.
Expand Down
7 changes: 4 additions & 3 deletions aggregator/pkg/storage/postgres/database_storage.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,9 +142,9 @@ func (d *DatabaseStorage) SaveCommitVerification(ctx context.Context, record *mo

stmt := `INSERT INTO commit_verification_records
(message_id, committee_id, participant_id, signer_address, source_chain_selector, dest_chain_selector,
onramp_address, offramp_address, signature_r, signature_s, ccv_node_data, verification_timestamp)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12)
ON CONFLICT (message_id, committee_id, signer_address, verification_timestamp)
onramp_address, offramp_address, signature_r, signature_s, ccv_node_data, verification_timestamp, idempotency_key)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
ON CONFLICT (message_id, committee_id, signer_address, idempotency_key)
DO NOTHING`

ccvNodeData, err := proto.Marshal(&record.MessageWithCCVNodeData)
Expand Down Expand Up @@ -174,6 +174,7 @@ func (d *DatabaseStorage) SaveCommitVerification(ctx context.Context, record *mo
record.IdentifierSigner.SignatureS[:],
ccvNodeData,
record.Timestamp,
record.IdempotencyKey,
)
if err != nil {
return fmt.Errorf("failed to save commit verification record: %w", err)
Expand Down
4 changes: 3 additions & 1 deletion aggregator/pkg/storage/postgres/database_storage_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (

"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/crypto"
"github.com/google/uuid"
"github.com/jmoiron/sqlx"
"github.com/stretchr/testify/require"
"github.com/testcontainers/testcontainers-go"
Expand Down Expand Up @@ -181,7 +182,8 @@ func createTestCommitVerificationRecord(msgWithCCV *pb.MessageWithCCVNodeData, s
SignatureS: s32,
CommitteeID: committeeID,
},
CommitteeID: committeeID,
CommitteeID: committeeID,
IdempotencyKey: uuid.NewString(), // Generate unique idempotency key for each record
}
}

Expand Down
150 changes: 44 additions & 106 deletions aggregator/tests/commit_verification_api_test.go
View file
Open in desktop

Large diffs are not rendered by default.

17 changes: 17 additions & 0 deletions aggregator/tests/fixture.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"time"

"github.com/ethereum/go-ethereum/crypto"
"github.com/google/uuid"
"github.com/stretchr/testify/require"

"github.com/smartcontractkit/chainlink-ccv/aggregator/pkg/model"
Expand Down Expand Up @@ -179,3 +180,19 @@ func NewMessageWithCCVNodeData(t *testing.T, message *protocol.Message, sourceVe
}
return ccvNodeData
}

// NewWriteCommitCCVNodeDataRequest creates a new WriteCommitCCVNodeDataRequest with a generated idempotency key.
func NewWriteCommitCCVNodeDataRequest(ccvNodeData *pb.MessageWithCCVNodeData) *pb.WriteCommitCCVNodeDataRequest {
return &pb.WriteCommitCCVNodeDataRequest{
CcvNodeData: ccvNodeData,
IdempotencyKey: uuid.New().String(),
}
}

// NewWriteCommitCCVNodeDataRequestWithKey creates a new WriteCommitCCVNodeDataRequest with a specific idempotency key.
func NewWriteCommitCCVNodeDataRequestWithKey(ccvNodeData *pb.MessageWithCCVNodeData, idempotencyKey string) *pb.WriteCommitCCVNodeDataRequest {
return &pb.WriteCommitCCVNodeDataRequest{
CcvNodeData: ccvNodeData,
IdempotencyKey: idempotencyKey,
}
}
2 changes: 1 addition & 1 deletion build/devenv/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ require (
require (
github.com/gorilla/websocket v1.5.3
github.com/smartcontractkit/chainlink-ccv v0.0.0-00010101000000-000000000000
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251028171604-3c9144b4db13
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251030133409-21ff07fff5e9
github.com/smartcontractkit/chainlink-testing-framework/wasp v1.51.1
google.golang.org/grpc v1.76.0
)
Expand Down
4 changes: 2 additions & 2 deletions build/devenv/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1024,8 +1024,8 @@ github.com/smartcontractkit/chainlink-deployments-framework v0.56.0 h1:VkzslEC/a
github.com/smartcontractkit/chainlink-deployments-framework v0.56.0/go.mod h1:ObH5HJ4yXzTmQLc6Af+ufrTVcQ+ocasHJ0YBZjw5ZCM=
github.com/smartcontractkit/chainlink-evm/gethwrappers v0.0.0-20250826201006-c81344a26fc3 h1:mJP6yJq2woOZchX0KvhLiKxDPaS0Vy4vTDFH4nnFkXs=
github.com/smartcontractkit/chainlink-evm/gethwrappers v0.0.0-20250826201006-c81344a26fc3/go.mod h1:3Lsp38qxen9PABVF+O5eocveQev+hyo9HLAgRodBD4Q=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251028171604-3c9144b4db13 h1:kbOA8/t8Eq+T70d4jCbuPjDCQAulnaOBnymtm6eR1ts=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251028171604-3c9144b4db13/go.mod h1:KJkb85Mfxr/2vjPvAWWpq0/QJMAP1Bts1wMWWhRn4/E=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251030133409-21ff07fff5e9 h1:vyj6oNj3cifQzg7jI/UCC59h8q3yPUzxnq3/lwrBMtc=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251030133409-21ff07fff5e9/go.mod h1:KJkb85Mfxr/2vjPvAWWpq0/QJMAP1Bts1wMWWhRn4/E=
github.com/smartcontractkit/chainlink-protos/cre/go v0.0.0-20250911124514-5874cc6d62b2 h1:1/KdO5AbUr3CmpLjMPuJXPo2wHMbfB8mldKLsg7D4M8=
github.com/smartcontractkit/chainlink-protos/cre/go v0.0.0-20250911124514-5874cc6d62b2/go.mod h1:jUC52kZzEnWF9tddHh85zolKybmLpbQ1oNA4FjOHt1Q=
github.com/smartcontractkit/chainlink-protos/job-distributor v0.13.1 h1:PWwLGimBt37eDzpbfZ9V/ZkW4oCjcwKjKiAwKlSfPc0=
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ require (
github.com/smartcontractkit/chainlink-deployments-framework v0.56.0
github.com/smartcontractkit/chainlink-evm v0.3.3
github.com/smartcontractkit/chainlink-evm/gethwrappers v0.0.0-20250826201006-c81344a26fc3
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251028171604-3c9144b4db13
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251030133409-21ff07fff5e9
github.com/smartcontractkit/chainlink-testing-framework/framework v0.10.33
github.com/stretchr/testify v1.11.1
github.com/testcontainers/testcontainers-go v0.39.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -752,8 +752,8 @@ github.com/smartcontractkit/chainlink-framework/metrics v0.0.0-20250717121125-23
github.com/smartcontractkit/chainlink-framework/metrics v0.0.0-20250717121125-2350c82883e2/go.mod h1:jo+cUqNcHwN8IF7SInQNXDZ8qzBsyMpnLdYbDswviFc=
github.com/smartcontractkit/chainlink-framework/multinode v0.0.0-20250729142306-508e798f6a5d h1:pTYIcsWHTMG5fAcbRUA8Qk5yscXKdSpopQ0DUEOjPik=
github.com/smartcontractkit/chainlink-framework/multinode v0.0.0-20250729142306-508e798f6a5d/go.mod h1:2JTBNp3FlRdO/nHc4dsc9bfxxMClMO1Qt8sLJgtreBY=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251028171604-3c9144b4db13 h1:kbOA8/t8Eq+T70d4jCbuPjDCQAulnaOBnymtm6eR1ts=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251028171604-3c9144b4db13/go.mod h1:KJkb85Mfxr/2vjPvAWWpq0/QJMAP1Bts1wMWWhRn4/E=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251030133409-21ff07fff5e9 h1:vyj6oNj3cifQzg7jI/UCC59h8q3yPUzxnq3/lwrBMtc=
github.com/smartcontractkit/chainlink-protos/chainlink-ccv/go v0.0.0-20251030133409-21ff07fff5e9/go.mod h1:KJkb85Mfxr/2vjPvAWWpq0/QJMAP1Bts1wMWWhRn4/E=
github.com/smartcontractkit/chainlink-protos/cre/go v0.0.0-20250911124514-5874cc6d62b2 h1:1/KdO5AbUr3CmpLjMPuJXPo2wHMbfB8mldKLsg7D4M8=
github.com/smartcontractkit/chainlink-protos/cre/go v0.0.0-20250911124514-5874cc6d62b2/go.mod h1:jUC52kZzEnWF9tddHh85zolKybmLpbQ1oNA4FjOHt1Q=
github.com/smartcontractkit/chainlink-protos/job-distributor v0.13.1 h1:PWwLGimBt37eDzpbfZ9V/ZkW4oCjcwKjKiAwKlSfPc0=
Expand Down
8 changes: 5 additions & 3 deletions integration/pkg/sourcereader/evm_source_reader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (

"github.com/ethereum/go-ethereum"
"github.com/ethereum/go-ethereum/common"
"github.com/google/uuid"

"github.com/smartcontractkit/chainlink-ccip/chains/evm/gobindings/generated/latest/onramp"
"github.com/smartcontractkit/chainlink-ccv/protocol"
Expand Down Expand Up @@ -305,9 +306,10 @@ func (r *EVMSourceReader) VerificationTasks(ctx context.Context, fromBlock, toBl

// Create verification task
results = append(results, verifiertypes.VerificationTask{
Message: *decodedMsg,
ReceiptBlobs: receiptBlobs,
BlockNumber: log.BlockNumber,
Message: *decodedMsg,
ReceiptBlobs: receiptBlobs,
BlockNumber: log.BlockNumber,
IdempotencyKey: uuid.NewString(),
})
}
return results, nil
Expand Down
13 changes: 9 additions & 4 deletions integration/storageaccess/aggregator_adapter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ func mapReceiptBlobs(receiptBlobs []protocol.ReceiptWithBlob) ([]*pb.ReceiptBlob
return result, nil
}

func mapCCVDataToCCVNodeDataProto(ccvData protocol.CCVData) (*pb.WriteCommitCCVNodeDataRequest, error) {
func mapCCVDataToCCVNodeDataProto(ccvData protocol.CCVData, idempotencyKey string) (*pb.WriteCommitCCVNodeDataRequest, error) {
receiptBlobs, err := mapReceiptBlobs(ccvData.ReceiptBlobs)
if err != nil {
return nil, err
Expand Down Expand Up @@ -81,14 +81,19 @@ func mapCCVDataToCCVNodeDataProto(ccvData protocol.CCVData) (*pb.WriteCommitCCVN
},
ReceiptBlobs: receiptBlobs,
},
IdempotencyKey: idempotencyKey, // Use provided idempotency key
}, nil
}

// WriteCCVNodeData writes CCV data to the aggregator via gRPC.
func (a *AggregatorWriter) WriteCCVNodeData(ctx context.Context, ccvDataList []protocol.CCVData) error {
func (a *AggregatorWriter) WriteCCVNodeData(ctx context.Context, ccvDataList []protocol.CCVData, idempotencyKeys []string) error {
if len(ccvDataList) != len(idempotencyKeys) {
return fmt.Errorf("ccvDataList and idempotencyKeys must have the same length: got %d and %d", len(ccvDataList), len(idempotencyKeys))
}

a.lggr.Info("Storing CCV data using aggregator ", "count", len(ccvDataList))
for _, ccvData := range ccvDataList {
req, err := mapCCVDataToCCVNodeDataProto(ccvData)
for i, ccvData := range ccvDataList {
req, err := mapCCVDataToCCVNodeDataProto(ccvData, idempotencyKeys[i])
if err != nil {
return err
}
Expand Down
3 changes: 2 additions & 1 deletion protocol/message_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -397,7 +397,8 @@ type QueryResponse struct {
// CCVNodeDataWriter defines the interface for verifiers to store CCV node data.
type CCVNodeDataWriter interface {
// WriteCCVNodeData stores multiple CCV node data entries in the offchain storage
WriteCCVNodeData(ctx context.Context, ccvDataList []CCVData) error
// idempotencyKeys should have the same length as ccvDataList, with each key corresponding to the CCVData at the same index
WriteCCVNodeData(ctx context.Context, ccvDataList []CCVData, idempotencyKeys []string) error
}

// OffchainStorageWriter defines the interface for verifiers to store CCV data.
Expand Down
12 changes: 8 additions & 4 deletions verifier/commit/verifier.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ func (cv *Verifier) ValidateMessage(message protocol.Message) error {
// VerifyMessages verifies a batch of messages using the new chain-agnostic format.
// It processes tasks concurrently and adds results directly to the batcher.
// Returns a BatchResult containing any verification errors that occurred.
func (cv *Verifier) VerifyMessages(ctx context.Context, tasks []verifier.VerificationTask, ccvDataBatcher *batcher.Batcher[protocol.CCVData]) batcher.BatchResult[verifier.VerificationError] {
func (cv *Verifier) VerifyMessages(ctx context.Context, tasks []verifier.VerificationTask, ccvDataBatcher *batcher.Batcher[verifier.CCVDataWithIdempotencyKey]) batcher.BatchResult[verifier.VerificationError] {
if len(tasks) == 0 {
return batcher.BatchResult[verifier.VerificationError]{Items: nil, Error: nil}
}
Expand Down Expand Up @@ -117,7 +117,7 @@ func (cv *Verifier) VerifyMessages(ctx context.Context, tasks []verifier.Verific

// verifyMessage verifies a single message (internal helper)
// Returns an error if verification fails, nil if successful.
func (cv *Verifier) verifyMessage(ctx context.Context, verificationTask verifier.VerificationTask, ccvDataBatcher *batcher.Batcher[protocol.CCVData]) error {
func (cv *Verifier) verifyMessage(ctx context.Context, verificationTask verifier.VerificationTask, ccvDataBatcher *batcher.Batcher[verifier.CCVDataWithIdempotencyKey]) error {
start := time.Now()
message := verificationTask.Message

Expand Down Expand Up @@ -182,8 +182,12 @@ func (cv *Verifier) verifyMessage(ctx context.Context, verificationTask verifier
return fmt.Errorf("failed to create CCV data for message 0x%x: %w", messageID, err)
}

// Add CCVData directly to batcher
if err := ccvDataBatcher.Add(*ccvData); err != nil {
// Add CCVData with idempotency key to batcher
ccvDataWithKey := verifier.CCVDataWithIdempotencyKey{
CCVData: *ccvData,
IdempotencyKey: verificationTask.IdempotencyKey,
}
if err := ccvDataBatcher.Add(ccvDataWithKey); err != nil {
return fmt.Errorf("failed to add CCV data to batcher for message 0x%x (nonce: %d, source chain: %d): %w", messageID, message.Nonce, message.SourceChainSelector, err)
}

Expand Down
2 changes: 1 addition & 1 deletion verifier/interfaces.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ type MessageSigner interface {
type Verifier interface {
// VerifyMessages performs verification of a batch of messages, adding successful results to the batcher.
// Returns a BatchResult containing any verification errors that occurred.
VerifyMessages(ctx context.Context, tasks []VerificationTask, ccvDataBatcher *batcher.Batcher[protocol.CCVData]) batcher.BatchResult[VerificationError]
VerifyMessages(ctx context.Context, tasks []VerificationTask, ccvDataBatcher *batcher.Batcher[CCVDataWithIdempotencyKey]) batcher.BatchResult[VerificationError]
}

// SourceReader defines the interface for reading CCIP messages from source chains.
Expand Down
6 changes: 4 additions & 2 deletions verifier/pkg/common/inmemory_storage.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ func (s *InMemoryOffchainStorage) WaitForStore(ctx context.Context) error {
}

// WriteCCVNodeData stores multiple CCV data entries in the offchain storage.
func (s *InMemoryOffchainStorage) WriteCCVNodeData(ctx context.Context, ccvDataList []protocol.CCVData) error {
func (s *InMemoryOffchainStorage) WriteCCVNodeData(ctx context.Context, ccvDataList []protocol.CCVData, idempotencyKeys []string) error {
if len(ccvDataList) == 0 {
return nil
}
Expand Down Expand Up @@ -385,5 +385,7 @@ type WriterOnlyView struct {
}

func (w *WriterOnlyView) WriteCCVData(ctx context.Context, ccvDataList []protocol.CCVData) error {
return w.storage.WriteCCVNodeData(ctx, ccvDataList)
// Generate empty idempotency keys for in-memory storage (they are not used)
idempotencyKeys := make([]string, len(ccvDataList))
return w.storage.WriteCCVNodeData(ctx, ccvDataList, idempotencyKeys)
}
34 changes: 22 additions & 12 deletions verifier/pkg/common/inmemory_storage_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package common

import (
"context"
"fmt"
"sort"
"testing"
"time"
Expand All @@ -12,6 +13,15 @@ import (
"github.com/smartcontractkit/chainlink-common/pkg/logger"
)

// generateIdempotencyKeys creates test idempotency keys for a given number of items.
func generateIdempotencyKeys(count int, prefix string) []string {
keys := make([]string, count)
for i := range keys {
keys[i] = fmt.Sprintf("%s-key-%d", prefix, i)
}
return keys
}

func createTestMessage(t *testing.T, nonce protocol.Nonce, sourceChainSelector, destChainSelector protocol.ChainSelector) protocol.Message {
// Create empty token transfer
tokenTransfer := protocol.NewEmptyTokenTransfer()
Expand Down Expand Up @@ -92,7 +102,7 @@ func TestInMemoryOffchainStorage_WriteCCVNodeData(t *testing.T) {
}

// Write data
err := storage.WriteCCVNodeData(ctx, testData)
err := storage.WriteCCVNodeData(ctx, testData, generateIdempotencyKeys(len(testData), "test"))
require.NoError(t, err)

// Retrieve and verify
Expand Down Expand Up @@ -142,7 +152,7 @@ func TestInMemoryOffchainStorage_GetCCVDataByTimestamp(t *testing.T) {
}

// Store first data at baseTime
err := storage.WriteCCVNodeData(ctx, testData1)
err := storage.WriteCCVNodeData(ctx, testData1, generateIdempotencyKeys(len(testData1), "test1"))
require.NoError(t, err)

// Update time provider for second batch
Expand All @@ -162,7 +172,7 @@ func TestInMemoryOffchainStorage_GetCCVDataByTimestamp(t *testing.T) {
},
}

err = storage.WriteCCVNodeData(ctx, testData2)
err = storage.WriteCCVNodeData(ctx, testData2, generateIdempotencyKeys(len(testData2), "test2"))
require.NoError(t, err)

// Update time provider for third batch
Expand All @@ -182,7 +192,7 @@ func TestInMemoryOffchainStorage_GetCCVDataByTimestamp(t *testing.T) {
},
}

err = storage.WriteCCVNodeData(ctx, testData3)
err = storage.WriteCCVNodeData(ctx, testData3, generateIdempotencyKeys(len(testData3), "test3"))
require.NoError(t, err)

return storage
Expand Down Expand Up @@ -308,7 +318,7 @@ func TestInMemoryOffchainStorage_GetCCVDataByMessageID(t *testing.T) {
}

// Store data
err := storage.WriteCCVNodeData(ctx, testData)
err := storage.WriteCCVNodeData(ctx, testData, generateIdempotencyKeys(len(testData), "test"))
require.NoError(t, err)

// Test finding existing message
Expand Down Expand Up @@ -362,10 +372,10 @@ func TestInMemoryOffchainStorage_MultipleVerifiers(t *testing.T) {
}

// Write data for both verifiers
err := storage.WriteCCVNodeData(ctx, data1)
err := storage.WriteCCVNodeData(ctx, data1, generateIdempotencyKeys(len(data1), "test1"))
require.NoError(t, err)

err = storage.WriteCCVNodeData(ctx, data2)
err = storage.WriteCCVNodeData(ctx, data2, generateIdempotencyKeys(len(data2), "test2"))
require.NoError(t, err)

// Verify verifier1 data
Expand Down Expand Up @@ -401,7 +411,7 @@ func TestInMemoryOffchainStorage_Clear(t *testing.T) {
},
}

err := storage.WriteCCVNodeData(ctx, testData)
err := storage.WriteCCVNodeData(ctx, testData, generateIdempotencyKeys(len(testData), "test"))
require.NoError(t, err)

// Verify data exists
Expand Down Expand Up @@ -431,7 +441,7 @@ func TestInMemoryOffchainStorage_EmptyData(t *testing.T) {
ctx := context.Background()

// Write empty data should not error
err := storage.WriteCCVNodeData(ctx, []protocol.CCVData{})
err := storage.WriteCCVNodeData(ctx, []protocol.CCVData{}, []string{})
require.NoError(t, err)

// Get data for non-existent verifier
Expand Down Expand Up @@ -462,7 +472,7 @@ func TestInMemoryOffchainStorage_TimestampHandling(t *testing.T) {
},
}

err := storage.WriteCCVNodeData(ctx, testData)
err := storage.WriteCCVNodeData(ctx, testData, generateIdempotencyKeys(len(testData), "test"))
require.NoError(t, err)

// Verify data was stored - timestamp is managed internally by storage entries
Expand Down Expand Up @@ -552,7 +562,7 @@ func setupReaderWithMessagesfunc(t *testing.T, baseTime int64, numMessages int,
}

// Store first data at baseTime
err := storage.WriteCCVNodeData(t.Context(), testData1)
err := storage.WriteCCVNodeData(t.Context(), testData1, generateIdempotencyKeys(len(testData1), "test1"))
require.NoError(t, err)
}

Expand Down Expand Up @@ -654,7 +664,7 @@ func TestEmptyReadsAndReadAfterEmpty(t *testing.T) {
Message: createTestMessage(t, 100, 1, 2),
},
}
err := storage.WriteCCVNodeData(t.Context(), testData1)
err := storage.WriteCCVNodeData(t.Context(), testData1, generateIdempotencyKeys(len(testData1), "test1"))
require.NoError(t, err)

// Next read should return the new message
Expand Down
Loading
Loading