Skip to content

WIP - parallel tx workers tastora test #4639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 16 commits into
base: release/v0.28.0
Choose a base branch
from
Draft

WIP - parallel tx workers tastora test #4639

wants to merge 16 commits into from

Conversation

@renaynay
Copy link
Member

@renaynay renaynay commented Oct 16, 2025

No description provided.

gupadhyaya and others added 11 commits October 15, 2025 14:47
@gupadhyaya @renaynay
queued submission tastora tests
764e14d
@gupadhyaya @renaynay
simplify test to only parallel submission case
bc47022
@gupadhyaya @renaynay
lint imports fix
e48a07d
@gupadhyaya @renaynay
more attempt to fix lint
c71989c
@renaynay
fix(state): pass long lived core accessor ctx into txclient setup as …
511751c 
…ctx is now used to control lifecycle of tx workers (#4635)
@renaynay
rene commit
9063409
@renaynay
cleanup
068e181
@gupadhyaya
feat: add parallel transaction submission test with worker accounts
de14980 
- Add TearDownSuite method to BlobTestSuite for proper cleanup
- Add comprehensive parallel transaction test in TransactionTestSuite
- Add WithTxWorkerAccounts option to Framework for configuring worker accounts
- Add TxWorkerAccounts field to Config struct with validation
- Test verifies --tx.worker.accounts feature works correctly
- Includes proper error handling and detailed logging for debugging
@gupadhyaya
chore: update defaultNodeTag to v0.28.1-arabica
30b0445 
- Update to use v0.28.1-arabica which contains fixes
- This version includes the queued submission feature with bug fixes
@gupadhyaya
fixes
f397946
@gupadhyaya
cleanup and fix test
6d9fd78
@github-actions github-actions bot added the kind:break! Attached to breaking PRs label Oct 16, 2025
renaynay
renaynay commented Oct 16, 2025
View reviewed changes
nodebuilder/tests/tastora/framework.go
// Add a delay to allow the node to fully initialize, especially when using worker accounts
if f.config.TxWorkerAccounts > 0 {
f.t.Logf("Waiting for bridge node with %d worker accounts to initialize...", f.config.TxWorkerAccounts)
time.Sleep(10 * time.Second)
Copy link
Member Author

@renaynay renaynay Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't do anything

renaynay
renaynay commented Oct 16, 2025
View reviewed changes
nodebuilder/tests/tastora/framework.go Outdated

// Retry transaction broadcasting with exponential backoff
var resp sdk.TxResponse
broadcastRetries := 3
Copy link
Member Author

@renaynay renaynay Oct 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general comment here is we should try to use TxClient inside of our framework here so we don't have to touch app code. Even if that means building a standalone txclient using the validator's signer.

@renaynay
Copy link
Member Author

renaynay commented Oct 16, 2025

Current progress is:

  • first round of txs doesn't work due to somehow submitting the exact same tx hash (from every single worker???)
  • 2nd and 3rd rounds work!
  • then eventually insufficient funds, borked
=== RUN   TestTransactionTestSuite/TestSubmitParallelTxs
    transaction_test.go:91: Starting round 1 with 8 parallel workers
    transaction_test.go:109: Round 1, Worker 5: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:109: Round 1, Worker 8: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:109: Round 1, Worker 3: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:109: Round 1, Worker 2: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:109: Round 1, Worker 1: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:109: Round 1, Worker 4: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:109: Round 1, Worker 6: FAILED - failed to setup a tx client: failed to start tx queue: failed to initialize worker accounts: failed to fund and grant worker accounts: failed to submit initialization transaction: broadcast tx error: hash DACDD8407E61C2CDA64264A966F3E6FEB6E72F2A465D429B0FD9FEC9DB8AE8CD error:  code: 19
    transaction_test.go:112: Round 1, Worker 7: SUCCESS - height 59
    transaction_test.go:117: Round 1 completed
    transaction_test.go:91: Starting round 2 with 8 parallel workers
    transaction_test.go:112: Round 2, Worker 6: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 4: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 7: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 3: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 5: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 8: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 1: SUCCESS - height 60
    transaction_test.go:112: Round 2, Worker 2: SUCCESS - height 60
    transaction_test.go:117: Round 2 completed
    transaction_test.go:91: Starting round 3 with 8 parallel workers
    transaction_test.go:112: Round 3, Worker 6: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 4: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 7: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 1: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 8: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 3: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 5: SUCCESS - height 61
    transaction_test.go:112: Round 3, Worker 2: SUCCESS - height 61
    transaction_test.go:117: Round 3 completed
    transaction_test.go:91: Starting round 4 with 8 parallel workers
    transaction_test.go:109: Round 4, Worker 2: FAILED - failed to submit blobs: broadcast tx error: hash 659BCFB28167FB15D1BA824AE2A229A4EEA6591B43456EA91C56A4D06B17C0DD error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 4, Worker 3: FAILED - failed to submit blobs: broadcast tx error: hash 916C8B72241F26E03BD90CE1FB8A43B3455D1F19E539B6CB0620FD294E0BB576 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 4, Worker 6: FAILED - failed to submit blobs: broadcast tx error: hash FEB191370F74A3FF03A3744F43D891D3C7E07FAEDBE17CB50455137BAA8AB0F8 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 4, Worker 4: FAILED - failed to submit blobs: broadcast tx error: hash 1323FF3646163662E601B95F494664B30FAB42DB16E18216430C7DEE64485ECA error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 4, Worker 7: FAILED - failed to submit blobs: broadcast tx error: hash CCEBD5A6F894B8570CB8F503DE4ECAD271FAAC423C876269D4E6AE5A7C55D059 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 4, Worker 8: FAILED - failed to submit blobs: broadcast tx error: hash AE5F0BA78814558B471AE794F47508E0C46DB7843D86B52AE1134D738AC4DDD3 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:112: Round 4, Worker 5: SUCCESS - height 62
    transaction_test.go:112: Round 4, Worker 1: SUCCESS - height 62
    transaction_test.go:117: Round 4 completed
    transaction_test.go:91: Starting round 5 with 8 parallel workers
    transaction_test.go:109: Round 5, Worker 5: FAILED - failed to submit blobs: broadcast tx error: hash 659BCFB28167FB15D1BA824AE2A229A4EEA6591B43456EA91C56A4D06B17C0DD error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 1: FAILED - failed to submit blobs: broadcast tx error: hash 916C8B72241F26E03BD90CE1FB8A43B3455D1F19E539B6CB0620FD294E0BB576 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 8: FAILED - failed to submit blobs: broadcast tx error: hash FEB191370F74A3FF03A3744F43D891D3C7E07FAEDBE17CB50455137BAA8AB0F8 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 3: FAILED - failed to submit blobs: broadcast tx error: hash 1323FF3646163662E601B95F494664B30FAB42DB16E18216430C7DEE64485ECA error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 2: FAILED - failed to submit blobs: broadcast tx error: hash CCEBD5A6F894B8570CB8F503DE4ECAD271FAAC423C876269D4E6AE5A7C55D059 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 4: FAILED - failed to submit blobs: broadcast tx error: hash AE5F0BA78814558B471AE794F47508E0C46DB7843D86B52AE1134D738AC4DDD3 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 6: FAILED - failed to submit blobs: broadcast tx error: hash 40F79F245EE8417681DEC11590C4701D93FC7F8B395BFC623EDD14D4707D4F10 error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:109: Round 5, Worker 7: FAILED - failed to submit blobs: broadcast tx error: hash 46117FC4B69B9F439BEF5A018765A3D81383251D9BE3ADBC41C83A47BD23F0DD error: spendable balance 4999974793utia is smaller than 5000000000utia: insufficient funds: insufficient funds code: 5
    transaction_test.go:117: Round 5 completed

@renaynay
Copy link
Member Author

renaynay commented Oct 16, 2025

update:

				txConfig := state.NewTxConfig(state.WithGasPrice(0.04))

fixed this bc default config was leading to some weird gas price estimation failures where we were submitting txs with insufficient fee returned... odd

Need to debug why the estimation is borked in tastora

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Wondertan Wondertan Awaiting requested review from Wondertan Wondertan will be requested when the pull request is marked ready for review Wondertan is a code owner

@vgonkivs vgonkivs Awaiting requested review from vgonkivs vgonkivs will be requested when the pull request is marked ready for review vgonkivs is a code owner

@walldiss walldiss Awaiting requested review from walldiss walldiss will be requested when the pull request is marked ready for review walldiss is a code owner

@gupadhyaya gupadhyaya Awaiting requested review from gupadhyaya gupadhyaya will be requested when the pull request is marked ready for review gupadhyaya is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

kind:break! Attached to breaking PRs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@renaynay @gupadhyaya