Skip to content

Conversation

plopezlpz
Copy link
Contributor

@plopezlpz plopezlpz commented May 11, 2025

This PR is to allow communities to be eventually moved to shards 128 and 256. We will follow these migration stages:

  1. Listen for community contenttopics in the current shards (i.e.: 32, 64 and custom) together with the new shards (128 and 256).
  2. After the majority of nodes have been upgraded we can start the second stage, prepare a PR for nodes to send on those new shards (128 and 256).
  3. After the majority of nodes have upgraded from step 2, we can prepare another PR for nodes to stop listening for community messages in shards 32 and 64.

TODO:

Relates to #6384

@status-im-auto
Copy link
Member

status-im-auto commented May 11, 2025

Jenkins Builds

Click to see older builds (121)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 1c84e47 #1 2025-05-11 17:18:48 ~2 min ios 📦zip
✔️ 1c84e47 #1 2025-05-11 17:19:00 ~3 min android 📦aar
✔️ 1c84e47 #1 2025-05-11 17:20:01 ~4 min macos 📦zip
✖️ 1c84e47 #1 2025-05-11 17:20:32 ~4 min tests 📄log
✔️ 1c84e47 #1 2025-05-11 17:20:39 ~4 min windows 📦zip
✔️ 1c84e47 #1 2025-05-11 17:21:54 ~5 min linux 📦zip
✔️ 1c84e47 #1 2025-05-11 17:21:56 ~5 min macos 📦zip
✔️ 1c84e47 #1 2025-05-11 17:26:36 ~10 min tests-rpc 📄log
✔️ 37ef179 #2 2025-05-12 03:06:06 ~2 min ios 📦zip
✔️ 37ef179 #2 2025-05-12 03:06:24 ~2 min android 📦aar
✔️ 37ef179 #2 2025-05-12 03:06:48 ~3 min windows 📦zip
✔️ 37ef179 #2 2025-05-12 03:07:37 ~4 min macos 📦zip
✔️ 37ef179 #2 2025-05-12 03:09:10 ~5 min macos 📦zip
✔️ 37ef179 #2 2025-05-12 03:09:12 ~5 min linux 📦zip
✔️ 37ef179 #2 2025-05-12 03:14:26 ~10 min tests-rpc 📄log
✔️ 37ef179 #2 2025-05-12 03:42:14 ~38 min tests 📄log
✔️ 9db6cb8 #3 2025-05-12 08:07:55 ~2 min ios 📦zip
✔️ 9db6cb8 #3 2025-05-12 08:08:12 ~2 min android 📦aar
✔️ 9db6cb8 #3 2025-05-12 08:08:33 ~3 min windows 📦zip
✔️ 9db6cb8 #3 2025-05-12 08:10:07 ~4 min macos 📦zip
✔️ 9db6cb8 #3 2025-05-12 08:10:55 ~5 min macos 📦zip
✔️ 9db6cb8 #3 2025-05-12 08:11:00 ~5 min linux 📦zip
✔️ 9db6cb8 #3 2025-05-12 08:15:47 ~10 min tests-rpc 📄log
✔️ 9db6cb8 #3 2025-05-12 08:44:31 ~39 min tests 📄log
✔️ 6634a74 #4 2025-05-12 08:19:36 ~2 min ios 📦zip
✔️ 6634a74 #4 2025-05-12 08:19:51 ~2 min android 📦aar
✔️ 6634a74 #4 2025-05-12 08:20:15 ~3 min windows 📦zip
✔️ 6634a74 #4 2025-05-12 08:20:52 ~3 min macos 📦zip
✔️ 6634a74 #4 2025-05-12 08:22:40 ~5 min linux 📦zip
✔️ 6634a74 #4 2025-05-12 08:22:46 ~5 min macos 📦zip
✔️ 6634a74 #4 2025-05-12 08:26:33 ~9 min tests-rpc 📄log
✔️ 6634a74 #4 2025-05-12 09:23:24 ~38 min tests 📄log
5efe5cb #5 2025-06-03 12:38:42 ~1 min android 📄log
5efe5cb #1 2025-06-03 12:38:53 ~1 min linux 📄log
5efe5cb #5 2025-06-03 12:39:09 ~1 min linux 📄log
✖️ 5efe5cb #5 2025-06-03 12:39:23 ~1 min tests 📄log
5efe5cb #5 2025-06-03 12:39:48 ~2 min ios 📄log
5efe5cb #5 2025-06-03 12:40:08 ~2 min macos 📄log
5efe5cb #5 2025-06-03 12:40:33 ~3 min windows 📄log
5efe5cb #5 2025-06-03 12:40:39 ~3 min macos 📄log
✖️ 5efe5cb #5 2025-06-03 12:48:26 ~11 min tests-rpc 📄log
✔️ 1db5780 #6 2025-06-03 13:05:15 ~1 min android 📦aar
✔️ 1db5780 #6 2025-06-03 13:06:25 ~3 min ios 📦zip
✔️ 1db5780 #6 2025-06-03 13:06:40 ~3 min macos 📦zip
✔️ 1db5780 #6 2025-06-03 13:06:40 ~3 min linux 📦zip
✔️ 1db5780 #6 2025-06-03 13:07:20 ~3 min macos 📦zip
✔️ 1db5780 #6 2025-06-03 13:08:20 ~4 min windows 📦zip
✔️ 1db5780 #6 2025-06-03 13:12:35 ~9 min tests-rpc 📄log
✔️ 1db5780 #2 2025-06-03 13:15:18 ~12 min linux 📦zip
✔️ 1db5780 #6 2025-06-03 13:35:00 ~31 min tests 📄log
✔️ f66cdbf #7 2025-08-19 07:07:51 ~3 min android 📦aar
✔️ f66cdbf #7 2025-08-19 07:07:52 ~2 min macos/status-go 📦zip
✔️ f66cdbf #7 2025-08-19 07:07:56 ~3 min linux/status-go 📦zip
✔️ f66cdbf #7 2025-08-19 07:09:07 ~4 min macos/status-go 📦zip
✔️ f66cdbf #7 2025-08-19 07:09:22 ~4 min ios 📦zip
✔️ f66cdbf #7 2025-08-19 07:09:40 ~4 min windows/status-go 📦zip
✖️ f66cdbf #7 2025-08-19 07:13:30 ~8 min tests-rpc 📄log
✔️ f66cdbf #3 2025-08-19 07:14:57 ~10 min linux/nwaku 📦zip
✔️ f66cdbf #7 2025-08-19 07:26:48 ~21 min tests 📄log
✔️ c2c290b #8 2025-08-19 07:27:06 ~3 min linux/status-go 📦zip
✔️ c2c290b #8 2025-08-19 07:27:12 ~3 min android 📦aar
✔️ c2c290b #8 2025-08-19 07:27:15 ~3 min macos/status-go 📦zip
✔️ c2c290b #8 2025-08-19 07:28:03 ~3 min macos/status-go 📦zip
✔️ c2c290b #8 2025-08-19 07:28:29 ~4 min ios 📦zip
✔️ c2c290b #8 2025-08-19 07:28:50 ~4 min windows/status-go 📦zip
✔️ c2c290b #4 2025-08-19 07:31:49 ~7 min linux/nwaku 📦zip
✖️ c2c290b #8 2025-08-19 07:32:31 ~8 min tests-rpc 📄log
✖️ c2c290b #8 2025-08-19 07:48:50 ~21 min tests 📄log
✔️ ba90867 #9 2025-08-19 08:09:49 ~2 min android 📦aar
✔️ ba90867 #9 2025-08-19 08:10:24 ~2 min macos/status-go 📦zip
✔️ ba90867 #9 2025-08-19 08:10:33 ~3 min linux/status-go 📦zip
✔️ ba90867 #9 2025-08-19 08:11:25 ~3 min macos/status-go 📦zip
✔️ ba90867 #9 2025-08-19 08:12:05 ~4 min windows/status-go 📦zip
✔️ ba90867 #9 2025-08-19 08:14:10 ~6 min ios 📦zip
✔️ ba90867 #5 2025-08-19 08:15:15 ~7 min linux/nwaku 📦zip
✖️ ba90867 #9 2025-08-19 08:15:43 ~8 min tests-rpc 📄log
✔️ ba90867 #9 2025-08-19 08:29:01 ~21 min tests 📄log
✔️ a553d1a #10 2025-08-20 11:35:00 ~2 min android 📦aar
✔️ a553d1a #10 2025-08-20 11:35:17 ~2 min linux/status-go 📦zip
✔️ a553d1a #10 2025-08-20 11:36:38 ~3 min macos/status-go 📦zip
✔️ a553d1a #10 2025-08-20 11:36:47 ~3 min macos/status-go 📦zip
✔️ a553d1a #10 2025-08-20 11:37:09 ~4 min ios 📦zip
✔️ a553d1a #10 2025-08-20 11:37:38 ~4 min windows/status-go 📦zip
✔️ a553d1a #10 2025-08-20 11:42:44 ~9 min tests-rpc 📄log
✔️ a553d1a #6 2025-08-20 11:44:49 ~12 min linux/nwaku 📦zip
✖️ a553d1a #10 2025-08-20 12:00:24 ~27 min tests 📄log
✔️ 023cabc #11 2025-08-20 11:44:12 ~2 min android 📦aar
✔️ 023cabc #11 2025-08-20 11:44:55 ~2 min macos/status-go 📦zip
✔️ 023cabc #11 2025-08-20 11:45:26 ~3 min linux/status-go 📦zip
✔️ 023cabc #11 2025-08-20 11:45:55 ~3 min macos/status-go 📦zip
✔️ 023cabc #11 2025-08-20 11:46:18 ~4 min ios 📦zip
✔️ 023cabc #11 2025-08-20 11:46:50 ~4 min windows/status-go 📦zip
✔️ 023cabc #11 2025-08-20 11:52:57 ~10 min tests-rpc 📄log
✔️ 023cabc #7 2025-08-20 11:54:47 ~9 min linux/nwaku 📦zip
✔️ 85170ab #12 2025-08-20 11:49:22 ~2 min android 📦aar
✔️ 85170ab #12 2025-08-20 11:49:50 ~2 min macos/status-go 📦zip
✔️ 85170ab #12 2025-08-20 11:50:05 ~3 min linux/status-go 📦zip
85170ab #12 2025-08-20 11:50:22 ~3 min macos/status-go 📄log
✔️ 85170ab #12 2025-08-20 11:51:14 ~4 min ios 📦zip
✔️ 85170ab #12 2025-08-20 11:52:01 ~4 min windows/status-go 📦zip
✖️ 85170ab #12 2025-08-20 12:00:46 ~7 min tests-rpc 📄log
✔️ 85170ab #8 2025-08-20 12:04:56 ~10 min linux/nwaku 📦zip
✔️ f927996 #13 2025-08-20 12:02:59 ~2 min android 📦aar
✔️ f927996 #13 2025-08-20 12:03:19 ~2 min macos/status-go 📦zip
f927996 #13 2025-08-20 12:03:47 ~3 min macos/status-go 📄log
✔️ f927996 #13 2025-08-20 12:03:58 ~3 min linux/status-go 📦zip
✔️ f927996 #13 2025-08-20 12:05:09 ~4 min windows/status-go 📦zip
✔️ f927996 #13 2025-08-20 12:06:52 ~6 min ios 📦zip
✔️ f927996 #13 2025-08-20 12:07:55 ~6 min tests-rpc 📄log
✔️ f927996 #9 2025-08-20 12:15:06 ~10 min linux/nwaku 📦zip
✔️ f927996 #11 2025-08-20 12:25:16 ~24 min tests 📄log
f927996 #14 2025-08-20 12:34:20 ~3 min macos/status-go 📄log
✔️ 4744db8 #14 2025-08-20 12:45:12 ~3 min android 📦aar
✔️ 4744db8 #14 2025-08-20 12:45:22 ~3 min macos/status-go 📦zip
✔️ 4744db8 #14 2025-08-20 12:45:29 ~3 min linux/status-go 📦zip
4744db8 #15 2025-08-20 12:45:35 ~3 min macos/status-go 📄log
✔️ 4744db8 #14 2025-08-20 12:47:13 ~4 min windows/status-go 📦zip
✔️ 4744db8 #14 2025-08-20 12:48:30 ~6 min ios 📦zip
✔️ 4744db8 #14 2025-08-20 12:49:25 ~7 min tests-rpc 📄log
✔️ 4744db8 #10 2025-08-20 12:52:13 ~10 min linux/nwaku 📦zip
✔️ 4744db8 #12 2025-08-20 13:08:12 ~25 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ d2bf53e #15 2025-08-22 09:12:57 ~3 min android 📦aar
✔️ d2bf53e #15 2025-08-22 09:13:05 ~3 min linux/status-go 📦zip
✔️ d2bf53e #15 2025-08-22 09:13:57 ~3 min macos/status-go 📦zip
✔️ d2bf53e #16 2025-08-22 09:14:05 ~4 min macos/status-go 📦zip
✔️ d2bf53e #15 2025-08-22 09:14:25 ~4 min ios 📦zip
✔️ d2bf53e #15 2025-08-22 09:14:45 ~4 min windows/status-go 📦zip
✖️ d2bf53e #15 2025-08-22 09:18:14 ~8 min tests-rpc 📄log
✔️ d2bf53e #11 2025-08-22 09:20:05 ~10 min linux/nwaku 📦zip
✔️ d2bf53e #13 2025-08-22 09:34:55 ~24 min tests 📄log
✔️ 1748e23 #16 2025-08-22 12:38:32 ~2 min android 📦aar
✔️ 1748e23 #16 2025-08-22 12:39:27 ~3 min macos/status-go 📦zip
✔️ 1748e23 #16 2025-08-22 12:39:52 ~3 min linux/status-go 📦zip
✔️ 1748e23 #16 2025-08-22 12:40:55 ~4 min windows/status-go 📦zip
✔️ 1748e23 #16 2025-08-22 12:41:02 ~4 min ios 📦zip
✔️ 1748e23 #12 2025-08-22 12:48:51 ~12 min linux/nwaku 📦zip
✔️ 1748e23 #16 2025-08-22 12:48:59 ~12 min tests-rpc 📄log
✖️ 1748e23 #14 2025-08-22 13:08:35 ~32 min tests 📄log
✔️ 1748e23 #15 2025-08-22 15:00:08 ~25 min tests 📄log

@plopezlpz plopezlpz force-pushed the feat/communities-in-own-shard-1 branch from 1c84e47 to 37ef179 Compare May 12, 2025 03:03
@plopezlpz plopezlpz changed the title Feat/communities-in-own-shard-1 feat(sharding)_:communities-in-own-shard-1 May 12, 2025
@plopezlpz plopezlpz changed the title feat(sharding)_:communities-in-own-shard-1 feat(sharding)_: communities-in-own-shard-1 May 12, 2025
Copy link

codecov bot commented May 12, 2025

Codecov Report

❌ Patch coverage is 66.06061% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.89%. Comparing base (96e8990) to head (1748e23).

Files with missing lines Patch % Lines
messaging/layers/transport/filters_manager.go 66.66% 16 Missing ⚠️
protocol/messenger_store_node_request_manager.go 55.55% 12 Missing and 4 partials ⚠️
messaging/layers/transport/transport.go 36.84% 7 Missing and 5 partials ⚠️
protocol/messenger_filter_init.go 12.50% 4 Missing and 3 partials ⚠️
messaging/waku/shard.go 73.68% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6573      +/-   ##
===========================================
- Coverage    59.51%   58.89%   -0.63%     
===========================================
  Files          826      826              
  Lines       123599   123665      +66     
===========================================
- Hits         73565    72830     -735     
- Misses       42488    43248     +760     
- Partials      7546     7587      +41     
Flag Coverage Δ
functional 30.46% <61.21%> (-0.16%) ⬇️
unit 55.49% <61.81%> (-0.43%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
messaging/adapters/filters.go 100.00% <100.00%> (ø)
messaging/api.go 78.16% <100.00%> (ø)
messaging/types/filters.go 100.00% <ø> (ø)
messaging/types/shard.go 100.00% <100.00%> (ø)
protocol/messenger_communities.go 45.11% <100.00%> (-16.09%) ⬇️
protocol/messenger_peers.go 11.11% <100.00%> (ø)
messaging/waku/shard.go 68.29% <73.68%> (-0.76%) ⬇️
protocol/messenger_filter_init.go 61.27% <12.50%> (-14.80%) ⬇️
messaging/layers/transport/transport.go 65.13% <36.84%> (-7.21%) ⬇️
messaging/layers/transport/filters_manager.go 77.86% <66.66%> (-5.87%) ⬇️
... and 1 more

... and 49 files with indirect coverage changes

if err != nil {
return nil, err
}

return filter, nil
// TODO temporary so not changing the return type, otherwise we should return a slice
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment seems not applied here.

@@ -24,7 +24,16 @@ func (m *Messenger) InitFilters() error {
rand.Seed(time.Now().Unix())

// Community requests will arrive in this pubsub topic
if err := m.SubscribeToPubsubTopic(wakuv2.DefaultNonProtectedPubsubTopic(), nil); err != nil {
// TODO depracate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that there are a few todos in the PR, would be good to have issue linked here to resolve the todos.

@plopezlpz plopezlpz force-pushed the feat/communities-in-own-shard-1 branch from c2c290b to ba90867 Compare August 19, 2025 08:07
@plopezlpz plopezlpz changed the title feat(sharding)_: communities-in-own-shard-1 feat: communities-in-own-shard-1 Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants