Range based syncing

[nogringo]

No description provided.

[codecov]

Codecov Report

❌ Patch coverage is 89.75410% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.38%. Comparing base (75228ab) to head (0ea6538).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...b/domain_layer/entities/filter_fetched_ranges.dart	82.82%	17 Missing ⚠️
...k/lib/domain_layer/usecases/requests/requests.dart	76.92%	6 Missing ⚠️
.../repositories/cache_manager/mem_cache_manager.dart	92.85%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #352      +/-   ##
==========================================
+ Coverage   71.68%   72.38%   +0.70%     
==========================================
  Files         142      144       +2     
  Lines        5301     5545     +244     
==========================================
+ Hits         3800     4014     +214     
- Misses       1501     1531      +30     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[1-leo]

Can you explain what happens on a high level? Is this only for queries or also for streams?
As I understand it, it's an optional usecase to help with range based syncing, right?
From the db structure its independent of events, and the filtersHash is stored?
I don't quite understand how this would work with changing environments. E.g., the relay has more events for the same query than 5 minutes ago.

It looks like a good approach, but I need to understand it better.

[nogringo]

High-level explanation

This PR introduces a coverage tracking system that records which time ranges have been successfully fetched from each relay for a given filter.

Queries and streams

The coverage is updated when NDK receive EOSE so it's for queries and streams.
Should we update the coverage on stream close / ndk destroy ? and / or during the stream ?

Optional usecase

Yes it's optional, the dev can use it or not.
Should we add an option to enable auto coverage on queries / streams ?

It's the first brick that we need to implement the sync usecase or to do more efficient queries and streams.

DB structure

Yes, it's independent of events.

Changing environments (new events in a previous query)

The coverage tracks what we asked for, not what exists.
There is no perfect solution in a decentralized network, the coverage usecase is here to get a picture of what time range we already fetched. And so if we want we can not refecth these ranges.
Maybe we need to save the fetched date for each time range ?

[1-leo]

thx for explaining!
Could this be integrated into our existing caching mechanism? To me, this sounds like a smarter cache.
Otherwise I am for merging (maybe mark it as @experimental) as long as it is optional. We can then better test it in realworld scenarios.

[frnandu]

• add experimental tag
• better naming for coverage usecase
• make it optional in Ndk config

[frnandu]

don't cache implementation need to implement the new methods?
I have compilation errors on isar/objectbox/sembast packages when checking out this branch....