Skip to content

Cosmetic filter fb #514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Sep 16, 2025
Merged

Cosmetic filter fb #514

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3419a9d
Cosmetic filters in a flatbuffer
atuchin-m Aug 15, 2025
c2bf91e
use last byte to encode permissions
atuchin-m Aug 18, 2025
c9d417c
unstage flat_map
atuchin-m Aug 18, 2025
50edf82
return flat map builder
atuchin-m Aug 25, 2025
16013a4
decompose network filters
atuchin-m Aug 25, 2025
d4dd188
decompose cosmetic filters
atuchin-m Aug 25, 2025
28f6c9b
cargo fmt
atuchin-m Aug 25, 2025
d87eeb3
Fix cargo clippy
atuchin-m Aug 26, 2025
db99da0
Fix cargo fmt & some review issues
atuchin-m Aug 26, 2025
1b51596
move make_flatbuffer to engine.rs
atuchin-m Aug 26, 2025
72e699f
update comments
atuchin-m Aug 26, 2025
35f30ff
add more tests
atuchin-m Aug 26, 2025
4ce4fb0
cargo fmt
atuchin-m Aug 26, 2025
02efd49
Fix doc format in .fbs
atuchin-m Sep 11, 2025
07888d9
add filter_data_context.rs
atuchin-m Sep 11, 2025
c08f3f1
Add hash to serialized data
atuchin-m Sep 11, 2025
7fd35dd
update hashes in tests
atuchin-m Sep 11, 2025
c60139b
move mod for generated code
atuchin-m Sep 11, 2025
93bb457
return ADBLOCK_RUST_DAT_MAGIC
atuchin-m Sep 15, 2025
26af341
Move version to .dat header
atuchin-m Sep 16, 2025
abd58d9
set ADBLOCK_FLATBUFFER_VERSION=2
atuchin-m Sep 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion benches/bench_rules.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ fn blocker_new(c: &mut Criterion) {
.collect();
let brave_list_rules: Vec<_> = rules_from_lists(&["data/brave/brave-main-list.txt"]).collect();
let engine = Engine::from_rules(&brave_list_rules, Default::default());
let engine_serialized = engine.serialize().unwrap();
let engine_serialized = engine.serialize().to_vec();

group.bench_function("el+ep", move |b| b.iter(|| get_engine(&easylist_rules)));
group.bench_function("brave-list", move |b| {
Expand Down
12 changes: 6 additions & 6 deletions benches/bench_serialization.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,19 @@ fn serialization(c: &mut Criterion) {
]);

let engine = Engine::from_rules(full_rules, Default::default());
b.iter(|| assert!(!engine.serialize().unwrap().is_empty()))
b.iter(|| assert!(!engine.serialize().to_vec().is_empty()))
});
group.bench_function("el", move |b| {
let full_rules = rules_from_lists(&["data/easylist.to/easylist/easylist.txt"]);

let engine = Engine::from_rules(full_rules, Default::default());
b.iter(|| assert!(!engine.serialize().unwrap().is_empty()))
b.iter(|| assert!(!engine.serialize().to_vec().is_empty()))
});
group.bench_function("slimlist", move |b| {
let full_rules = rules_from_lists(&["data/slim-list.txt"]);

let engine = Engine::from_rules(full_rules, Default::default());
b.iter(|| assert!(!engine.serialize().unwrap().is_empty()))
b.iter(|| assert!(!engine.serialize().to_vec().is_empty()))
});

group.finish();
Expand All @@ -48,7 +48,7 @@ fn deserialization(c: &mut Criterion) {
]);

let engine = Engine::from_rules(full_rules, Default::default());
let serialized = engine.serialize().unwrap();
let serialized = engine.serialize().to_vec();

b.iter(|| {
let mut deserialized = Engine::default();
Expand All @@ -59,7 +59,7 @@ fn deserialization(c: &mut Criterion) {
let full_rules = rules_from_lists(&["data/easylist.to/easylist/easylist.txt"]);

let engine = Engine::from_rules(full_rules, Default::default());
let serialized = engine.serialize().unwrap();
let serialized = engine.serialize().to_vec();

b.iter(|| {
let mut deserialized = Engine::default();
Expand All @@ -70,7 +70,7 @@ fn deserialization(c: &mut Criterion) {
let full_rules = rules_from_lists(&["data/slim-list.txt"]);

let engine = Engine::from_rules(full_rules, Default::default());
let serialized = engine.serialize().unwrap();
let serialized = engine.serialize().to_vec();

b.iter(|| {
let mut deserialized = Engine::default();
Expand Down
2 changes: 1 addition & 1 deletion examples/generate-dat.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ fn main() {
)
.unwrap();
assert!(engine.check_network_request(&request).exception.is_some());
let serialized = engine.serialize().expect("Could not serialize!");
let serialized = engine.serialize().to_vec();

// Write to file
let mut file = File::create("engine.dat").expect("Could not create serialization file");
Expand Down
2 changes: 1 addition & 1 deletion js/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ fn engine_url_cosmetic_resources(mut cx: FunctionContext) -> JsResult<JsValue> {
fn engine_serialize(mut cx: FunctionContext) -> JsResult<JsArrayBuffer> {
let this = cx.argument::<JsBox<Engine>>(0)?;
let serialized = if let Ok(engine) = this.0.lock() {
engine.serialize().unwrap()
engine.serialize().to_vec()
} else {
cx.throw_error("Failed to acquire lock on engine")?
};
Expand Down
12 changes: 7 additions & 5 deletions src/blocker.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ use serde::Serialize;
use std::collections::HashSet;
use std::ops::DerefMut;

use crate::filters::fb_builder::NetworkFilterListId;
use crate::filters::fb_network::FilterDataContextRef;
use crate::filters::fb_network_builder::NetworkFilterListId;
use crate::filters::network::NetworkFilterMaskHelper;
use crate::network_filter_list::NetworkFilterList;
use crate::regex_manager::{RegexManager, RegexManagerDiscardPolicy};
Expand Down Expand Up @@ -440,11 +440,13 @@ impl Blocker {
network_filters: Vec<crate::filters::network::NetworkFilter>,
options: &BlockerOptions,
) -> Self {
use crate::filters::{fb_builder::make_flatbuffer, fb_network::FilterDataContext};
use crate::engine::Engine;
use crate::FilterSet;

let memory = make_flatbuffer(network_filters, options.enable_optimizations);
let filter_data_context = FilterDataContext::new(memory);
Self::from_context(filter_data_context)
let mut filter_set = FilterSet::new(true);
filter_set.network_filters = network_filters;
Copy link
Collaborator

@antonok-edm antonok-edm Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit odd to have Blocker::new() build a FilterSet, then manually move network filters into the set, then build an Engine, and finally make the blocker from the engine. Understood this is test only, but perhaps we should consider a different constructor method for the tests.

Copy link
Collaborator Author

@atuchin-m atuchin-m Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that test ideally these test contractors should be removed in favor of building the engine directly (or another high level test method).
Although, preserving them allow to reduce the diff in the PR (we don't need to rewrite a lot of tests). So I suggest to change it in another PR.

P.S. We can avoid using Engine completely because it's a flatbuffer root table. And we need to get serialized flatbuffer data.

let engine = Engine::from_filter_set(filter_set, options.enable_optimizations);
Self::from_context(engine.filter_data_context())
}

pub fn use_tags(&mut self, tags: &[&str]) {
Expand Down
Loading
Loading