You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Create a bloom filter of taken usernames with a 1 in 1000 chance of false positives and 10000 initial capacity.
70
+
* Create a bloom filter to track the usernames which are taken with a 1 in 1000 chance of a false positive and 10000 as the initial capacity.
84
71
```bash
85
72
127.0.0.1:6379> BF.RESERVE usernames 0.001 10000
86
73
OK
@@ -113,12 +100,12 @@ OK
113
100
3) (integer) 1
114
101
```
115
102
116
-
*Get how many users have been created
103
+
*Check how many users have been added to the filter
117
104
```bash
118
105
127.0.0.1:6379> BF.CARD usernames
119
106
(integer) 4
120
107
```
121
-
*Get the information surrounding your bloom filter of users
108
+
*View your bloom filter object's usage:
122
109
```bash
123
110
127.0.0.1:6379> BF.INFO usernames
124
111
1) Capacity
@@ -138,12 +125,12 @@ OK
138
125
15) Max scaled capacity
139
126
16) (integer) 26214300
140
127
```
141
-
* If you want to limit the number of people who can sign upyou can create a non scaling filter. This will create a filter than can only have 1000 items added with a 1 in 10,000 chance of a false positive.
128
+
* If you know the exact number of users who will sign up, or if want to limit the number of items regardless, you can create a non scaling bloom filter. This will create a filter that has capacity for only 1000 items with a 1 in 100,000 chance of a false positive.
* If you anticipate once initial users have signed up after a while you will get a large increase in users you can set a custom expansion rate.
133
+
* If you anticipate that, once initial users have signed up, you will get a large increase in users after a while, you can set a custom expansion rate.
@@ -227,27 +214,27 @@ Example of default bloom filter information:
227
214
228
215
### When to adjust default configurations
229
216
230
-
Adjusting the default configurations can be beneficial in several scenarios:
217
+
Adjusting the default configurations below can be used to update the behavior of the default bloom object created and are beneficial in different scenarios:
231
218
232
-
-**Higher capacity (bf.bloom-capacity)**: Increase this value when you expect to store many items in most of your bloom filters. This reduces the need for scaling operations which can improve performance.
219
+
-**Higher capacity (bf.bloom-capacity)**: Increase this value when you expect to store a larger number of items in all your new bloom filter objects. It increases the overall capacity for non scaling filters. For scaling filters, it increases the starting/initial capacity which potentially reduces the need for additional scaling operations which can improve performance.
233
220
234
-
-**Lower false positive rate (bf.bloom-fp-rate)**: Decrease this value when accuracy is critical for your application. For example, in fraud detection or security applications where false positives are costly.
221
+
-**Lower false positive rate (bf.bloom-fp-rate)**: Decrease this value when correctness is critical for your application and you want to reduce the number of false positives in all your new bloom filter objects. For example, in fraud detection or security applications where false positives are costly.
235
222
236
-
-**Higher expansion rate (bf.bloom-expansion)**: Increase this value when you want faster growth of bloom filters that need to scale. This reduces the number of scaling operations but uses more memory quicker.
223
+
-**Higher expansion rate (bf.bloom-expansion)**: Increase this value when you want faster growth of bloom filters that will scale across all your new scaling bloom filter objects. This reduces the number of scaling operations but uses more memory quicker.
237
224
238
-
-**Lower tightening ratio (bf.bloom-tightening-ratio)**: Adjust this when you want to maintain a more consistent false positive rate across multiple scaling operations. (Not advisable to change this default)
225
+
-**Lower tightening ratio (bf.bloom-tightening-ratio)**: Lower this when you want to maintain a more consistent false positive rate across multiple scaling operations for all your new bloom filter objects. (Not advisable to change this default, as it is already set to a strict value of 0.5)
239
226
240
-
-**Random seed (bf.bloom-use-random-seed)**: Set to false only when you need deterministic behavior for testing or reproducibility.
227
+
-**Fixed seed (bf.bloom-use-random-seed)**: Set to false only when you want the new bloom filter object creations to use a fixed seed for deterministic occurrence of false positives during item add/exists operations. It can be used for testing or reproducibility.
241
228
242
-
You can modify these default values using the CONFIG SET command:
229
+
You can modify these default values using the CONFIG SET command. However, note that the effect of the configuration change is only applicable to the bloom objects created after the configuration change where the property is not specified through the command already.
243
230
244
231
Example usage of changing all the different properties:
245
232
```bash
246
-
CONFIG SET bf.bloom-fp-rate 0.001
247
-
CONFIG SET bf.bloom-capacity 1000
248
-
CONFIG SET bf.bloom-expansion 4
249
-
CONFIG SET bf.bloom-tightening-ratio 0.6
250
-
CONFIG SET bf.bloom-use-random-seed false
233
+
127.0.0.1:6379>CONFIG SET bf.bloom-fp-rate 0.001
234
+
127.0.0.1:6379>CONFIG SET bf.bloom-capacity 1000
235
+
127.0.0.1:6379>CONFIG SET bf.bloom-expansion 4
236
+
127.0.0.1:6379>CONFIG SET bf.bloom-tightening-ratio 0.6
237
+
127.0.0.1:6379>CONFIG SET bf.bloom-use-random-seed false
251
238
```
252
239
253
240
### Memory usage limit
@@ -256,19 +243,21 @@ The `bf.bloom-memory-usage-limit` configuration (default 128MB) controls the max
256
243
257
244
Example usage of increasing the limit:
258
245
```bash
259
-
CONFIG SET bf.bloom-memory-usage-limit 256mb
246
+
127.0.0.1:6379>CONFIG SET bf.bloom-memory-usage-limit 268435456
260
247
```
261
248
262
-
This setting is particularly important for production environments for several reasons:
249
+
Having a limit on the max memory per bloom object prevents them from growing unbounded, thus maintaining server performance during serialization.
263
250
264
-
-**Resource protection**: Prevents a single bloom filter from consuming excessive memory
265
-
-**Denial of service prevention**: Protects against attacks that might try to create enormous filters
266
-
-**Predictable scaling**: Ensures bloom filters have a known upper bound on resource usage
267
-
268
-
If your use case requires exceptionally large bloom filters, you can increase this limit. However, be aware that very large bloom filters might impact overall system performance and memory availability for other operations.
251
+
If your use case requires bloom filters with capacity beyond what this limit supports, you can increase this configuration value. However, be aware it can impact overall system performance and memory availability for other operations.
269
252
270
253
When a bloom filter reaches this memory limit, any operation that would cause it to exceed the limit will fail with an error message indicating that the memory limit would be exceeded.
The bloom commands which involve adding items or checking the existence of items have a time complexity of O(N * K) where N is the number of hash functions used by the bloom filter and K is the number of elements being inserted. This means that both BF.ADD and BF.EXISTS are both O(N) as they only operate on one item.
@@ -336,6 +325,13 @@ There are two notable validations bloom filters faces.
336
325
337
326
The memory usage limit per bloom filter by default is defined by the `BF.BLOOM-MEMORY-USAGE-LIMIT` module configuration which has a default value of 128 MB. If a command results in a creation / scale out causing the overall memory usage to exceed this limit, the command is rejected. This config is modifiable and can be increased as needed.
338
327
328
+
The `BF.INFO` command's `SIZE` field can be used to find out the current size of a bloom filter.
2. Number of sub filters (in case of scalable bloom filters):
340
336
341
337
When a bloom filter scales out, a new sub filter is added. The limit on the number of sub filters depends on the false positive rate and tightening ratio. Each sub filter has a stricter false positive, and this is controlled by the tightening ratio. If a command attempting a scale out results in the sub filter reaching a false positive of 0, the command is rejected.
@@ -360,10 +356,3 @@ The `BF.INFO` command's `MAXSCALEDCAPACITY` field can be used to find out the ma
0 commit comments