fix(adhoc): map regex operators to REGEXP instead of ILIKE#1794
Open
fix(adhoc): map regex operators to REGEXP instead of ILIKE#1794
Conversation
Grafana's ad-hoc filter operators `=~` ("Matches regex") and `!~`
("Does not match regex") were translated to `ILIKE` and `NOT ILIKE` in
the `additional_table_filters` setting. ILIKE is a LIKE pattern, not a
regex, so the operator names and semantics did not match, and — more
importantly — ILIKE prevents primary-key and skip-index pruning on
indexed columns, producing very slow queries over large tables.
Map `=~` / `!~` to ClickHouse's `REGEXP` / `NOT REGEXP` (RE2-backed,
alias for `match()`). Anchored patterns can still benefit from prefix
pruning, and the semantics now match the operator label shown in the
Grafana UI.
Adds unit tests for the new mapping plus an e2e regression guard that
runs the full `additional_table_filters` SQL shape against a real
ClickHouse fixture, including a pattern (`^(info|warn)$`) that would
match zero rows under ILIKE — so any future regression back to LIKE
semantics fails loudly.
Fixes #1443
The previous SETTINGS additional_table_filters={...} SQL shape was
mangled by Monaco editor's brace-auto-close when typed through
page.keyboard.type(), so the filter never reached ClickHouse and
queries returned unfiltered rows.
Unit tests in src/data/adHocFilter.test.ts cover the exact SETTINGS
shape AdHocFilter.apply() produces; E2E tests now cover what unit
tests cannot — that ClickHouse actually accepts REGEXP / NOT REGEXP
as a filter operator.
Two defences against Monaco swallowing/mangling the query: 1. Press Escape after typing the SQL to dismiss any suggestion popup that may have captured focus mid-stream (common after keywords like `NOT `). 2. Rewrite the NOT REGEXP predicate as `NOT (col REGEXP '...')` so the `NOT` keyword is not immediately adjacent to `REGEXP`, which was triggering keyword autocomplete on the next token.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1443
Summary
=~("Matches regex") and!~("Does not match regex") were being translated toILIKE/NOT ILIKEin theadditional_table_filterssetting. That is semantically wrong (ILIKE is a LIKE pattern, not a regex) and, per the reporter, prevents primary-key / skip-index pruning on indexed columns, producing very slow queries over large tables.=~/!~to ClickHouse'sREGEXP/NOT REGEXP(RE2-backed, alias formatch()). Anchored patterns can still benefit from prefix pruning, and the semantics now match the operator label shown in the UI.src/data/adHocFilter.tschanges runtime behaviour. The Query Builder'sILIKE/NOT ILIKEoptions inFilterEditor.tsxare untouched — they're an explicit user choice and unrelated to the regex operators.