Revert "Add CJK (Chinese, Japanese, Korean) search support" #2321
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
|
@Maklu I had to revert the PR since a test was failing and signaling a legit bug: with SQLite we show the stemmed content in the list of search results, because we use the FTS highlight function. The test was catching that but I didn't notice the suite wasn't running in your PR. You can reproduce locally by just running the suite, and also by seeing how results show the stemmed words. Would you mind opening a new PR making sure that works as expected, please? |
Contributor
|
Got it - the issue was FTS5's highlight()/snippet() returning stemmed content from the FTS table. Fixed in #2337: store original content in main table, stemmed content only in FTS table, use Search::Highlighter instead of FTS5 functions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reverts #2299