Skip to content

Conversation

@sanikolaev
Copy link
Collaborator

When adding a columnar attribute (e.g., MULTI64 ENGINE='columnar') to an RT table that already has STRING attributes (blobs), the blob pool file (.spb) was being truncated, causing "Blob offset out of bounds" errors during index checks and potential crashes during queries.

Root cause:

  • CSphIndex_VLN::AddRemoveAttribute was creating a new .spb file when adding a columnar attribute, but Alter_AddRemoveRowwiseAttr (which writes blob data) was not called for columnar attributes
  • This resulted in the new file containing only the header and update space, overwriting the existing blob data

Fix:

  • Preserve existing blob data by reading it from the old blob pool file before Reset() and writing it to the new temporary file
  • Also clean up the temporary .tmp.spa file that was left behind when adding columnar attributes to tables with non-columnar attributes

Test:

  • Added test case in test_479/test.xml to verify blob data preservation after ALTER TABLE ADD COLUMN with columnar attribute

Related issue #3661

When adding a columnar attribute (e.g., MULTI64 ENGINE='columnar') to an
RT table that already has STRING attributes (blobs), the blob pool file
(.spb) was being truncated, causing "Blob offset out of bounds" errors
during index checks and potential crashes during queries.

Root cause:
- CSphIndex_VLN::AddRemoveAttribute was creating a new .spb file when
  adding a columnar attribute, but Alter_AddRemoveRowwiseAttr (which
  writes blob data) was not called for columnar attributes
- This resulted in the new file containing only the header and update
  space, overwriting the existing blob data

Fix:
- Preserve existing blob data by reading it from the old blob pool file
  before Reset() and writing it to the new temporary file
- Also clean up the temporary .tmp.spa file that was left behind when
  adding columnar attributes to tables with non-columnar attributes

Test:
- Added test case in test_479/test.xml to verify blob data preservation
  after ALTER TABLE ADD COLUMN with columnar attribute

Related issue #3661
@github-actions
Copy link
Contributor

clt

❌ CLT tests in test/clt-tests/buddy-plugins/
✅ OK: 8
❌ Failed: 1
⏳ Duration: 88s
👉 Check Action Results for commit b09d594

Failed tests:

🔧 Edit failed tests in UI:

test/clt-tests/buddy-plugins/test-inconsistent-comunication-with-buddy.rec
––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
apt-get install jq -y > /dev/null; echo $?
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?mode=raw" -d "query=SHOW TABLES"; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?mode=raw&query=SHOW%20TABLES"; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?raw_response=true" -d "query=SHOW TABLES"; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?raw_response=true&query=SHOW%20TABLES"; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?mode=raw" -d "query=SHOW VARIABLES"; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?mode=raw&query=SHOW%20VARIABLES"; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?raw_response=true" -d "query=SHOW VARIABLES"; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?raw_response=true&query=SHOW%20VARIABLES"; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?mode=raw" -d "query=SHOW META"; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?mode=raw&query=SHOW%20META"; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?raw_response=true" -d "query=SHOW META"; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?raw_response=true&query=SHOW%20META"; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?mode=raw" -d "query=SHOW QUERIES" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?mode=raw&query=SHOW%20QUERIES" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?raw_response=true" -d "query=SHOW QUERIES" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
[
  {
    "total": %{NUMBER},
    "error": "",
    "warning": "",
    "columns": [
      {
        "id": {
          "type": "long long"
        }
      },
      {
        "query": {
          "type": "string"
        }
      },
      {
        "time": {
          "type": "string"
        }
      },
      {
        "protocol": {
          "type": "string"
        }
      },
      {
        "host": {
          "type": "string"
        }
      }
    ],
    "data": [
      {
        "id": %{NUMBER},
        "query": "#!/[a-zA-Z\s\_]+/!#",
        "time": "%{NUMBER}#!/(s|ms|us)/!#",
        "protocol": "http",
        "host": "127.0.0.1:%{NUMBER}"
      },
      {
        "id": %{NUMBER},
        "query": "#!/[a-zA-Z\s\_]+/!#",
        "time": "%{NUMBER}#!/(s|ms|us)/!# ago",
        "protocol": "http",
        "host": "127.0.0.1:%{NUMBER}"
      },
      {
        "id": %{NUMBER},
        "query": "#!/[a-zA-Z\s\_]+/!#",
        "time": "%{NUMBER}#!/(s|ms|us)/!# ago",
        "protocol": "http",
        "host": "127.0.0.1:%{NUMBER}"
-       }
+       },
-     ]
+       {
-   }
+         "id": 27,
- ]
+         "query": "",
+         "time": "2926w ago",
+         "protocol": "http",
+         "host": "127.0.0.1:45764"
+       }
+     ]
+   }
+ ]
––– input –––
curl -s "http://localhost:9308/sql?raw_response=true&query=SHOW%20QUERIES" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?mode=raw" -d "query=SHOW VERSION" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?mode=raw&query=SHOW%20VERSION" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK
––– input –––
curl -s -X POST "http://localhost:9308/sql?raw_response=true" -d "query=SHOW VERSION" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/sql?raw_response=true&query=SHOW%20VERSION" | jq '.[0].data |= sort_by(.id)'; echo
––– output –––
OK

@sanikolaev sanikolaev merged commit af36e14 into master Jan 25, 2026
96 of 97 checks passed
@sanikolaev sanikolaev deleted the issue-3661 branch January 25, 2026 16:40
sanikolaev added a commit that referenced this pull request Jan 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants