🆕 Update buddy version from 3.46.3 to 3.47.0 by githubmanticore · Pull Request #4606 · manticoresoftware/manticoresearch

githubmanticore · 2026-06-01T14:58:55Z

Update buddy version from 3.46.3 to 3.47.0 which includes:

2f784a7 feature(sharding): sharding rollback, upgrades, tests, various fixes (#665)

github-actions · 2026-06-01T15:18:46Z

Linux debug test results

1 210 tests 1 159 ✅ 1h 54m 44s ⏱️
1 suites 51 💤
1 files 0 ❌

Results for commit 13ae0e6.

♻️ This comment has been updated with latest results.

github-actions · 2026-06-03T12:35:07Z

clt

❌ CLT tests in test/clt-tests/sharding/functional/functional-sharding-and-cluster-negative test/clt-tests/sharding/functional/functional-sharding-positive test/clt-tests/sharding/functional/test-dist-table-shards-5000 test/clt-tests/sharding/replication/create-table-1-node-10-shards test/clt-tests/sharding/replication/create-table-2-nodes-4-shards test/clt-tests/sharding/replication/create-table-3-nodes-6-shards test/clt-tests/sharding/replication/create-table-5-nodes-60-shards test/clt-tests/sharding/replication/test-data-manipulation test/clt-tests/sharding/replication/test-distributed-inserts-with-replication test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication test/clt-tests/sharding/syntax/sharding-syntax-negative test/clt-tests/sharding/syntax/sharding-syntax-positive test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure
✅ OK: 12
❌ Failed: 1
⏳ Duration: 284s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

Edit test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure.rec

test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:test_products (id bigint, name string, price float) shards='2' rf='2'" & sleep 3; export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; sleep 1; echo "Node 2 killed"
––– output –––
- Node 2 killed
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ Node 2 killed
––– input –––
sleep 15; echo "Wait completed"
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 10; mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as shard_count FROM system.sharding_table WHERE table = 'test_products'"
––– output –––
- +-------------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
- | shard_count |
- +-------------+
- |           2 |
- +-------------+

github-actions · 2026-06-03T12:44:49Z

clt

❌ CLT tests in test/clt-tests/sharding/rollback/data-consistency-multiple-failures test/clt-tests/sharding/rollback/master-node-failure test/clt-tests/sharding/rollback/master-takeover-while-rollback test/clt-tests/sharding/rollback/mixed-rf-tables-node-failure test/clt-tests/sharding/rollback/new-node-addition-rf-replication test/clt-tests/sharding/rollback/node-failure-mid-creation-recovery test/clt-tests/sharding/rollback/node-killed-rf-maintained test/clt-tests/sharding/rollback/node-rejoin-stale-cluster-cleanup test/clt-tests/sharding/rollback/node-removal-rf-maintained test/clt-tests/sharding/rollback/rebalance-master-killed-mid-operation test/clt-tests/sharding/rollback/rf1-no-rebalance-on-failure test/clt-tests/sharding/rollback/sequential-node-failures-high-rf test/clt-tests/sharding/regression/test-columnar-engine-shard
✅ OK: 1
❌ Failed: 12
⏳ Duration: 860s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

test/clt-tests/sharding/rollback/sequential-node-failures-high-rf.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='3' rf='3'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'Widget A'), (2, 'Widget B'), (3, 'Widget C')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:4312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 20 grep -qm1 'Skipping rebalance for table' <(tail -n 100 -f /var/log/manticore-{3,4}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P3306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-3/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-3/searchd.log); echo $?
––– output –––
- 0
+ tail: /var/log/manticore-3/searchd.log: file truncated
+ 124
––– input –––
for i in 1 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:4312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK

test/clt-tests/sharding/rollback/master-takeover-while-rollback.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:1312 127.0.0.1:2312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312

test/clt-tests/sharding/rollback/node-failure-mid-creation-recovery.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:orders (id bigint, total float) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO orders (id, total) VALUES (1, 100), (2, 200)"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
for port in 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 2306: synced
- Port 3306: synced
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced

test/clt-tests/sharding/rollback/data-consistency-multiple-failures.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, account string, amount float, ts int) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, account, amount, ts) VALUES (1, 'ACC001', 100.50, 1000), (2, 'ACC002', 200.75, 2000), (3, 'ACC003', 150.25, 3000), (4, 'ACC001', 300.00, 4000), (5, 'ACC002', 250.50, 5000)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"
––– output –––
- 5	1002.000000
+ +------+-------------+
+ |    5 | 1002.000000 |
+ +------+-------------+
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "INSERT INTO ${TABLE_NAME} (id, account, amount, ts) VALUES (6, 'ACC003', 175.00, 6000), (7, 'ACC001', 225.75, 7000)"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{1,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 1 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"
––– output –––
- 7	1402.75
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
––– input –––
for port in 1306 2306 3306; do echo "Node $port:"; mysql -h0 -P$port -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"; done
––– output –––
Node 1306:
- 7	1402.75
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
Node 2306:
- 7	1402.75
+ +------+------------+
- Node 3306:
+ |    3 | 476.250000 |
- 7	1402.75
+ +------+------------+
+ Node 3306:
+ +------+-------------+
+ |    7 | 1402.750000 |
+ +------+-------------+
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS ${TABLE_NAME}"; echo $?
––– output –––
OK

test/clt-tests/sharding/rollback/new-node-addition-rf-replication.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:existing_table (id bigint, data string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -sN -e "SELECT COUNT(*) FROM system.sharding_table WHERE table = 'existing_table'"; done
––– output –––
- 2
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
- 2
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P3306 -e "JOIN CLUSTER c at '127.0.0.1:1312'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: cluster 'c', no nodes available(127.0.0.1:1312), error: '127.0.0.1:1312': retries limit exceeded
+ 1
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done
––– output –––
- Port 1306: Node synced
- Port 2306: Node synced
- Port 3306: Node synced
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM system.sharding_state"
––– output –––
- 3
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_state' in search request
––– input –––
sleep 15; echo "Cluster stabilized"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS existing_table"
––– output –––
OK

test/clt-tests/sharding/rollback/node-rejoin-stale-cluster-cleanup.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'alpha'), (2, 'beta')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-2/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
- |    2 |
- +------+

test/clt-tests/sharding/rollback/rf1-no-rebalance-on-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='1'"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: local
- system.t_s1: 127.0.0.1:2312
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'Widget A'), (2, 'Widget B'), (3, 'Widget C'), (4, 'Widget D')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as total FROM ${TABLE_NAME}\\G" | grep total
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'Skipping rebalance for table' <(tail -n 1000 -f /var/log/manticore-1/searchd.log); echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: local
- system.t_s1: 127.0.0.1:2312
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as cnt FROM ${TABLE_NAME}\\G" | grep -E 'cnt:' | awk '{print $2}'
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; for port in 1306 2306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as cnt FROM ${TABLE_NAME}\\G" | grep -E 'cnt:' | awk '{print $2}'
––– output –––
OK

test/clt-tests/sharding/rollback/node-removal-rf-maintained.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:items (id bigint, name string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO items (id, name) VALUES (1, 'Item1'), (2, 'Item2'), (3, 'Item3')"; echo $?
––– output –––
OK
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced
––– input –––
mysql -h0 -P1306 -e "show tables from system\\G" | grep -E 'items_s[0-9]' | wc -l
––– output –––
- 2
+ 0

test/clt-tests/sharding/rollback/node-killed-rf-maintained.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:data (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (1, 'test1'), (2, 'test2')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
sleep 10; mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (3, 'test3')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 20
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM data"
––– output –––
- 3
+ +------+
+ |    3 |
+ +------+
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM data"
––– output –––
- 3
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'data' in search request
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS data"; echo $?
––– output –––
OK

test/clt-tests/sharding/rollback/master-node-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:data (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
sleep 5; mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (1, 'test_value')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
for port in 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 2306: synced
- Port 3306: synced
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced

test/clt-tests/sharding/rollback/rebalance-master-killed-mid-operation.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='3' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'one'), (2, 'two'), (3, 'three')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s2: 127.0.0.1:3312 127.0.0.1:4312
––– input –––
export INSTANCE=4; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 4 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 20 grep -qm1 'Rebalancing due to inactive nodes' <(tail -n 100 -f /var/log/manticore-1/searchd.log); echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Master killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 30 grep -qm1 'becoming master' <(tail -n 100 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
sleep 5
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-{2,3}/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:3312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:3312 127.0.0.1:2312
- system.t_s2: 127.0.0.1:3312 127.0.0.1:2312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK

test/clt-tests/sharding/rollback/mixed-rf-tables-node-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:norep (id bigint, value string) shards='2' rf='1'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Waiting timeout exceeded.
+ 1
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:rep (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO norep (id, value) VALUES (1, 'a'), (2, 'b')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO rep (id, value) VALUES (1, 'x'), (2, 'y')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 1306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Skipping rebalance for table norep' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 2306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM rep"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-2/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 1306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:1312 127.0.0.1:2312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM norep"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM rep"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'rep' in search request
- |    2 |
- +------+

github-actions · 2026-06-03T14:37:19Z

clt

❌ CLT tests in test/clt-tests/mcl/auto-embeddings-endpoints test/clt-tests/mcl/auto-embeddings-error-handling test/clt-tests/mcl/auto-embeddings-from-vector-check test/clt-tests/mcl/auto-embeddings-hnsw-configs test/clt-tests/mcl/auto-embeddings-jina-remote test/clt-tests/mcl/auto-embeddings-json-api test/clt-tests/mcl/auto-embeddings-knn-select-scenarios test/clt-tests/mcl/auto-embeddings-model-provider-validation test/clt-tests/mcl/auto-embeddings-openai-remote test/clt-tests/mcl/auto-embeddings-parameter-combinations test/clt-tests/mcl/auto-embeddings-qwen test/clt-tests/mcl/auto-embeddings-show-create-table test/clt-tests/mcl/auto-embeddings-syntax-check
✅ OK: 4
❌ Failed: 9
⏳ Duration: 124s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

test/clt-tests/mcl/auto-embeddings-show-create-table.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
php /manticore/test/clt-tests/mcl/mock-embeddings-server.php --port 8080 --provider openai > /tmp/mock-server-8080.log 2>&1 &
MOCK_PID_8080=$!
echo "Mock server started on port 8080 (PID: $MOCK_PID_8080)"
sleep 1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_api_url (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_api_url" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_no_api_url (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2' FROM='title')" 2>&1; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_show_no_api_url': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_no_api_url" 2>&1
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
-        Table: test_show_no_api_url
- Create Table: CREATE TABLE test_show_no_api_url (
- id bigint,
- title text,
- embedding float_vector knn_type='hnsw' hnsw_similarity='L2' model_name='sentence-transformers/all-MiniLM-L6-v2' from='title' api_timeout='10'
- )
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_api_timeout (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_api_timeout" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_no_api_timeout (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_no_api_timeout" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_api_timeout_zero (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='0')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_api_timeout_zero" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_from (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_from" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_api_key_security (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='secret-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_api_key_security" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_show_all_params (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_show_all_params" 2>&1
––– output –––
OK

test/clt-tests/mcl/auto-embeddings-hnsw-configs.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_l2 (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_l2': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_cosine (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='cosine'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_cosine': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_ip (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='ip'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_ip': prealloc: Failed to download model configuration
+ 1
––– input –––
for table in test_l2 test_cosine test_ip; do
    mysql -h0 -P9306 -e "INSERT INTO $table (id, content) VALUES
        (1, 'machine learning'),
        (2, 'deep learning'),
        (3, 'cooking recipes')" 2>/dev/null
done
echo "Data inserted into all tables"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_l2; OPTIMIZE TABLE test_l2 OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_cosine; OPTIMIZE TABLE test_cosine OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_ip; OPTIMIZE TABLE test_ip OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SELECT id FROM test_l2 WHERE KNN(vec, 2, 'artificial intelligence')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_l2: requested KNN search attribute 'vec' not found
- id: 1
- *************************** 2. row ***************************
- id: 2
- *************************** 3. row ***************************
- id: 3
––– input –––
mysql -h0 -P9306 -E -e "SELECT id FROM test_cosine WHERE KNN(vec, 2, 'artificial intelligence')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_cosine: requested KNN search attribute 'vec' not found
- id: 1
- *************************** 2. row ***************************
- id: 2
- *************************** 3. row ***************************
- id: 3
––– input –––
mysql -h0 -P9306 -E -e "SELECT id FROM test_ip WHERE KNN(vec, 2, 'artificial intelligence')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_ip: requested KNN search attribute 'vec' not found
- id: 1
- *************************** 2. row ***************************
- id: 2
- *************************** 3. row ***************************
- id: 3
––– input –––
echo "L2 (Euclidean) distances:"
mysql -h0 -P9306 -E -e "SELECT id, content, KNN_DIST() as distance FROM test_l2 WHERE KNN(vec, 3, 'neural networks') ORDER BY distance"
––– output –––
L2 (Euclidean) distances:
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_l2: requested KNN search attribute 'vec' not found
-       id: 2
-  content: deep learning
- distance: 0.491#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-       id: 1
-  content: machine learning
- distance: 0.617#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-       id: 3
-  content: cooking recipes
- distance: 1.423#!/[0-9]{0,5}/!#
––– input –––
echo "Cosine similarity distances (smaller = more similar):"
mysql -h0 -P9306 -E -e "SELECT id, content, KNN_DIST() as distance FROM test_cosine WHERE KNN(vec, 3, 'neural networks') ORDER BY distance"
––– output –––
Cosine similarity distances (smaller = more similar):
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_cosine: requested KNN search attribute 'vec' not found
-       id: 2
-  content: deep learning
- distance: 0.245#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-       id: 1
-  content: machine learning
- distance: 0.308#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-       id: 3
-  content: cooking recipes
- distance: 0.711#!/[0-9]{0,5}/!#
––– input –––
echo "Inner product distances (smaller = more similar):"
mysql -h0 -P9306 -E -e "SELECT id, content, KNN_DIST() as distance FROM test_ip WHERE KNN(vec, 3, 'neural networks') ORDER BY distance"
––– output –––
Inner product distances (smaller = more similar):
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_ip: requested KNN search attribute 'vec' not found
-       id: 2
-  content: deep learning
- distance: 0.245#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-       id: 1
-  content: machine learning
- distance: 0.308#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-       id: 3
-  content: cooking recipes
- distance: 0.711#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_hnsw_m4 (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    HNSW_M='4'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_hnsw_m4': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_hnsw_m32 (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    HNSW_M='32'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_hnsw_m32': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_hnsw_ef (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    HNSW_M='16'
    HNSW_EF_CONSTRUCTION='500'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_hnsw_ef': prealloc: Failed to download model configuration
+ 1
––– input –––
for table in test_hnsw_m4 test_hnsw_m32 test_hnsw_ef; do
    mysql -h0 -P9306 -e "INSERT INTO $table (id, content) VALUES (1, 'test document')" 2>/dev/null
done
echo "HNSW configurations test completed"
––– output –––
OK
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_hnsw_m4"
––– output –––
OK
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_hnsw_m32"
––– output –––
OK
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_hnsw_ef"
––– output –––
OK
––– input –––
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO test_hnsw_m4 (content) VALUES ('document number $i with various content')" 2>/dev/null
    mysql -h0 -P9306 -e "INSERT INTO test_hnsw_m32 (content) VALUES ('document number $i with various content')" 2>/dev/null
done
echo "Additional documents inserted for performance comparison"
––– output –––
OK
––– input –––
echo "Search with M=4 (faster, less connections):"
mysql -h0 -P9306 -E -e "SELECT id FROM test_hnsw_m4 WHERE KNN(vec, 3, 'document content') LIMIT 3"
––– output –––
Search with M=4 (faster, less connections):
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_hnsw_m4: requested KNN search attribute 'vec' not found
- id: %{NUMBER}
- *************************** 2. row ***************************
- id: %{NUMBER}
- *************************** 3. row ***************************
- id: %{NUMBER}
––– input –––
echo "Search with M=32 (slower, more connections, potentially more accurate):"
mysql -h0 -P9306 -E -e "SELECT id FROM test_hnsw_m32 WHERE KNN(vec, 3, 'document content') LIMIT 3"
––– output –––
Search with M=32 (slower, more connections, potentially more accurate):
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_hnsw_m32: requested KNN search attribute 'vec' not found
- id: %{NUMBER}
- *************************** 2. row ***************************
- id: %{NUMBER}
- *************************** 3. row ***************************
- id: %{NUMBER}
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_invalid_metric (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='invalid_metric'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'" 2>&1
––– output –––
OK

test/clt-tests/mcl/auto-embeddings-from-vector-check.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_title1 (title1 TEXT, title2 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'title1') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_title1': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_title2 (title1 TEXT, title2 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'title2') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_title2': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_title1 (id, title1, title2) VALUES(1, 'machine learning artificial intelligence', 'banana fruit sweet yellow')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_title2 (id, title1, title2) VALUES(1, 'machine learning artificial intelligence', 'banana fruit sweet yellow')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as count_table1 FROM test_from_title1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as count_table2 FROM test_from_title2"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title1 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print "title1_vector_md5: " $1}'
––– output –––
- title1_vector_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title1: parse error: unknown column: embedding_vector
+ title1_vector_md5: d41d8cd98f00b204e9800998ecf8427e
––– input –––
mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title2 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print "title2_vector_md5: " $1}'
––– output –––
- title2_vector_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title2: parse error: unknown column: embedding_vector
+ title2_vector_md5: d41d8cd98f00b204e9800998ecf8427e
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_title1 (id, title1, title2) VALUES(2, 'machine learning artificial intelligence', 'banana fruit sweet yellow')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title1 WHERE id=2" | grep -v embedding_vector | md5sum | awk '{print "consistent_md5: " $1}'
––– output –––
- consistent_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title1: parse error: unknown column: embedding_vector
+ consistent_md5: d41d8cd98f00b204e9800998ecf8427e
––– input –––
MD5_TITLE1=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title1 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); MD5_TITLE2=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title2 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); echo "title1_md5: $MD5_TITLE1"; echo "title2_md5: $MD5_TITLE2"; if [ "$MD5_TITLE1" != "$MD5_TITLE2" ]; then echo "SUCCESS: FROM clause produces different vectors"; else echo "FAIL: FROM clause produces same vectors"; fi
––– output –––
- title1_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title1: parse error: unknown column: embedding_vector
- title2_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title2: parse error: unknown column: embedding_vector
- SUCCESS: FROM clause produces different vectors
+ title1_md5: d41d8cd98f00b204e9800998ecf8427e
+ title2_md5: d41d8cd98f00b204e9800998ecf8427e
+ FAIL: FROM clause produces same vectors
––– input –––
MD5_SAME1=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title1 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); MD5_SAME2=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title1 WHERE id=2" | grep -v embedding_vector | md5sum | awk '{print $1}'); echo "same_input_md5_1: $MD5_SAME1"; echo "same_input_md5_2: $MD5_SAME2"; if [ "$MD5_SAME1" = "$MD5_SAME2" ]; then echo "SUCCESS: Same input produces same vector"; else echo "FAIL: Same input produces different vectors"; fi
––– output –––
- same_input_md5_1: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title1: parse error: unknown column: embedding_vector
- same_input_md5_2: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title1: parse error: unknown column: embedding_vector
- SUCCESS: Same input produces same vector
+ same_input_md5_1: d41d8cd98f00b204e9800998ecf8427e
+ same_input_md5_2: d41d8cd98f00b204e9800998ecf8427e
+ SUCCESS: Same input produces same vector
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_invalid (title1 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'invalid-model-name' FROM = 'title1') " 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_invalid_field (title1 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'nonexistent_field') " 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_no_from_specified (title1 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2') " 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_set_to_empty (title1 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = '') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_set_to_empty': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_multi_from (title1 TEXT, title2 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'title1, title2') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_multi_from': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_multi_from (id, title1, title2) VALUES(1, 'deep learning neural networks', 'computer vision algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_multi_from WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print "multi_field_md5: " $1}'
––– output –––
- multi_field_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_multi_from: parse error: unknown column: embedding_vector
+ multi_field_md5: d41d8cd98f00b204e9800998ecf8427e
––– input –––
MD5_MULTI=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_multi_from WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); MD5_SINGLE=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_title1 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); echo "multi_field_md5: $MD5_MULTI"; echo "single_field_md5: $MD5_SINGLE"; if [ "$MD5_MULTI" != "$MD5_SINGLE" ]; then echo "SUCCESS: Multi-field FROM produces different vector"; else echo "INFO: Multi-field vs single-field comparison"; fi
––– output –––
- multi_field_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_multi_from: parse error: unknown column: embedding_vector
- single_field_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_title1: parse error: unknown column: embedding_vector
- #!/SUCCESS: Multi-field FROM produces different vector|INFO: Multi-field vs single-field comparison/!#
+ multi_field_md5: d41d8cd98f00b204e9800998ecf8427e
+ single_field_md5: d41d8cd98f00b204e9800998ecf8427e
+ INFO: Multi-field vs single-field comparison
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_no_from (title1 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2') "
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_no_from (id, title1) VALUES(1, 'test title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_empty_t1 (title1 TEXT, title2 TEXT, title3 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'title1') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_empty_t1': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_empty_t2 (title1 TEXT, title2 TEXT, title3 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'title2') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_empty_t2': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_empty_t3 (title1 TEXT, title2 TEXT, title3 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = 'title3') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_empty_t3': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_empty_all (title1 TEXT, title2 TEXT, title3 TEXT, embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2' FROM = '') "; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_from_empty_all': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_empty_t1 (id, title1, title2, title3) VALUES(1, 'neural ranking transformers', 'fresh orange citrus', 'race car tire pressure')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_empty_t2 (id, title1, title2, title3) VALUES(1, 'neural ranking transformers', 'fresh orange citrus', 'race car tire pressure')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_empty_t3 (id, title1, title2, title3) VALUES(1, 'neural ranking transformers', 'fresh orange citrus', 'race car tire pressure')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_empty_all (id, title1, title2, title3) VALUES(1, 'neural ranking transformers', 'fresh orange citrus', 'race car tire pressure')"; echo $?
––– output –––
OK
––– input –––
MD5_T1=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_empty_t1 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); MD5_T2=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_empty_t2 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); MD5_T3=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_empty_t3 WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); MD5_ALL=$(mysql -h0 -P9306 -e "SELECT embedding_vector FROM test_from_empty_all WHERE id=1" | grep -v embedding_vector | md5sum | awk '{print $1}'); echo "md5_t1: $MD5_T1"; echo "md5_t2: $MD5_T2"; echo "md5_t3: $MD5_T3"; echo "md5_all: $MD5_ALL"; UCOUNT=$(printf "%s\n%s\n%s\n%s\n" "$MD5_T1" "$MD5_T2" "$MD5_T3" "$MD5_ALL" | sort -u | wc -l); if [ "$UCOUNT" -eq 4 ]; then echo "SUCCESS: all FROM variants produce different vectors"; else echo "FAIL: expected 4 different vectors, got $UCOUNT"; fi
––– output –––
- md5_t1: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_empty_t1: parse error: unknown column: embedding_vector
- md5_t2: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_empty_t2: parse error: unknown column: embedding_vector
- md5_t3: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_empty_t3: parse error: unknown column: embedding_vector
- md5_all: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_from_empty_all: parse error: unknown column: embedding_vector
- SUCCESS: all FROM variants produce different vectors
+ md5_t1: d41d8cd98f00b204e9800998ecf8427e
+ md5_t2: d41d8cd98f00b204e9800998ecf8427e
+ md5_t3: d41d8cd98f00b204e9800998ecf8427e
+ md5_all: d41d8cd98f00b204e9800998ecf8427e
+ FAIL: expected 4 different vectors, got 1

test/clt-tests/mcl/auto-embeddings-endpoints.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
apt-get install jq -y > /dev/null; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE emb_test (
    id BIGINT,
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'emb_test': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO emb_test (id, title, content) VALUES
    (1, 'machine learning', 'neural networks and deep learning'),
    (2, 'computer vision', 'image recognition and processing'),
    (3, 'natural language', 'text analysis and understanding')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK emb_test; OPTIMIZE TABLE emb_test OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM emb_test WHERE KNN(vec, 2, 'artificial intelligence')"
––– output –––
- +----------+
+ ERROR 1064 (42000) at line 1: table emb_test: requested KNN search attribute 'vec' not found
- | count(*) |
- +----------+
- |        3 |
- +----------+
––– input –––
curl -s "http://localhost:9308/cli?select%20id,%20title%20from%20emb_test%20where%20knn(vec,%202,%20'artificial%20intelligence')" | grep -v 'rows in set'
––– output –––
- +----+------------------+
+ Query OK, 0 rows affected (0.004 sec)
- | id | title            |
- +----+------------------+
- | 1  | machine learning |
- | 2  | computer vision  |
- | 3  | natural language |
- +----+------------------+
––– input –––
curl -s "http://localhost:9308/cli_json?select%20id,%20title,%20@knn_dist%20from%20emb_test%20where%20knn(vec,%201,%20'learning')" | jq -r '.[0].data[0] | "ID: \(.id)\nTitle: \(.title)\nDistance: \(.["@knn_dist"] | tostring)"'
––– output –––
- ID: 1
+ jq: error (at <stdin>:0): Cannot index object with number
- Title: machine learning
- Distance: #!/1\.082[0-9]*/!#
––– input –––
curl -s -X POST "http://localhost:9308/sql?mode=raw" -d "select count(*) from emb_test where knn(vec, 2, 'neural networks')" | jq -r '.[0].data[0]."count(*)"'
––– output –––
- 3
+ jq: error (at <stdin>:0): Cannot index object with number
––– input –––
curl -s -X POST http://localhost:9308/insert -d '{"index":"emb_test","id":10,"doc":{"title":"quantum computing","content":"quantum algorithms"}}' | jq -r '.created'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:9308/search -d '{"index":"emb_test","knn":{"field":"vec","query":"quantum","k":1}}' | jq -r '.hits.hits[0]._source.title'
––– output –––
- quantum computing
+ null
––– input –––
curl -s -X POST http://localhost:9308/search -d '{"index":"emb_test","knn":{"field":"vec","query_text":"quantum","k":1}}'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE chunk_test (
    id BIGINT,
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'chunk_test': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO chunk_test (id, title) VALUES
    (1, 'machine learning'),
    (2, 'deep learning'),
    (3, 'reinforcement learning')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM chunk_test WHERE KNN(vec, 1, 'learning')"
––– output –––
- +----------+
+ ERROR 1064 (42000) at line 1: table chunk_test: requested KNN search attribute 'vec' not found
- | count(*) |
- +----------+
- |        3 |
- +----------+
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK chunk_test; OPTIMIZE TABLE chunk_test OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM chunk_test WHERE KNN(vec, 1, 'learning')"
––– output –––
- +----------+
+ ERROR 1064 (42000) at line 1: table chunk_test: requested KNN search attribute 'vec' not found
- | count(*) |
- +----------+
- |        3 |
- +----------+

test/clt-tests/mcl/auto-embeddings-json-api.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
apt-get install jq -y > /dev/null; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_json_columnar (
    title TEXT,
    content TEXT,
    embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_json_columnar': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE test_json_columnar" | grep -o "model_name='sentence-transformers/all-MiniLM-L6-v2'"
––– output –––
- model_name='sentence-transformers/all-MiniLM-L6-v2'
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
––– input –––
curl -s -X POST http://localhost:9308/insert -d '{"index":"test_json_columnar","id":1,"doc":{"title":"machine learning","content":"neural networks"}}' | jq -r 'if ._id then ._id else "inserted" end'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_json_columnar WHERE KNN(embedding, 1, 'machine learning neural networks')"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: table test_json_columnar: requested KNN search attribute 'embedding' not found
- | id   |
- +------+
- |    1 |
- +------+
––– input –––
curl -s -X POST http://localhost:9308/bulk -H "Content-Type: application/x-ndjson" -d '
{"insert":{"index":"test_json_columnar","id":2,"doc":{"title":"computer vision","content":"image recognition"}}}
{"insert":{"index":"test_json_columnar","id":3,"doc":{"title":"NLP","content":"text processing"}}}
' | jq '{created: .items[0].bulk.created}'
––– output –––
OK
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_json_columnar WHERE id IN (2,3)"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:9308/replace -d '{"index":"test_json_columnar","id":1,"doc":{"title":"updated ML","content":"updated networks"}}' | jq -r '.result'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT title FROM test_json_columnar WHERE id=1 AND KNN(embedding, 1, 'updated ML networks')"
––– output –––
- +------------+
+ ERROR 1064 (42000) at line 1: table test_json_columnar: requested KNN search attribute 'embedding' not found
- | title      |
- +------------+
- | updated ML |
- +------------+
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_json_columnar (id, title, content) VALUES (100, 'test', 'data')";
curl -s -X POST http://localhost:9308/insert -d '{"index":"test_json_columnar","id":101,"doc":{"title":"test","content":"data"}}' > /dev/null
––– output –––
OK
––– input –––
mysql -h0 -P9306 --batch --skip-column-names -e "SELECT embedding FROM test_json_columnar WHERE id=100" > /tmp/v1.txt
mysql -h0 -P9306 --batch --skip-column-names -e "SELECT embedding FROM test_json_columnar WHERE id=101" > /tmp/v2.txt
diff -q /tmp/v1.txt /tmp/v2.txt > /dev/null && echo "Vectors identical" || echo "Vectors differ"
––– output –––
- Vectors identical
+ ERROR 1064 (42000) at line 1: table test_json_columnar: parse error: unknown column: embedding
+ ERROR 1064 (42000) at line 1: table test_json_columnar: parse error: unknown column: embedding
+ Vectors identical
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_json_columnar"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_json_columnar; OPTIMIZE TABLE test_json_columnar OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
VECTOR=$(python3 -c "print(','.join(['0.01']*384))")
curl -s -X POST http://localhost:9308/search -d "{\"index\":\"test_json_columnar\",\"knn\":{\"field\":\"embedding\",\"query_vector\":[$VECTOR],\"k\":2}}" | jq -r '.hits.total // "0"'
––– output –––
- 5
+ 0
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE no_auto_embed (title TEXT, vec FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='384' HNSW_SIMILARITY='l2') engine='columnar'"
––– output –––
OK
––– input –––
VECTOR=$(python3 -c "print(','.join(['0.5']*384))")
curl -s -X POST http://localhost:9308/insert -d "{\"index\":\"no_auto_embed\",\"id\":1,\"doc\":{\"title\":\"test\",\"vec\":[$VECTOR]}}" | jq -r 'if ._id then ._id else "inserted" end'
––– output –––
OK
––– input –––
QUERY_VEC=$(python3 -c "print(','.join(['0.5']*384))")
curl -s -X POST http://localhost:9308/search -d "{\"index\":\"no_auto_embed\",\"knn\":{\"field\":\"vec\",\"query_vector\":[$QUERY_VEC],\"k\":1}}" | jq -r '.hits.total // "0"'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_json_rowwise (
    title TEXT,
    content TEXT,
    embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
) engine='rowwise'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_json_rowwise': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE test_json_rowwise" | grep -o "model_name='sentence-transformers/all-MiniLM-L6-v2'"
––– output –––
- model_name='sentence-transformers/all-MiniLM-L6-v2'
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
––– input –––
curl -s -X POST http://localhost:9308/insert -d '{"index":"test_json_rowwise","id":1,"doc":{"title":"machine learning","content":"neural networks"}}' | jq -r 'if ._id then ._id else "inserted" end'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_json_rowwise WHERE KNN(embedding, 1, 'machine learning neural networks')"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: table test_json_rowwise: requested KNN search attribute 'embedding' not found
- | id   |
- +------+
- |    1 |
- +------+
––– input –––
curl -s -X POST http://localhost:9308/bulk -H "Content-Type: application/x-ndjson" -d '
{"insert":{"index":"test_json_rowwise","id":2,"doc":{"title":"computer vision","content":"image recognition"}}}
{"insert":{"index":"test_json_rowwise","id":3,"doc":{"title":"NLP","content":"text processing"}}}
' | jq '{created: .items[0].bulk.created}'
––– output –––
OK
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_json_rowwise WHERE id IN (2,3)"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:9308/replace -d '{"index":"test_json_rowwise","id":1,"doc":{"title":"updated ML","content":"updated networks"}}' | jq -r '.result'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT title FROM test_json_rowwise WHERE id=1 AND KNN(embedding, 1, 'updated ML networks')"
––– output –––
- +------------+
+ ERROR 1064 (42000) at line 1: table test_json_rowwise: requested KNN search attribute 'embedding' not found
- | title      |
- +------------+
- | updated ML |
- +------------+
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_json_rowwise (id, title, content) VALUES (100, 'test', 'data')";
curl -s -X POST http://localhost:9308/insert -d '{"index":"test_json_rowwise","id":101,"doc":{"title":"test","content":"data"}}' > /dev/null
––– output –––
OK
––– input –––
mysql -h0 -P9306 --batch --skip-column-names -e "SELECT embedding FROM test_json_rowwise WHERE id=100" > /tmp/v1.txt
mysql -h0 -P9306 --batch --skip-column-names -e "SELECT embedding FROM test_json_rowwise WHERE id=101" > /tmp/v2.txt
diff -q /tmp/v1.txt /tmp/v2.txt > /dev/null && echo "Vectors identical" || echo "Vectors differ"
––– output –––
- Vectors identical
+ ERROR 1064 (42000) at line 1: table test_json_rowwise: parse error: unknown column: embedding
+ ERROR 1064 (42000) at line 1: table test_json_rowwise: parse error: unknown column: embedding
+ Vectors identical
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_json_rowwise"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_json_rowwise; OPTIMIZE TABLE test_json_rowwise OPTION sync=1, cutoff=1"; echo $?
––– output –––
OK
––– input –––
VECTOR=$(python3 -c "print(','.join(['0.01']*384))")
curl -s -X POST http://localhost:9308/search -d "{\"index\":\"test_json_rowwise\",\"knn\":{\"field\":\"embedding\",\"query_vector\":[$VECTOR],\"k\":2}}" | jq -r '.hits.total // "0"'
––– output –––
- 5
+ 0

test/clt-tests/mcl/auto-embeddings-parameter-combinations.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
php /manticore/test/clt-tests/mcl/mock-embeddings-server.php --port 8080 --provider openai > /tmp/mock-server-8080.log 2>&1 &
MOCK_PID_8080=$!
echo "Mock server started on port 8080 (PID: $MOCK_PID_8080)"
sleep 1
––– output –––
OK
––– input –––
php /manticore/test/clt-tests/mcl/mock-embeddings-server.php --port 8081 --provider voyage > /tmp/mock-server-8081.log 2>&1 &
MOCK_PID_8081=$!
echo "Mock server started on port 8081 (PID: $MOCK_PID_8081)"
sleep 1
––– output –––
OK
––– input –––
php /manticore/test/clt-tests/mcl/mock-embeddings-server.php --port 8083 > /tmp/mock-server-8083.log 2>&1 &
MOCK_PID_8083=$!
echo "Generic mock server started on port 8083 (PID: $MOCK_PID_8083)"
sleep 1
––– output –––
OK
––– input –––
php /manticore/test/clt-tests/mcl/mock-embeddings-server.php --port 8084 --provider openai --delay 6.0 > /tmp/mock-server-8084.log 2>&1 &
MOCK_PID_8084=$!
echo "Slow mock server started on port 8084 (PID: $MOCK_PID_8084)"
sleep 1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_all_params (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='valid-key' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_all_params" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_url_timeout_no_key (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='30')" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_url_invalid_key (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='wrong' API_URL='http://localhost:8080/v1/embeddings')" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_timeout_slow_server (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8084/v1/embeddings' API_TIMEOUT='5')" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_url_provider_mismatch (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8081/v1/embeddings')" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_url_provider_override (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8083/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_url_provider_override (id, title) VALUES (1, 'test provider=openai')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_registry_url1 (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_registry_url2 (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8083/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_registry_url1 (id, title) VALUES (1, 'test document 1')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_registry_url2 (id, title) VALUES (1, 'test document 2')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_registry_timeout1 (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8084/v1/embeddings' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_registry_timeout2 (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8084/v1/embeddings' API_TIMEOUT='60')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_registry_timeout1 (id, title) VALUES (1, 'test delay=6.0')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_registry_timeout2 (id, title) VALUES (1, 'test delay=6.0')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_registry_same1 (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_registry_same2 (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_registry_same1 (id, title) VALUES (1, 'test document 1')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_registry_same2 (id, title) VALUES (1, 'test document 2')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_long_url (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings?param1=value1&param2=value2&param3=value3&param4=value4&param5=value5&param6=value6&param7=value7&param8=value8&param9=value9&param10=value10')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_special_chars_url (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings?param=value&other=test')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_auth_url (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://user:pass@localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_concurrent_alter (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_nonexistent_column (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title' API_KEY='${OPENAI_API_KEY}')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "ALTER TABLE test_nonexistent_column MODIFY COLUMN nonexistent API_URL='http://localhost:8080/v1/embeddings'" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_local_api_url (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2' FROM='title' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_local_api_url': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_local_api_url (id, title) VALUES (1, 'test document')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_local_api_timeout (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2' FROM='title' API_TIMEOUT='30')" 2>&1; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_local_api_timeout': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_local_api_timeout (id, title) VALUES (1, 'test document')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_local_api_key (title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2' FROM='title' API_KEY='any-key')" 2>&1; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_local_api_key': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_local_api_key (id, title) VALUES (1, 'test document')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_from_api_url (title TEXT, description TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='openai/text-embedding-ada-002' FROM='title,description' API_KEY='test-key' API_URL='http://localhost:8080/v1/embeddings')" 2>&1; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_from_api_url (id, title, description) VALUES (1, 'test title', 'test description')" 2>&1; echo $?
––– output –––
OK

test/clt-tests/mcl/auto-embeddings-qwen.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_qwen (title TEXT, vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='Qwen/Qwen3-Embedding-0.6B' FROM='title')"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_qwen': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_qwen"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
-        Table: test_qwen
- Create Table: CREATE TABLE test_qwen (
- id bigint,
- title text,
- vec float_vector knn_type='hnsw' hnsw_similarity='L2' model_name='Qwen/Qwen3-Embedding-0.6B' from='title' api_timeout='10'
- )
––– input –––
mysql -h0 -P9306 -e "insert into test_qwen(id, title) values(1, 'book'),(2, 'bread');"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total_records FROM test_qwen"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select id, title, knn_dist() from test_qwen where knn(vec, 3, 'loaf')"
––– output –––
- +------+-------+------------+
+ ERROR 1064 (42000) at line 1: table test_qwen: requested KNN search attribute 'vec' not found
- | id   | title | knn_dist() |
- +------+-------+------------+
- |    2 | bread | #!/0\.3[0-9]*/!# |
- |    1 | book  | #!/0\.[45][0-9]*/!# |
- +------+-------+------------+

test/clt-tests/mcl/auto-embeddings-error-handling.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_dims (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='384' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
)" 2>&1
# Check if table was actually created
mysql -h0 -P9306 -e "SHOW TABLES LIKE 'test_dims'" | grep -q "test_dims" && echo "Table created" || echo "Table not created"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_auto_dims (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
)"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_auto_dims': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_auto_dims (id, title) VALUES (1, 'Test document')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_no_model (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    FROM='title'
)"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_no_from (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
)" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_from (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM=''
)"
––– output –––
+ ERROR 1064 (42000) at line 1: error adding table 'test_empty_from': prealloc: Failed to download model configuration
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_bad_model (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='non-existent-model/invalid-name'
    FROM='title'
)" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_no_prefix (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='all-MiniLM-L6-v2'
    FROM='content_text'
)" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_bad_from (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='non_existent_field'
)"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_circular (
    title TEXT,
    vec1 FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='vec1'
)" 2>&1
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_empty': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, content) VALUES (1, '')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, content) VALUES (2, NULL)" 2>&1
––– output –––
OK
––– input –––
mysql -NE -h0 -P9306 -e "SELECT COUNT(*) FROM test_empty"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_rowwise (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='rowwise'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_rowwise': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_rowwise (id, content) VALUES
    (1, 'machine learning'),
    (2, 'deep learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_rowwise; OPTIMIZE TABLE test_rowwise OPTION sync=1, cutoff=1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_rowwise WHERE KNN(vec, 1, 'artificial intelligence')"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: table test_rowwise: requested KNN search attribute 'vec' not found
- | id   |
- +------+
- |    1 |
- |    2 |
- +------+
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_vec_columnar (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' engine='columnar'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
)"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_vec_columnar': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_vec_columnar (id, content) VALUES
    (1, 'machine learning'),
    (2, 'deep learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_vec_columnar; OPTIMIZE TABLE test_vec_columnar OPTION sync=1, cutoff=1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_vec_columnar WHERE KNN(vec, 1, 'artificial intelligence')"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: table test_vec_columnar: requested KNN search attribute 'vec' not found
- | id   |
- +------+
- |    1 |
- |    2 |
- +------+
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_full_columnar (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_full_columnar': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_full_columnar (id, content) VALUES
    (1, 'machine learning'),
    (2, 'deep learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "FLUSH RAMCHUNK test_full_columnar; OPTIMIZE TABLE test_full_columnar OPTION sync=1, cutoff=1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_full_columnar WHERE KNN(vec, 1, 'artificial intelligence')"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: table test_full_columnar: requested KNN search attribute 'vec' not found
- | id   |
- +------+
- |    1 |
- |    2 |
- +------+
––– input –––
echo "Row-wise (default):"
mysql -h0 -P9306 -e "SHOW CREATE TABLE test_rowwise\G" | grep -E "vec.*float_vector"
––– output –––
Row-wise (default):
- vec float_vector knn_type='hnsw' hnsw_similarity='L2' model_name='sentence-transformers/all-MiniLM-L6-v2' from='content' api_timeout='10'
––– input –––
echo "Vec columnar only:"
mysql -h0 -P9306 -e "SHOW CREATE TABLE test_vec_columnar\G" | grep -E "vec.*float_vector"
––– output –––
Vec columnar only:
- vec float_vector engine='columnar' knn_type='hnsw' hnsw_similarity='L2' model_name='sentence-transformers/all-MiniLM-L6-v2' from='content' api_timeout='10'
––– input –––
echo "Full columnar:"
mysql -h0 -P9306 -e "SHOW CREATE TABLE test_full_columnar\G" | grep -E "(vec.*float_vector|engine='columnar')"
––– output –––
Full columnar:
- vec float_vector knn_type='hnsw' hnsw_similarity='L2' model_name='sentence-transformers/all-MiniLM-L6-v2' from='content' api_timeout='10'
- ) engine='columnar'
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_auto_dims WHERE KNN(wrong_field, 1, 'test')" 2>&1 | grep -o "wrong_field.*not found"
––– output –––
OK

test/clt-tests/mcl/auto-embeddings-knn-select-scenarios.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_knn_select (id BIGINT, title TEXT, content TEXT, category string attribute, embedding FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2' MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2' FROM='title, content') engine='columnar'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: error adding table 'test_knn_select': prealloc: Failed to download model configuration
+ 1
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_knn_select"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
-        Table: test_knn_select
- Create Table: CREATE TABLE test_knn_select (
- id bigint,
- title text,
- content text,
- category string attribute,
- embedding float_vector knn_type='hnsw' hnsw_similarity='L2' model_name='sentence-transformers/all-MiniLM-L6-v2' from='title, content' api_timeout='10'
- ) engine='columnar'
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_knn_select (id, title, content, category) VALUES (1, 'machine learning algorithms', 'deep neural networks and artificial intelligence', 'technology'), (2, 'cooking recipes', 'delicious pasta and italian cuisine', 'food'), (3, 'travel destinations', 'beautiful beaches and mountain hiking', 'travel'), (4, 'artificial intelligence', 'machine learning and deep learning models', 'technology'), (5, 'healthy eating', 'nutritious meals and balanced diet', 'food')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total_records FROM test_knn_select"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, (0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4,0.2, 0.3, 0.4,0.2, 0.3, 0.4,0.2, 0.3, 0.4,0.2, 0.3, 0.4,0.2, 0.3, 0.4,0.2, 0.3, 0.4,0.2, 0.3))"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 141.302#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 4
-      title: artificial intelligence
- knn_dist(): 141.419#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 2
-      title: cooking recipes
- knn_dist(): 141.779#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 3
-      title: travel destinations
- knn_dist(): 142.266#!/[0-9]{0,5}/!#
- *************************** 5. row ***************************
-         id: 5
-      title: healthy eating
- knn_dist(): 142.354#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'machine learning artificial intelligence')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 4
-      title: artificial intelligence
- knn_dist(): 0.590#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 0.601#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 2
-      title: cooking recipes
- knn_dist(): 1.779#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 3
-      title: travel destinations
- knn_dist(): 1.940#!/[0-9]{0,5}/!#
- *************************** 5. row ***************************
-         id: 5
-      title: healthy eating
- knn_dist(): 1.976#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 2, '')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 2
-      title: cooking recipes
- knn_dist(): 1.885#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 1.922#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 3
-      title: travel destinations
- knn_dist(): 1.935#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 4
-      title: artificial intelligence
- knn_dist(): 1.947#!/[0-9]{0,5}/!#
- *************************** 5. row ***************************
-         id: 5
-      title: healthy eating
- knn_dist(): 1.977#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 2, 'food cooking recipes')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 2
-      title: cooking recipes
- knn_dist(): 0.807#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 5
-      title: healthy eating
- knn_dist(): 1.310#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 4
-      title: artificial intelligence
- knn_dist(): 1.739#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 1.747#!/[0-9]{0,5}/!#
- *************************** 5. row ***************************
-         id: 3
-      title: travel destinations
- knn_dist(): 1.831#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 2, 1)"
––– output –––
- *************************** 1. row ***************************
-         id: 4
-      title: artificial intelligence
- knn_dist(): 0.225#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 2
-      title: cooking recipes
- knn_dist(): 1.762#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 3
-      title: travel destinations
- knn_dist(): 1.838#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 5
-      title: healthy eating
- knn_dist(): 1.974#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, category, knn_dist() FROM test_knn_select WHERE knn(embedding, 5, 'technology computers') AND category = 'technology'"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 4
-      title: artificial intelligence
-   category: technology
- knn_dist(): 1.328#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 1
-      title: machine learning algorithms
-   category: technology
- knn_dist(): 1.357#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'travel vacation destinations', {ef=100, rescore=1, oversampling=2.0})"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 3
-      title: travel destinations
- knn_dist(): 0.595#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 2
-      title: cooking recipes
- knn_dist(): 1.772#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 4
-      title: artificial intelligence
- knn_dist(): 1.796#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 5
-      title: healthy eating
- knn_dist(): 1.869#!/[0-9]{0,5}/!#
- *************************** 5. row ***************************
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 1.875#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title FROM test_knn_select WHERE knn(embedding, 3, 'search query with special chars @#$%^&*()')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-    id: 4
- title: artificial intelligence
- *************************** 2. row ***************************
-    id: 3
- title: travel destinations
- *************************** 3. row ***************************
-    id: 1
- title: machine learning algorithms
- *************************** 4. row ***************************
-    id: 5
- title: healthy eating
- *************************** 5. row ***************************
-    id: 2
- title: cooking recipes
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'artificial intelligence') AND MATCH('machine')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 4
-      title: artificial intelligence
- knn_dist(): 0.846#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 0.908#!/[0-9]{0,5}/!#
––– input –––
mysql -h0 -P9306 -E -e "SELECT id, title, knn_dist() FROM test_knn_select WHERE knn(embedding, 5, 'healthy nutrition food')"
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
-         id: 5
-      title: healthy eating
- knn_dist(): 0.472#!/[0-9]{0,5}/!#
- *************************** 2. row ***************************
-         id: 2
-      title: cooking recipes
- knn_dist(): 1.500#!/[0-9]{0,5}/!#
- *************************** 3. row ***************************
-         id: 3
-      title: travel destinations
- knn_dist(): 1.803#!/[0-9]{0,5}/!#
- *************************** 4. row ***************************
-         id: 1
-      title: machine learning algorithms
- knn_dist(): 1.977#!/[0-9]{0,5}/!#
- *************************** 5. row ***************************
-         id: 4
-      title: artificial intelligence
- knn_dist(): 1.989#!/[0-9]{0,5}/!#
––– input –––
QUERY1_MD5=$(mysql -h0 -P9306 -E -e "SELECT id, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'machine learning')" | md5sum | awk '{print $1}'); QUERY2_MD5=$(mysql -h0 -P9306 -E -e "SELECT id, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'machine learning')" | md5sum | awk '{print $1}'); echo "query1_md5: $QUERY1_MD5"; echo "query2_md5: $QUERY2_MD5"; if [ "$QUERY1_MD5" = "$QUERY2_MD5" ]; then echo "SUCCESS: Same text query produces consistent results"; else echo "FAIL: Same text query produces different results"; fi
––– output –––
- query1_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
- query2_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
- SUCCESS: Same text query produces consistent results
+ query1_md5: d41d8cd98f00b204e9800998ecf8427e
+ query2_md5: d41d8cd98f00b204e9800998ecf8427e
+ SUCCESS: Same text query produces consistent results
––– input –––
TECH_MD5=$(mysql -h0 -P9306 -E -e "SELECT id, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'technology computers')" | md5sum | awk '{print $1}'); FOOD_MD5=$(mysql -h0 -P9306 -E -e "SELECT id, knn_dist() FROM test_knn_select WHERE knn(embedding, 3, 'food cooking')" | md5sum | awk '{print $1}'); echo "tech_query_md5: $TECH_MD5"; echo "food_query_md5: $FOOD_MD5"; if [ "$TECH_MD5" != "$FOOD_MD5" ]; then echo "SUCCESS: Different text queries produce different results"; else echo "INFO: Query comparison result"; fi
––– output –––
- tech_query_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
- food_query_md5: #!/[a-f0-9]{32}/!#
+ ERROR 1064 (42000) at line 1: table test_knn_select: requested KNN search attribute 'embedding' not found
- #!/SUCCESS: Different text queries produce different results|INFO: Query comparison result/!#
+ tech_query_md5: d41d8cd98f00b204e9800998ecf8427e
+ food_query_md5: d41d8cd98f00b204e9800998ecf8427e
+ INFO: Query comparison result
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_no_auto (id BIGINT, title TEXT, embedding FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='384' HNSW_SIMILARITY='l2') engine='columnar'"; mysql -h0 -P9306 -e "INSERT INTO test_no_auto (id, title, embedding) VALUES (1, 'test', (0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2, 0.3, 0.4))"; mysql -h0 -P9306 -E -e "SELECT id, title FROM test_no_auto WHERE knn(embedding, 2, 'test query')"
––– output –––
OK

github-actions · 2026-06-03T14:41:58Z

clt

❌ CLT tests in test/clt-tests/sharding/functional/functional-sharding-and-cluster-negative test/clt-tests/sharding/functional/functional-sharding-positive test/clt-tests/sharding/functional/test-dist-table-shards-5000 test/clt-tests/sharding/replication/create-table-1-node-10-shards test/clt-tests/sharding/replication/create-table-2-nodes-4-shards test/clt-tests/sharding/replication/create-table-3-nodes-6-shards test/clt-tests/sharding/replication/create-table-5-nodes-60-shards test/clt-tests/sharding/replication/test-data-manipulation test/clt-tests/sharding/replication/test-distributed-inserts-with-replication test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication test/clt-tests/sharding/syntax/sharding-syntax-negative test/clt-tests/sharding/syntax/sharding-syntax-positive test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure
✅ OK: 12
❌ Failed: 1
⏳ Duration: 296s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

Edit test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure.rec

test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:test_products (id bigint, name string, price float) shards='2' rf='2'" & sleep 3; export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; sleep 1; echo "Node 2 killed"
––– output –––
- Node 2 killed
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ Node 2 killed
––– input –––
sleep 15; echo "Wait completed"
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 10; mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as shard_count FROM system.sharding_table WHERE table = 'test_products'"
––– output –––
- +-------------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
- | shard_count |
- +-------------+
- |           2 |
- +-------------+

github-actions · 2026-06-03T14:51:06Z

clt

❌ CLT tests in test/clt-tests/sharding/rollback/data-consistency-multiple-failures test/clt-tests/sharding/rollback/master-node-failure test/clt-tests/sharding/rollback/master-takeover-while-rollback test/clt-tests/sharding/rollback/mixed-rf-tables-node-failure test/clt-tests/sharding/rollback/new-node-addition-rf-replication test/clt-tests/sharding/rollback/node-failure-mid-creation-recovery test/clt-tests/sharding/rollback/node-killed-rf-maintained test/clt-tests/sharding/rollback/node-rejoin-stale-cluster-cleanup test/clt-tests/sharding/rollback/node-removal-rf-maintained test/clt-tests/sharding/rollback/rebalance-master-killed-mid-operation test/clt-tests/sharding/rollback/rf1-no-rebalance-on-failure test/clt-tests/sharding/rollback/sequential-node-failures-high-rf test/clt-tests/sharding/regression/test-columnar-engine-shard
✅ OK: 1
❌ Failed: 12
⏳ Duration: 880s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

test/clt-tests/sharding/rollback/sequential-node-failures-high-rf.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='3' rf='3'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'Widget A'), (2, 'Widget B'), (3, 'Widget C')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:4312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 20 grep -qm1 'Skipping rebalance for table' <(tail -n 100 -f /var/log/manticore-{3,4}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P3306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-3/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-3/searchd.log); echo $?
––– output –––
- 0
+ tail: /var/log/manticore-3/searchd.log: file truncated
+ 124
––– input –––
for i in 1 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:4312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK

test/clt-tests/sharding/rollback/master-takeover-while-rollback.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:1312 127.0.0.1:2312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312

test/clt-tests/sharding/rollback/node-failure-mid-creation-recovery.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:orders (id bigint, total float) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO orders (id, total) VALUES (1, 100), (2, 200)"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
for port in 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 2306: synced
- Port 3306: synced
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced

test/clt-tests/sharding/rollback/data-consistency-multiple-failures.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, account string, amount float, ts int) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, account, amount, ts) VALUES (1, 'ACC001', 100.50, 1000), (2, 'ACC002', 200.75, 2000), (3, 'ACC003', 150.25, 3000), (4, 'ACC001', 300.00, 4000), (5, 'ACC002', 250.50, 5000)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"
––– output –––
- 5	1002.000000
+ +------+-------------+
+ |    5 | 1002.000000 |
+ +------+-------------+
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "INSERT INTO ${TABLE_NAME} (id, account, amount, ts) VALUES (6, 'ACC003', 175.00, 6000), (7, 'ACC001', 225.75, 7000)"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{1,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 1 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"
––– output –––
- 7	1402.75
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
––– input –––
for port in 1306 2306 3306; do echo "Node $port:"; mysql -h0 -P$port -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"; done
––– output –––
Node 1306:
- 7	1402.75
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
Node 2306:
- 7	1402.75
+ +------+------------+
- Node 3306:
+ |    3 | 476.250000 |
- 7	1402.75
+ +------+------------+
+ Node 3306:
+ +------+-------------+
+ |    7 | 1402.750000 |
+ +------+-------------+
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS ${TABLE_NAME}"; echo $?
––– output –––
OK

test/clt-tests/sharding/rollback/new-node-addition-rf-replication.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:existing_table (id bigint, data string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -sN -e "SELECT COUNT(*) FROM system.sharding_table WHERE table = 'existing_table'"; done
––– output –––
- 2
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
- 2
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P3306 -e "JOIN CLUSTER c at '127.0.0.1:1312'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: cluster 'c', no nodes available(127.0.0.1:1312), error: '127.0.0.1:1312': retries limit exceeded
+ 1
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done
––– output –––
- Port 1306: Node synced
- Port 2306: Node synced
- Port 3306: Node synced
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM system.sharding_state"
––– output –––
- 3
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_state' in search request
––– input –––
sleep 15; echo "Cluster stabilized"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS existing_table"
––– output –––
OK

test/clt-tests/sharding/rollback/node-rejoin-stale-cluster-cleanup.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'alpha'), (2, 'beta')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-2/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
- |    2 |
- +------+

test/clt-tests/sharding/rollback/rf1-no-rebalance-on-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='1'"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: local
- system.t_s1: 127.0.0.1:2312
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'Widget A'), (2, 'Widget B'), (3, 'Widget C'), (4, 'Widget D')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as total FROM ${TABLE_NAME}\\G" | grep total
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'Skipping rebalance for table' <(tail -n 1000 -f /var/log/manticore-1/searchd.log); echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: local
- system.t_s1: 127.0.0.1:2312
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as cnt FROM ${TABLE_NAME}\\G" | grep -E 'cnt:' | awk '{print $2}'
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; for port in 1306 2306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as cnt FROM ${TABLE_NAME}\\G" | grep -E 'cnt:' | awk '{print $2}'
––– output –––
OK

test/clt-tests/sharding/rollback/node-removal-rf-maintained.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:items (id bigint, name string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO items (id, name) VALUES (1, 'Item1'), (2, 'Item2'), (3, 'Item3')"; echo $?
––– output –––
OK
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced
––– input –––
mysql -h0 -P1306 -e "show tables from system\\G" | grep -E 'items_s[0-9]' | wc -l
––– output –––
- 2
+ 0

test/clt-tests/sharding/rollback/node-killed-rf-maintained.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:data (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (1, 'test1'), (2, 'test2')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
sleep 10; mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (3, 'test3')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 20
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM data"
––– output –––
- 3
+ +------+
+ |    3 |
+ +------+
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM data"
––– output –––
- 3
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'data' in search request
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS data"; echo $?
––– output –––
OK

test/clt-tests/sharding/rollback/master-node-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:data (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
sleep 5; mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (1, 'test_value')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
for port in 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 2306: synced
- Port 3306: synced
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced

test/clt-tests/sharding/rollback/rebalance-master-killed-mid-operation.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='3' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'one'), (2, 'two'), (3, 'three')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s2: 127.0.0.1:3312 127.0.0.1:4312
––– input –––
export INSTANCE=4; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 4 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 20 grep -qm1 'Rebalancing due to inactive nodes' <(tail -n 100 -f /var/log/manticore-1/searchd.log); echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Master killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 30 grep -qm1 'becoming master' <(tail -n 100 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
sleep 5
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-{2,3}/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:3312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:3312 127.0.0.1:2312
- system.t_s2: 127.0.0.1:3312 127.0.0.1:2312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK

test/clt-tests/sharding/rollback/mixed-rf-tables-node-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:norep (id bigint, value string) shards='2' rf='1'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Waiting timeout exceeded.
+ 1
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:rep (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO norep (id, value) VALUES (1, 'a'), (2, 'b')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO rep (id, value) VALUES (1, 'x'), (2, 'y')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 1306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Skipping rebalance for table norep' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 2306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM rep"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-2/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 1306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:1312 127.0.0.1:2312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM norep"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM rep"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'rep' in search request
- |    2 |
- +------+

github-actions · 2026-06-03T15:48:32Z

clt

❌ CLT tests in test/clt-tests/sharding/functional/functional-sharding-and-cluster-negative test/clt-tests/sharding/functional/functional-sharding-positive test/clt-tests/sharding/functional/test-dist-table-shards-5000 test/clt-tests/sharding/replication/create-table-1-node-10-shards test/clt-tests/sharding/replication/create-table-2-nodes-4-shards test/clt-tests/sharding/replication/create-table-3-nodes-6-shards test/clt-tests/sharding/replication/create-table-5-nodes-60-shards test/clt-tests/sharding/replication/test-data-manipulation test/clt-tests/sharding/replication/test-distributed-inserts-with-replication test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication test/clt-tests/sharding/syntax/sharding-syntax-negative test/clt-tests/sharding/syntax/sharding-syntax-positive test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure
✅ OK: 12
❌ Failed: 1
⏳ Duration: 305s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

Edit test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure.rec

test/clt-tests/sharding/rollback/automatic-rollback-on-creation-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:test_products (id bigint, name string, price float) shards='2' rf='2'" & sleep 3; export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; sleep 1; echo "Node 2 killed"
––– output –––
- Node 2 killed
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ Node 2 killed
––– input –––
sleep 15; echo "Wait completed"
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 10; mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as shard_count FROM system.sharding_table WHERE table = 'test_products'"
––– output –––
- +-------------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
- | shard_count |
- +-------------+
- |           2 |
- +-------------+

github-actions · 2026-06-03T15:58:08Z

clt

❌ CLT tests in test/clt-tests/sharding/rollback/data-consistency-multiple-failures test/clt-tests/sharding/rollback/master-node-failure test/clt-tests/sharding/rollback/master-takeover-while-rollback test/clt-tests/sharding/rollback/mixed-rf-tables-node-failure test/clt-tests/sharding/rollback/new-node-addition-rf-replication test/clt-tests/sharding/rollback/node-failure-mid-creation-recovery test/clt-tests/sharding/rollback/node-killed-rf-maintained test/clt-tests/sharding/rollback/node-rejoin-stale-cluster-cleanup test/clt-tests/sharding/rollback/node-removal-rf-maintained test/clt-tests/sharding/rollback/rebalance-master-killed-mid-operation test/clt-tests/sharding/rollback/rf1-no-rebalance-on-failure test/clt-tests/sharding/rollback/sequential-node-failures-high-rf test/clt-tests/sharding/regression/test-columnar-engine-shard
✅ OK: 1
❌ Failed: 12
⏳ Duration: 897s
👉 Check Action Results for commit e1f50e0

Failed tests:

🔧 Edit failed tests in UI:

test/clt-tests/sharding/rollback/sequential-node-failures-high-rf.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='3' rf='3'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'Widget A'), (2, 'Widget B'), (3, 'Widget C')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:4312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 20 grep -qm1 'Skipping rebalance for table' <(tail -n 100 -f /var/log/manticore-{3,4}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P3306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-3/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-3/searchd.log); echo $?
––– output –––
- 0
+ tail: /var/log/manticore-3/searchd.log: file truncated
+ 124
––– input –––
for i in 1 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312 127.0.0.1:4312
- system.t_s2: 127.0.0.1:1312 127.0.0.1:2312 127.0.0.1:4312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK

test/clt-tests/sharding/rollback/master-takeover-while-rollback.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:1312 127.0.0.1:2312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312

test/clt-tests/sharding/rollback/node-failure-mid-creation-recovery.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:orders (id bigint, total float) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO orders (id, total) VALUES (1, 100), (2, 200)"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
for port in 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 2306: synced
- Port 3306: synced
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced

test/clt-tests/sharding/rollback/data-consistency-multiple-failures.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, account string, amount float, ts int) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, account, amount, ts) VALUES (1, 'ACC001', 100.50, 1000), (2, 'ACC002', 200.75, 2000), (3, 'ACC003', 150.25, 3000), (4, 'ACC001', 300.00, 4000), (5, 'ACC002', 250.50, 5000)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"
––– output –––
- 5	1002.000000
+ +------+-------------+
+ |    5 | 1002.000000 |
+ +------+-------------+
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "INSERT INTO ${TABLE_NAME} (id, account, amount, ts) VALUES (6, 'ACC003', 175.00, 6000), (7, 'ACC001', 225.75, 7000)"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'becoming master' <(tail -n 1000 -f /var/log/manticore-{1,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
for i in 1 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 2
––– output –––
OK
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"
––– output –––
- 7	1402.75
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
––– input –––
for port in 1306 2306 3306; do echo "Node $port:"; mysql -h0 -P$port -sN -e "SELECT COUNT(*), SUM(amount) FROM ${TABLE_NAME}"; done
––– output –––
Node 1306:
- 7	1402.75
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
Node 2306:
- 7	1402.75
+ +------+------------+
- Node 3306:
+ |    3 | 476.250000 |
- 7	1402.75
+ +------+------------+
+ Node 3306:
+ +------+-------------+
+ |    7 | 1402.750000 |
+ +------+-------------+
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS ${TABLE_NAME}"; echo $?
––– output –––
OK

test/clt-tests/sharding/rollback/new-node-addition-rf-replication.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:existing_table (id bigint, data string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -sN -e "SELECT COUNT(*) FROM system.sharding_table WHERE table = 'existing_table'"; done
––– output –––
- 2
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
- 2
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_table' in search request
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P3306 -e "JOIN CLUSTER c at '127.0.0.1:1312'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: cluster 'c', no nodes available(127.0.0.1:1312), error: '127.0.0.1:1312': retries limit exceeded
+ 1
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done
––– output –––
- Port 1306: Node synced
- Port 2306: Node synced
- Port 3306: Node synced
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM system.sharding_state"
––– output –––
- 3
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'system.sharding_state' in search request
––– input –––
sleep 15; echo "Cluster stabilized"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS existing_table"
––– output –––
OK

test/clt-tests/sharding/rollback/node-rejoin-stale-cluster-cleanup.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'alpha'), (2, 'beta')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-2/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 't' in search request
- |    2 |
- +------+

test/clt-tests/sharding/rollback/rf1-no-rebalance-on-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='2' rf='1'"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: local
- system.t_s1: 127.0.0.1:2312
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'Widget A'), (2, 'Widget B'), (3, 'Widget C'), (4, 'Widget D')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as total FROM ${TABLE_NAME}\\G" | grep total
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 10 grep -qm1 'Skipping rebalance for table' <(tail -n 1000 -f /var/log/manticore-1/searchd.log); echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: local
- system.t_s1: 127.0.0.1:2312
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as cnt FROM ${TABLE_NAME}\\G" | grep -E 'cnt:' | awk '{print $2}'
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; for port in 1306 2306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT COUNT(*) as cnt FROM ${TABLE_NAME}\\G" | grep -E 'cnt:' | awk '{print $2}'
––– output –––
OK

test/clt-tests/sharding/rollback/node-removal-rf-maintained.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:items (id bigint, name string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO items (id, name) VALUES (1, 'Item1'), (2, 'Item2'), (3, 'Item3')"; echo $?
––– output –––
OK
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced
––– input –––
mysql -h0 -P1306 -e "show tables from system\\G" | grep -E 'items_s[0-9]' | wc -l
––– output –––
- 2
+ 0

test/clt-tests/sharding/rollback/node-killed-rf-maintained.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:data (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (1, 'test1'), (2, 'test2')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=2; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 2 killed"
––– output –––
OK
––– input –––
sleep 10; mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (3, 'test3')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 20
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_c_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM data"
––– output –––
- 3
+ +------+
+ |    3 |
+ +------+
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM data"
––– output –––
- 3
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'data' in search request
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS data"; echo $?
––– output –––
OK

test/clt-tests/sharding/rollback/master-node-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE c:data (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Cluster 'c' does not exist
+ 1
––– input –––
sleep 5; mysql -h0 -P1306 -e "INSERT INTO data (id, value) VALUES (1, 'test_value')"; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
for port in 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 2306: synced
- Port 3306: synced
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
sleep 15; mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_c_status'\\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_c_status
-   Value: primary
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: synced"; done
––– output –––
- Port 1306: synced
- Port 2306: synced
- Port 3306: synced

test/clt-tests/sharding/rollback/rebalance-master-killed-mid-operation.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:${TABLE_NAME} (id bigint, value string) shards='3' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO ${TABLE_NAME} (id, value) VALUES (1, 'one'), (2, 'two'), (3, 'three')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:2312 127.0.0.1:3312
- system.t_s2: 127.0.0.1:3312 127.0.0.1:4312
––– input –––
export INSTANCE=4; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 4 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 20 grep -qm1 'Rebalancing due to inactive nodes' <(tail -n 100 -f /var/log/manticore-1/searchd.log); echo $?
––– output –––
OK
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Master killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 30 grep -qm1 'becoming master' <(tail -n 100 -f /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
OK
––– input –––
sleep 5
––– output –––
OK
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-{2,3}/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-{2,3}/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3 4; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "$TABLE_NAME"
––– output –––
- system.t_s0: 127.0.0.1:3312 127.0.0.1:2312
- system.t_s1: 127.0.0.1:3312 127.0.0.1:2312
- system.t_s2: 127.0.0.1:3312 127.0.0.1:2312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM ${TABLE_NAME}"
––– output –––
OK

test/clt-tests/sharding/rollback/mixed-rf-tables-node-failure.rec

––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c TABLE_NAME=t
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
show_shards() {
	shards=$(mysql -h0 -P$1 -e "show create table $2 option force=1\G" | grep "Create Table" | cut -d: -f2-)

	(
		# Handle local shards
		echo $shards | grep -oP "local='[^']*'" | sed "s/local='//g; s/'//g" | tr ',' '\n' | while read -r tbl; do
			[ -n "$tbl" ] && echo "$tbl: local"
		done

		# Handle agent shards
		echo $shards | grep -oP "agent='[^']*'" | sed "s/agent='//g; s/'//g" | tr '|' '\n' | awk -F: '{agents[$3] = agents[$3] ? agents[$3] " " $1 ":" $2 : $1 ":" $2} END {for (t in agents) print t ": " agents[t]}'
	) | sort
}
––– output –––
OK
––– input –––
check_cluster_status() {
	mysql -h0 -P$1 -e "SHOW STATUS LIKE 'cluster_${2}_status'\\G" | grep -q "Value: primary" && echo "OK" || echo "FAIL"
}
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:norep (id bigint, value string) shards='2' rf='1'"; echo $?
––– output –––
- 0
+ ERROR 1064 (42000) at line 1: Waiting timeout exceeded.
+ 1
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE ${CLUSTER_NAME}:rep (id bigint, value string) shards='2' rf='2'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO norep (id, value) VALUES (1, 'a'), (2, 'b')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "INSERT INTO rep (id, value) VALUES (1, 'x'), (2, 'y')"; echo $?
––– output –––
OK
––– input –––
show_shards 1306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 1306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:1312 127.0.0.1:2312
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
export INSTANCE=1; stdbuf -oL searchd --stopwait -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null; echo "Node 1 killed"
––– output –––
OK
––– input –––
timeout 30 bash -c 'while lsof -i :${INSTANCE}306 &>/dev/null; do sleep 1; done'
––– output –––
OK
––– input –––
timeout 60 grep -qm1 'Skipping rebalance for table norep' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
timeout 60 grep -qm1 'Rebalancing completed' <(tail -n 100 -f /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 2306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 2306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:2312 127.0.0.1:3312
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P2306 -sN -e "SELECT COUNT(*) FROM rep"
––– output –––
OK
––– input –––
truncate -s 0 /var/log/manticore-2/searchd.log
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
timeout 90 grep -qm1 'Rebalancing completed' <(tail -n 100 -F /var/log/manticore-2/searchd.log); echo $?
––– output –––
- 0
+ 124
––– input –––
for i in 1 2 3; do check_cluster_status ${i}306 "$CLUSTER_NAME"; done | uniq
––– output –––
OK
––– input –––
show_shards 1306 "norep"
––– output –––
- system.norep_s0: 127.0.0.1:1312
- system.norep_s1: 127.0.0.1:2312
––– input –––
show_shards 1306 "rep"
––– output –––
- system.rep_s0: 127.0.0.1:1312 127.0.0.1:2312
+ ERROR 1064 (42000) at line 1: You have an error in your query. Please, double-check it.
- system.rep_s1: 127.0.0.1:2312 127.0.0.1:3312
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM norep"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -sN -e "SELECT COUNT(*) FROM rep"
––– output –––
- +------+
+ ERROR 1064 (42000) at line 1: unknown local table(s) 'rep' in search request
- |    2 |
- +------+

Bump buddy version from 3.46.3 to 3.46.4

459cd71

Bump buddy version from 3.46.3 to 3.47.0

13ae0e6

githubmanticore changed the title ~~🆕 Update buddy version from 3.46.3 to 3.46.4~~ 🆕 Update buddy version from 3.46.3 to 3.47.0 Jun 3, 2026

donhardman merged commit 973a0a3 into main Jun 3, 2026
120 of 130 checks passed

donhardman deleted the update-buddy-version branch June 3, 2026 17:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🆕 Update buddy version from 3.46.3 to 3.47.0#4606

🆕 Update buddy version from 3.46.3 to 3.47.0#4606
donhardman merged 2 commits into
mainfrom
update-buddy-version

githubmanticore commented Jun 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

githubmanticore commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linux debug test results

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

github-actions Bot commented Jun 3, 2026

clt

Failed tests:

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

githubmanticore commented Jun 1, 2026 •

edited

Loading

github-actions Bot commented Jun 1, 2026 •

edited

Loading