Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix] (inverted index) fix wrong result in complex compound expr (#40630) #40631

Merged
merged 3 commits into from
Sep 11, 2024
Merged

[fix] (inverted index) fix wrong result in complex compound expr (#40630) #40631

merged 3 commits into from
Sep 11, 2024

Conversation

csun5285
Copy link
Contributor

Proposed changes

pick from branch-2.1 #40630

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

run buildall

1 similar comment
@csun5285
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.87% (8143/21503)
Line Coverage: 29.60% (67020/226411)
Region Coverage: 29.09% (34580/118870)
Branch Coverage: 25.00% (17819/71284)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f2e2ccc9c511305ed28d449fdbe45f65468a6fae_f2e2ccc9c511305ed28d449fdbe45f65468a6fae/report/index.html

@xiaokang xiaokang changed the title [fix] (inverted index) fix error result (#40630) [fix] (inverted index) fix wrong result in complex compound (#40630) Sep 10, 2024
@xiaokang xiaokang changed the title [fix] (inverted index) fix wrong result in complex compound (#40630) [fix] (inverted index) fix wrong result in complex compound expr (#40630) Sep 10, 2024
@csun5285
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.85% (8139/21503)
Line Coverage: 29.59% (66984/226411)
Region Coverage: 29.07% (34557/118870)
Branch Coverage: 24.98% (17806/71284)
Coverage Report: http://coverage.selectdb-in.cc/coverage/87fcba71db343b8085bb2e3b75a614a9a5d854ae_87fcba71db343b8085bb2e3b75a614a9a5d854ae/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 48955 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 87fcba71db343b8085bb2e3b75a614a9a5d854ae, data reload: false

------ Round 1 ----------------------------------
q1	17817	4348	4343	4343
q2	2034	180	142	142
q3	10319	1925	1892	1892
q4	10331	1270	1322	1270
q5	8903	3849	3901	3849
q6	234	124	119	119
q7	2031	1618	1577	1577
q8	9265	2696	2694	2694
q9	10145	9931	9827	9827
q10	8620	3570	3463	3463
q11	426	252	251	251
q12	465	293	310	293
q13	18340	3988	4011	3988
q14	359	325	330	325
q15	511	458	458	458
q16	547	438	470	438
q17	1121	983	957	957
q18	7218	6845	6987	6845
q19	1689	1549	1504	1504
q20	553	304	297	297
q21	4458	4099	4027	4027
q22	499	396	399	396
Total cold run time: 115885 ms
Total hot run time: 48955 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4323	4287	4302	4287
q2	322	226	224	224
q3	4171	4147	4101	4101
q4	2737	2734	2733	2733
q5	7124	7075	7062	7062
q6	237	118	118	118
q7	3275	2916	2800	2800
q8	4286	4457	4473	4457
q9	13556	13477	13565	13477
q10	4233	4290	4219	4219
q11	756	677	674	674
q12	1026	864	849	849
q13	7100	3708	3775	3708
q14	454	426	422	422
q15	497	466	444	444
q16	627	583	584	583
q17	3825	3840	3856	3840
q18	8839	8686	8629	8629
q19	1700	1639	1628	1628
q20	2372	2123	2134	2123
q21	8469	8394	8380	8380
q22	1060	958	945	945
Total cold run time: 80989 ms
Total hot run time: 75703 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212850 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 87fcba71db343b8085bb2e3b75a614a9a5d854ae, data reload: false

query1	929	396	417	396
query2	6567	2201	2120	2120
query3	6924	208	207	207
query4	23093	21595	21733	21595
query5	19761	6520	6536	6520
query6	282	221	227	221
query7	4328	300	306	300
query8	264	260	237	237
query9	3030	2628	2564	2564
query10	462	306	305	305
query11	15726	15567	15109	15109
query12	122	74	71	71
query13	1027	434	433	433
query14	17002	13567	13895	13567
query15	370	220	225	220
query16	6640	295	265	265
query17	1852	927	914	914
query18	888	316	307	307
query19	204	153	150	150
query20	79	75	80	75
query21	192	92	101	92
query22	5306	5194	5037	5037
query23	34241	33605	33585	33585
query24	7836	6308	6279	6279
query25	522	435	433	433
query26	1273	161	160	160
query27	2428	293	288	288
query28	6074	2231	2222	2222
query29	2922	2718	2874	2718
query30	238	171	169	169
query31	965	729	748	729
query32	71	63	63	63
query33	457	263	261	261
query34	875	488	477	477
query35	1133	940	941	940
query36	1362	1319	1111	1111
query37	176	58	64	58
query38	3126	2893	2968	2893
query39	1362	1324	1330	1324
query40	311	95	96	95
query41	40	37	36	36
query42	89	89	80	80
query43	672	698	565	565
query44	1208	721	730	721
query45	253	232	227	227
query46	1224	960	965	960
query47	1894	1798	1714	1714
query48	531	400	403	400
query49	652	384	395	384
query50	866	659	629	629
query51	4735	4665	4666	4665
query52	93	83	84	83
query53	239	191	178	178
query54	2658	2450	2475	2450
query55	92	87	87	87
query56	232	216	222	216
query57	1211	1108	1272	1108
query58	218	214	209	209
query59	3497	3294	3284	3284
query60	227	213	205	205
query61	95	94	101	94
query62	835	511	490	490
query63	205	175	173	173
query64	3512	1614	1492	1492
query65	3656	3568	3565	3565
query66	766	420	393	393
query67	16134	15502	15025	15025
query68	9243	651	668	651
query69	494	273	272	272
query70	1703	1418	1314	1314
query71	396	307	318	307
query72	6857	4770	4774	4770
query73	771	323	326	323
query74	6286	5851	5911	5851
query75	4662	3644	3700	3644
query76	4749	1180	1222	1180
query77	659	265	264	264
query78	12478	11640	12742	11640
query79	7282	637	656	637
query80	2320	388	386	386
query81	522	238	240	238
query82	665	104	102	102
query83	172	141	135	135
query84	258	70	72	70
query85	1299	318	318	318
query86	360	295	309	295
query87	3211	3024	3044	3024
query88	4383	2404	2394	2394
query89	374	307	302	302
query90	1787	220	212	212
query91	157	131	128	128
query92	59	52	54	52
query93	4202	541	558	541
query94	799	211	206	206
query95	2039	1955	2009	1955
query96	647	329	334	329
query97	6471	6314	6348	6314
query98	232	215	200	200
query99	3021	861	853	853
Total cold run time: 314812 ms
Total hot run time: 212850 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.43 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 87fcba71db343b8085bb2e3b75a614a9a5d854ae, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.05
query4	1.80	0.07	0.07
query5	0.55	0.53	0.52
query6	1.24	0.62	0.67
query7	0.01	0.01	0.02
query8	0.03	0.02	0.02
query9	0.54	0.50	0.49
query10	0.55	0.55	0.54
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.80	0.79	0.76
query15	0.76	0.75	0.77
query16	0.36	0.37	0.37
query17	1.01	1.02	0.99
query18	0.22	0.26	0.24
query19	1.90	1.85	1.87
query20	0.01	0.01	0.01
query21	15.45	0.56	0.55
query22	1.98	2.10	1.51
query23	17.16	1.10	1.01
query24	3.86	2.25	1.64
query25	0.39	0.10	0.07
query26	0.59	0.15	0.16
query27	0.04	0.04	0.04
query28	7.40	0.71	0.74
query29	12.64	2.38	2.34
query30	0.60	0.53	0.52
query31	2.81	0.39	0.37
query32	3.38	0.50	0.49
query33	3.10	3.08	3.07
query34	15.25	4.78	4.79
query35	4.86	4.83	4.84
query36	1.06	1.02	1.03
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.13
query41	0.07	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.01	0.02
Total cold run time: 101.93 s
Total hot run time: 31.43 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 87fcba71db343b8085bb2e3b75a614a9a5d854ae with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

@csun5285
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.86% (8141/21503)
Line Coverage: 29.59% (67000/226411)
Region Coverage: 29.08% (34570/118870)
Branch Coverage: 24.99% (17812/71284)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0becca2306c1fe2aa9d751e11f50db86c78e1b54_0becca2306c1fe2aa9d751e11f50db86c78e1b54/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49101 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0becca2306c1fe2aa9d751e11f50db86c78e1b54, data reload: false

------ Round 1 ----------------------------------
q1	18169	4345	4316	4316
q2	2071	154	145	145
q3	10267	1887	1946	1887
q4	10118	1256	1312	1256
q5	8418	3912	3931	3912
q6	233	124	124	124
q7	2046	1650	1566	1566
q8	9602	2744	2716	2716
q9	13562	9857	9779	9779
q10	8638	3520	3506	3506
q11	423	250	253	250
q12	468	294	302	294
q13	18351	3995	4043	3995
q14	351	338	323	323
q15	515	457	460	457
q16	534	462	464	462
q17	1124	970	996	970
q18	7379	7011	6807	6807
q19	1699	1627	1549	1549
q20	524	307	295	295
q21	4394	4090	4101	4090
q22	490	405	402	402
Total cold run time: 119376 ms
Total hot run time: 49101 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4317	4283	4306	4283
q2	325	222	222	222
q3	4187	4142	4125	4125
q4	2735	2747	2763	2747
q5	7175	7104	7046	7046
q6	236	118	117	117
q7	3229	2850	2798	2798
q8	4307	4454	4474	4454
q9	13660	13566	13429	13429
q10	4239	4224	4258	4224
q11	756	681	671	671
q12	1010	832	834	832
q13	6933	3773	3733	3733
q14	450	421	419	419
q15	503	475	455	455
q16	614	573	573	573
q17	3833	3754	3841	3754
q18	8777	8736	8807	8736
q19	1705	1695	1673	1673
q20	2429	2152	2116	2116
q21	8329	8346	8511	8346
q22	1014	944	951	944
Total cold run time: 80763 ms
Total hot run time: 75697 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212529 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0becca2306c1fe2aa9d751e11f50db86c78e1b54, data reload: false

query1	941	392	408	392
query2	6548	2303	2273	2273
query3	6917	211	207	207
query4	23543	21450	21550	21450
query5	19742	6491	6497	6491
query6	289	217	229	217
query7	4325	300	303	300
query8	271	264	278	264
query9	3050	2664	2585	2585
query10	473	297	314	297
query11	15667	15305	14993	14993
query12	129	75	73	73
query13	1045	436	432	432
query14	17222	13508	13620	13508
query15	367	219	224	219
query16	6022	281	261	261
query17	1735	934	908	908
query18	897	314	321	314
query19	210	153	148	148
query20	83	77	78	77
query21	195	98	92	92
query22	5235	5064	4998	4998
query23	34593	33523	33534	33523
query24	7808	6332	6340	6332
query25	539	429	424	424
query26	1317	159	162	159
query27	2402	298	292	292
query28	6060	2246	2225	2225
query29	2930	2582	2806	2582
query30	248	174	174	174
query31	976	757	765	757
query32	68	63	63	63
query33	463	254	257	254
query34	862	466	475	466
query35	1129	909	962	909
query36	1449	1134	1164	1134
query37	170	61	57	57
query38	3015	2931	2872	2872
query39	1373	1311	1337	1311
query40	307	94	93	93
query41	40	37	36	36
query42	85	89	94	89
query43	645	577	626	577
query44	1142	724	713	713
query45	249	227	224	224
query46	1229	958	963	958
query47	1919	1672	1666	1666
query48	499	417	400	400
query49	644	385	370	370
query50	861	612	609	609
query51	4797	4647	4646	4646
query52	92	82	87	82
query53	230	186	180	180
query54	2676	2499	2474	2474
query55	80	82	80	80
query56	226	220	207	207
query57	1206	1096	1085	1085
query58	223	203	221	203
query59	3741	3434	3233	3233
query60	219	198	204	198
query61	95	91	92	91
query62	858	503	511	503
query63	200	172	171	171
query64	3557	1595	1532	1532
query65	3640	3532	3554	3532
query66	783	437	422	422
query67	15642	15292	15410	15292
query68	6507	645	633	633
query69	463	260	278	260
query70	1487	1305	1370	1305
query71	377	315	310	310
query72	6683	4801	4780	4780
query73	749	322	321	321
query74	6230	5903	5901	5901
query75	4447	3719	3705	3705
query76	3360	1150	1157	1150
query77	537	253	260	253
query78	12468	11616	11671	11616
query79	8243	643	629	629
query80	3389	378	374	374
query81	511	233	242	233
query82	1662	93	95	93
query83	190	135	131	131
query84	256	69	69	69
query85	1400	307	314	307
query86	371	325	285	285
query87	3189	3053	3023	3023
query88	4885	2360	2333	2333
query89	411	296	271	271
query90	1777	207	218	207
query91	167	131	131	131
query92	56	53	53	53
query93	6202	533	567	533
query94	910	199	201	199
query95	2001	2010	1963	1963
query96	676	339	336	336
query97	6481	6407	6388	6388
query98	239	209	196	196
query99	2833	870	842	842
Total cold run time: 315350 ms
Total hot run time: 212529 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.87 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0becca2306c1fe2aa9d751e11f50db86c78e1b54, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.06	0.05
query4	1.77	0.06	0.06
query5	0.54	0.52	0.53
query6	1.24	0.66	0.61
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.51	0.49	0.49
query10	0.55	0.53	0.55
query11	0.12	0.09	0.09
query12	0.11	0.08	0.09
query13	0.63	0.62	0.62
query14	0.79	0.79	0.80
query15	0.78	0.76	0.76
query16	0.36	0.37	0.36
query17	0.99	1.01	1.01
query18	0.21	0.26	0.25
query19	1.95	1.84	1.85
query20	0.02	0.00	0.00
query21	15.45	0.54	0.54
query22	2.45	2.44	1.54
query23	17.20	0.99	1.10
query24	4.62	1.06	1.27
query25	0.38	0.08	0.05
query26	0.60	0.16	0.16
query27	0.05	0.04	0.04
query28	8.12	0.75	0.70
query29	12.75	2.38	2.33
query30	0.60	0.53	0.50
query31	2.82	0.40	0.37
query32	3.35	0.50	0.50
query33	3.09	3.09	3.06
query34	15.24	4.79	4.79
query35	4.88	4.87	4.83
query36	1.05	1.01	1.02
query37	0.06	0.04	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.15	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.98 s
Total hot run time: 30.87 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0becca2306c1fe2aa9d751e11f50db86c78e1b54 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.7 seconds inserted 10000000 Rows, about 483K ops/s

@xiaokang xiaokang merged commit 9829d44 into apache:branch-2.0 Sep 11, 2024
18 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
dev/2.0.15-merged kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@csun5285 @doris-robot @xiaokang