Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix-2.0](column) Fix wrong has_null flag after filter_by_selector #40849

Merged
merged 1 commit into from
Sep 20, 2024

Conversation

zclllyybb
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

in master and branch-2.1 it fixed by refactor #40769. for 2.0 we pick #40756 as a minimal modification

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49772 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc0a120f89bc2d5de0de9259d22283900e9d32ae, data reload: false

------ Round 1 ----------------------------------
q1	17551	4433	4359	4359
q2	2037	185	150	150
q3	10234	1883	1953	1883
q4	10106	1259	1313	1259
q5	8510	4010	3932	3932
q6	235	123	122	122
q7	2023	1611	1566	1566
q8	9321	2771	2730	2730
q9	10193	10314	10422	10314
q10	8642	3553	3571	3553
q11	413	249	247	247
q12	464	294	297	294
q13	18336	3968	4065	3968
q14	351	325	338	325
q15	506	463	458	458
q16	543	459	464	459
q17	1153	972	956	956
q18	7330	6964	6846	6846
q19	1703	1568	1553	1553
q20	549	321	315	315
q21	4476	4138	4097	4097
q22	495	406	386	386
Total cold run time: 115171 ms
Total hot run time: 49772 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4368	4308	4311	4308
q2	325	231	226	226
q3	4175	4134	4161	4134
q4	2754	2753	2754	2753
q5	7177	7109	7152	7109
q6	241	119	118	118
q7	3305	2882	2858	2858
q8	4362	4479	4526	4479
q9	13754	13686	13556	13556
q10	4238	4297	4295	4295
q11	786	707	702	702
q12	1017	867	843	843
q13	6964	3778	3730	3730
q14	454	422	436	422
q15	497	464	455	455
q16	651	603	592	592
q17	3851	3886	3804	3804
q18	8788	8846	8796	8796
q19	1723	1716	1707	1707
q20	2405	2194	2084	2084
q21	8462	8493	8524	8493
q22	1022	901	922	901
Total cold run time: 81319 ms
Total hot run time: 76365 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.84% (8140/21513)
Line Coverage: 29.59% (66987/226408)
Region Coverage: 29.07% (34559/118880)
Branch Coverage: 24.97% (17804/71288)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fc0a120f89bc2d5de0de9259d22283900e9d32ae_fc0a120f89bc2d5de0de9259d22283900e9d32ae/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 213057 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc0a120f89bc2d5de0de9259d22283900e9d32ae, data reload: false

query1	932	414	379	379
query2	6531	2263	2103	2103
query3	6919	206	202	202
query4	23176	21648	21451	21451
query5	19760	6514	6463	6463
query6	290	219	233	219
query7	4324	294	301	294
query8	255	270	227	227
query9	3068	2633	2543	2543
query10	475	317	301	301
query11	16034	14996	15152	14996
query12	125	76	76	76
query13	1043	442	434	434
query14	16989	13282	13956	13282
query15	373	216	227	216
query16	6468	287	263	263
query17	1718	950	916	916
query18	890	314	314	314
query19	203	152	150	150
query20	81	75	75	75
query21	186	100	98	98
query22	5264	4996	4944	4944
query23	34123	33464	33380	33380
query24	7915	6278	6284	6278
query25	535	429	425	425
query26	1267	160	156	156
query27	2429	294	291	291
query28	6094	2265	2245	2245
query29	2821	2793	2798	2793
query30	242	171	169	169
query31	960	712	765	712
query32	70	61	63	61
query33	452	253	262	253
query34	881	482	475	475
query35	1130	913	956	913
query36	1243	1337	1262	1262
query37	179	60	61	60
query38	3087	2907	2867	2867
query39	1364	1326	1316	1316
query40	305	95	108	95
query41	39	35	36	35
query42	89	97	87	87
query43	614	627	589	589
query44	1229	712	717	712
query45	242	234	223	223
query46	1228	943	973	943
query47	1938	1697	1743	1697
query48	512	407	407	407
query49	668	372	374	372
query50	853	641	612	612
query51	4785	4658	4659	4658
query52	94	77	91	77
query53	232	188	183	183
query54	2672	2431	2443	2431
query55	85	90	80	80
query56	226	200	209	200
query57	1230	1004	1119	1004
query58	218	199	224	199
query59	3591	3308	3143	3143
query60	212	211	193	193
query61	97	93	105	93
query62	817	500	416	416
query63	199	175	176	175
query64	5827	1582	1493	1493
query65	3643	3523	3514	3514
query66	646	414	391	391
query67	16349	15423	15285	15285
query68	9428	631	640	631
query69	481	261	274	261
query70	1494	1375	1589	1375
query71	403	311	304	304
query72	6850	4872	4798	4798
query73	762	321	318	318
query74	6339	5876	5898	5876
query75	5197	3707	3663	3663
query76	5275	1161	1147	1147
query77	904	255	257	255
query78	12709	12908	14124	12908
query79	12819	621	630	621
query80	908	385	375	375
query81	495	236	234	234
query82	454	98	95	95
query83	189	133	132	132
query84	267	70	71	70
query85	787	313	312	312
query86	331	285	287	285
query87	3307	3026	3015	3015
query88	4936	2372	2353	2353
query89	358	281	301	281
query90	2635	213	213	213
query91	158	129	126	126
query92	65	51	53	51
query93	2643	539	552	539
query94	1243	206	207	206
query95	1976	1959	1945	1945
query96	668	341	331	331
query97	6462	6351	6398	6351
query98	218	198	201	198
query99	3077	879	893	879
Total cold run time: 322374 ms
Total hot run time: 213057 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.93 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc0a120f89bc2d5de0de9259d22283900e9d32ae, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.03
query3	0.25	0.05	0.04
query4	1.80	0.07	0.10
query5	0.53	0.52	0.52
query6	1.24	0.63	0.62
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.51	0.50	0.48
query10	0.53	0.53	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.77	0.80	0.77
query15	0.78	0.76	0.75
query16	0.37	0.36	0.36
query17	1.01	1.03	1.00
query18	0.21	0.25	0.26
query19	1.94	1.86	1.84
query20	0.02	0.01	0.01
query21	15.46	0.55	0.55
query22	2.30	2.72	1.61
query23	16.91	1.17	0.85
query24	6.70	1.30	1.14
query25	0.42	0.08	0.05
query26	0.72	0.17	0.15
query27	0.06	0.03	0.03
query28	6.01	0.74	0.76
query29	12.60	2.38	2.39
query30	0.57	0.59	0.53
query31	2.80	0.39	0.38
query32	3.38	0.50	0.49
query33	3.10	3.06	3.09
query34	15.26	4.77	4.77
query35	4.85	4.84	4.87
query36	1.07	1.02	1.01
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.54 s
Total hot run time: 30.93 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit fc0a120f89bc2d5de0de9259d22283900e9d32ae with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@HappenLee HappenLee merged commit f4674f6 into apache:branch-2.0 Sep 20, 2024
19 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants