Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" #40997

Merged
merged 1 commit into from
Sep 20, 2024

Conversation

bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Sep 19, 2024

picks #40946

opt

update
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@bobhan1
Copy link
Contributor Author

bobhan1 commented Sep 19, 2024

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49283 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b, data reload: false

------ Round 1 ----------------------------------
q1	17716	4413	4373	4373
q2	2077	159	144	144
q3	10282	1942	1910	1910
q4	10404	1253	1324	1253
q5	8608	3871	3957	3871
q6	235	125	124	124
q7	2071	1631	1600	1600
q8	9500	2787	2748	2748
q9	10428	9837	9932	9837
q10	8645	3543	3536	3536
q11	413	250	251	250
q12	472	302	294	294
q13	18320	3982	4007	3982
q14	363	331	335	331
q15	519	446	453	446
q16	540	474	458	458
q17	1150	953	971	953
q18	7370	6930	6851	6851
q19	1682	1569	1477	1477
q20	546	317	313	313
q21	4498	4159	4136	4136
q22	486	400	396	396
Total cold run time: 116325 ms
Total hot run time: 49283 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4331	4327	4372	4327
q2	329	225	226	225
q3	4202	4145	4123	4123
q4	2764	2767	2746	2746
q5	7162	7115	7085	7085
q6	242	119	120	119
q7	3227	2868	2846	2846
q8	4382	4485	4551	4485
q9	13537	13625	13510	13510
q10	4279	4293	4222	4222
q11	736	675	685	675
q12	1020	833	827	827
q13	6738	3761	3705	3705
q14	457	424	411	411
q15	505	459	449	449
q16	643	607	606	606
q17	3782	3874	3857	3857
q18	8831	8852	8750	8750
q19	1718	1658	1639	1639
q20	2435	2071	2131	2071
q21	8576	8560	8484	8484
q22	1067	934	956	934
Total cold run time: 80963 ms
Total hot run time: 76096 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211577 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b, data reload: false

query1	934	387	436	387
query2	6533	2168	2116	2116
query3	6921	207	200	200
query4	22944	21713	21433	21433
query5	19734	6522	6559	6522
query6	285	224	240	224
query7	4327	299	296	296
query8	269	259	252	252
query9	3117	2692	2620	2620
query10	461	322	322	322
query11	16137	14915	14976	14915
query12	130	72	79	72
query13	1039	448	454	448
query14	17972	13704	13178	13178
query15	365	222	225	222
query16	6486	282	266	266
query17	1914	943	919	919
query18	897	321	311	311
query19	208	152	148	148
query20	82	75	80	75
query21	194	97	96	96
query22	5195	5034	4941	4941
query23	34418	33511	33497	33497
query24	6882	6281	6270	6270
query25	547	435	435	435
query26	806	167	161	161
query27	2254	300	295	295
query28	6118	2307	2244	2244
query29	2926	2826	2815	2815
query30	239	169	168	168
query31	965	739	735	735
query32	71	65	56	56
query33	445	274	273	273
query34	856	487	480	480
query35	1128	933	968	933
query36	1217	1244	1152	1152
query37	92	59	62	59
query38	3034	2948	2914	2914
query39	1380	1325	1314	1314
query40	210	97	102	97
query41	40	43	37	37
query42	93	86	85	85
query43	627	680	565	565
query44	1185	712	719	712
query45	243	233	227	227
query46	1234	959	965	959
query47	1911	1694	1847	1694
query48	515	421	403	403
query49	615	371	391	371
query50	867	627	624	624
query51	4766	4680	4639	4639
query52	89	79	72	72
query53	233	187	182	182
query54	2665	2473	2464	2464
query55	86	81	85	81
query56	214	208	201	201
query57	1259	1155	1173	1155
query58	224	220	213	213
query59	3476	3254	3415	3254
query60	222	209	209	209
query61	100	95	94	94
query62	833	479	514	479
query63	215	179	176	176
query64	3225	1609	1519	1519
query65	3611	3513	3562	3513
query66	727	414	425	414
query67	15825	14852	16646	14852
query68	9629	655	648	648
query69	512	283	292	283
query70	1524	1474	1291	1291
query71	414	318	321	318
query72	6836	4625	4957	4625
query73	755	321	326	321
query74	6368	5839	5923	5839
query75	5323	3714	3685	3685
query76	5487	1111	1189	1111
query77	940	257	259	257
query78	12471	11783	11608	11608
query79	8020	626	638	626
query80	1566	392	385	385
query81	507	243	244	243
query82	1642	98	98	98
query83	164	131	133	131
query84	253	72	70	70
query85	904	325	310	310
query86	341	290	294	290
query87	3258	2973	3026	2973
query88	4712	2298	2292	2292
query89	451	269	278	269
query90	1968	212	220	212
query91	165	130	139	130
query92	54	50	52	50
query93	6310	581	608	581
query94	696	212	208	208
query95	1973	1926	2074	1926
query96	642	326	324	324
query97	6542	6372	6423	6372
query98	219	198	201	198
query99	3029	848	926	848
Total cold run time: 318507 ms
Total hot run time: 211577 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.29 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.03
query3	0.24	0.04	0.05
query4	1.82	0.07	0.07
query5	0.54	0.53	0.52
query6	1.25	0.65	0.60
query7	0.01	0.01	0.02
query8	0.05	0.03	0.02
query9	0.53	0.49	0.48
query10	0.53	0.54	0.54
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.61	0.61	0.62
query14	0.78	0.78	0.79
query15	0.78	0.76	0.78
query16	0.38	0.40	0.38
query17	0.99	1.03	1.03
query18	0.25	0.24	0.23
query19	1.91	1.84	1.87
query20	0.02	0.01	0.01
query21	15.45	0.63	0.55
query22	2.05	2.38	2.08
query23	17.48	1.09	0.79
query24	5.88	1.13	1.04
query25	0.36	0.08	0.06
query26	0.68	0.15	0.15
query27	0.04	0.05	0.03
query28	7.09	0.74	0.71
query29	12.71	2.39	2.35
query30	0.57	0.53	0.59
query31	2.81	0.38	0.38
query32	3.38	0.52	0.49
query33	3.08	3.08	3.08
query34	15.28	4.81	4.79
query35	4.87	4.84	4.85
query36	1.08	1.03	1.01
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.15
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.2 s
Total hot run time: 31.29 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.84% (8141/21513)
Line Coverage: 29.59% (66998/226423)
Region Coverage: 29.08% (34566/118884)
Branch Coverage: 24.98% (17810/71294)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1dc16ec281a2ce7d86c87cd968ee7173a405942b_1dc16ec281a2ce7d86c87cd968ee7173a405942b/report/index.html

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

@dataroaring dataroaring merged commit 761a8eb into apache:branch-2.0 Sep 20, 2024
21 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants