Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](statistics)Support convert ip type to double for column stats. #48823

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Jibing-Li
Copy link
Contributor

What problem does this PR solve?

Support convert ip type to double for column stats so than we can use the collected min/max value for IPv4/IPv6 type.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Jibing-Li Jibing-Li marked this pull request as ready for review March 7, 2025 08:42
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33125 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6494bf6c8f50dc6e147581b41cb3df960ff371e4, data reload: false

------ Round 1 ----------------------------------
q1	17568	5169	5078	5078
q2	2044	290	171	171
q3	10421	1369	703	703
q4	10230	1055	525	525
q5	7565	2414	2349	2349
q6	196	168	140	140
q7	908	746	606	606
q8	9292	1283	1085	1085
q9	5206	4854	4860	4854
q10	6801	2319	1880	1880
q11	457	285	260	260
q12	347	354	217	217
q13	17756	3664	3588	3588
q14	227	221	206	206
q15	531	501	473	473
q16	624	621	592	592
q17	584	852	334	334
q18	7015	6512	6417	6417
q19	1221	946	540	540
q20	325	323	193	193
q21	2837	2180	1941	1941
q22	1046	1027	973	973
Total cold run time: 103201 ms
Total hot run time: 33125 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5121	5150	5138	5138
q2	230	326	228	228
q3	2181	2727	2250	2250
q4	1421	1828	1387	1387
q5	4284	4117	4152	4117
q6	205	165	125	125
q7	1884	1984	1747	1747
q8	2628	2605	2572	2572
q9	7266	7150	7167	7150
q10	2996	3217	2798	2798
q11	578	513	501	501
q12	727	763	611	611
q13	3457	3892	3331	3331
q14	277	290	278	278
q15	513	482	476	476
q16	651	670	646	646
q17	1127	1595	1341	1341
q18	7778	7616	7286	7286
q19	817	846	1050	846
q20	1964	2039	1858	1858
q21	5405	4898	4900	4898
q22	1052	1017	973	973
Total cold run time: 52562 ms
Total hot run time: 50557 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185599 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6494bf6c8f50dc6e147581b41cb3df960ff371e4, data reload: false

query1	976	380	381	380
query2	6549	1908	1878	1878
query3	6800	230	224	224
query4	26644	23698	23057	23057
query5	4329	657	479	479
query6	318	204	193	193
query7	4603	514	293	293
query8	290	244	231	231
query9	8581	2532	2520	2520
query10	494	312	249	249
query11	15867	15123	14981	14981
query12	161	118	104	104
query13	1654	534	392	392
query14	9414	6265	6210	6210
query15	209	189	163	163
query16	7219	600	482	482
query17	1172	734	544	544
query18	1956	395	299	299
query19	193	181	159	159
query20	119	115	115	115
query21	205	120	100	100
query22	4197	4278	4236	4236
query23	34048	33066	32873	32873
query24	8117	2402	2428	2402
query25	531	477	393	393
query26	1238	272	158	158
query27	2596	492	328	328
query28	4301	2400	2403	2400
query29	753	567	431	431
query30	285	219	191	191
query31	944	838	816	816
query32	79	65	64	64
query33	562	363	301	301
query34	773	849	508	508
query35	795	828	732	732
query36	988	1001	891	891
query37	114	97	78	78
query38	4390	4182	4017	4017
query39	1480	1404	1410	1404
query40	210	117	101	101
query41	54	51	51	51
query42	122	108	105	105
query43	501	509	464	464
query44	1289	777	792	777
query45	177	172	169	169
query46	839	1036	618	618
query47	1760	1769	1699	1699
query48	376	416	295	295
query49	772	514	425	425
query50	684	730	412	412
query51	4230	4174	4173	4173
query52	108	112	93	93
query53	232	256	186	186
query54	479	487	414	414
query55	87	81	83	81
query56	285	269	259	259
query57	1123	1155	1076	1076
query58	251	239	240	239
query59	2698	2742	2758	2742
query60	287	276	261	261
query61	124	122	120	120
query62	817	714	703	703
query63	250	194	195	194
query64	4338	1062	673	673
query65	4462	4348	4384	4348
query66	1139	417	328	328
query67	15827	15582	15224	15224
query68	7876	874	519	519
query69	463	300	268	268
query70	1199	1112	1086	1086
query71	418	282	264	264
query72	5548	3634	3790	3634
query73	744	752	353	353
query74	8977	9130	9161	9130
query75	3384	3217	2729	2729
query76	3301	1198	764	764
query77	547	387	281	281
query78	10019	10184	9370	9370
query79	2107	825	589	589
query80	607	527	455	455
query81	498	264	225	225
query82	203	127	95	95
query83	176	173	163	163
query84	248	94	85	85
query85	740	362	310	310
query86	375	315	279	279
query87	4401	4607	4339	4339
query88	3892	2221	2227	2221
query89	388	324	290	290
query90	1970	206	207	206
query91	138	143	110	110
query92	76	62	60	60
query93	1804	1063	576	576
query94	658	397	308	308
query95	346	268	262	262
query96	478	570	286	286
query97	3289	3414	3354	3354
query98	235	204	206	204
query99	1308	1405	1278	1278
Total cold run time: 273878 ms
Total hot run time: 185599 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.8 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6494bf6c8f50dc6e147581b41cb3df960ff371e4, data reload: false

query1	0.04	0.03	0.04
query2	0.06	0.04	0.03
query3	0.24	0.06	0.07
query4	1.61	0.11	0.10
query5	0.57	0.54	0.56
query6	1.20	0.73	0.73
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.52
query10	0.57	0.61	0.57
query11	0.17	0.11	0.11
query12	0.14	0.11	0.11
query13	0.63	0.60	0.59
query14	2.67	2.81	2.69
query15	0.92	0.86	0.85
query16	0.39	0.39	0.38
query17	1.05	1.00	1.02
query18	0.21	0.19	0.20
query19	1.86	1.80	2.02
query20	0.02	0.01	0.02
query21	15.35	0.90	0.54
query22	0.74	1.31	0.72
query23	14.77	1.33	0.62
query24	6.90	1.91	1.78
query25	0.51	0.21	0.15
query26	0.60	0.16	0.13
query27	0.04	0.06	0.05
query28	10.87	0.78	0.43
query29	12.56	3.98	3.29
query30	0.25	0.09	0.07
query31	2.82	0.58	0.40
query32	3.22	0.55	0.46
query33	3.07	2.98	3.04
query34	15.60	5.16	4.51
query35	4.57	4.55	4.52
query36	0.66	0.50	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.13	0.12
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.05 s
Total hot run time: 31.8 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 7, 2025
Copy link
Contributor

github-actions bot commented Mar 7, 2025

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Mar 7, 2025

PR approved by anyone and no changes requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants