Skip to content

[chore](float) Unify floating number to string - #66375

Open
jacktengg wants to merge 2 commits into
apache:masterfrom
jacktengg:fix-260803
Open

[chore](float) Unify floating number to string#66375
jacktengg wants to merge 2 commits into
apache:masterfrom
jacktengg:fix-260803

Conversation

@jacktengg

@jacktengg jacktengg commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #65302

Problem Summary:

Floating number to string is now handled correctly by CastToString::from_number in PR #65609, special handing in DataTypeNumberSerDe::to_olap_string is not necessary now.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Floating number to string is now handled correctly by CastToString::from_number in PR apache#65609, special handing in DataTypeNumberSerDe<T>::to_olap_string is not necessary now.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@jacktengg

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28865 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 19e6613119e1a17849264038f7635094ce31d960, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17744	3975	3915	3915
q2	2062	327	206	206
q3	10280	1407	787	787
q4	4681	462	331	331
q5	7487	852	558	558
q6	180	174	143	143
q7	746	804	600	600
q8	9423	1539	1498	1498
q9	5366	4016	4036	4016
q10	6764	1628	1352	1352
q11	486	346	339	339
q12	760	581	455	455
q13	18076	3281	2740	2740
q14	258	255	244	244
q15	q16	734	715	658	658
q17	1035	955	930	930
q18	6915	5598	5539	5539
q19	1327	1262	1119	1119
q20	823	684	610	610
q21	5956	2587	2520	2520
q22	440	357	305	305
Total cold run time: 101543 ms
Total hot run time: 28865 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4282	4173	4259	4173
q2	279	312	211	211
q3	4554	4960	4393	4393
q4	2392	2293	1436	1436
q5	4286	4117	4170	4117
q6	225	167	129	129
q7	1718	1594	1396	1396
q8	2920	2200	2092	2092
q9	7486	7574	7620	7574
q10	4259	4327	3862	3862
q11	556	421	383	383
q12	712	720	529	529
q13	3288	3482	3079	3079
q14	292	293	271	271
q15	q16	687	747	618	618
q17	1311	1300	1305	1300
q18	7706	7263	7225	7225
q19	1134	1106	1110	1106
q20	2234	2228	1937	1937
q21	5330	4781	4511	4511
q22	511	477	408	408
Total cold run time: 56162 ms
Total hot run time: 50750 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 170306 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 19e6613119e1a17849264038f7635094ce31d960, data reload: false

query5	4326	628	451	451
query6	452	222	220	220
query7	4914	595	364	364
query8	339	185	170	170
query9	8849	4122	4088	4088
query10	534	365	304	304
query11	5843	2193	1981	1981
query12	159	100	96	96
query13	1255	612	424	424
query14	6033	4685	4364	4364
query14_1	3779	3755	3755	3755
query15	204	207	174	174
query16	1028	450	427	427
query17	909	671	561	561
query18	2440	478	341	341
query19	205	193	151	151
query20	112	105	106	105
query21	231	163	137	137
query22	13102	12943	12783	12783
query23	17385	16387	15987	15987
query23_1	16090	16164	16134	16134
query24	7610	1723	1250	1250
query24_1	1265	1257	1214	1214
query25	560	453	389	389
query26	1326	356	211	211
query27	2626	552	401	401
query28	4511	2053	2049	2049
query29	1100	619	487	487
query30	340	269	228	228
query31	1118	1070	962	962
query32	135	59	60	59
query33	502	321	260	260
query34	1180	1141	654	654
query35	757	740	650	650
query36	771	767	719	719
query37	150	113	104	104
query38	1853	1645	1612	1612
query39	819	823	800	800
query39_1	797	791	796	791
query40	245	163	137	137
query41	66	64	64	64
query42	98	92	92	92
query43	321	319	280	280
query44	1421	804	780	780
query45	187	177	165	165
query46	1080	1194	703	703
query47	1537	1542	1469	1469
query48	411	382	300	300
query49	585	416	317	317
query50	1112	430	320	320
query51	10330	10537	10645	10537
query52	87	91	76	76
query53	269	282	208	208
query54	290	231	223	223
query55	76	76	68	68
query56	306	305	300	300
query57	997	993	925	925
query58	266	264	268	264
query59	1587	1592	1381	1381
query60	306	303	245	245
query61	155	187	151	151
query62	401	314	267	267
query63	231	196	196	196
query64	2866	1022	856	856
query65	3894	3828	3800	3800
query66	1842	475	367	367
query67	28171	28204	28114	28114
query68	3151	1603	1038	1038
query69	395	310	260	260
query70	869	787	792	787
query71	375	342	320	320
query72	2974	2634	2561	2561
query73	886	759	440	440
query74	4610	4499	4281	4281
query75	2374	2350	2028	2028
query76	2326	1137	775	775
query77	357	387	304	304
query78	11158	11079	10630	10630
query79	1378	1187	738	738
query80	693	560	491	491
query81	458	341	287	287
query82	581	153	115	115
query83	418	323	309	309
query84	342	162	137	137
query85	991	691	624	624
query86	325	237	216	216
query87	1795	1813	1717	1717
query88	3746	2843	2833	2833
query89	404	333	280	280
query90	1882	209	200	200
query91	202	194	170	170
query92	64	60	54	54
query93	1646	1550	1052	1052
query94	537	340	301	301
query95	781	607	472	472
query96	1065	805	355	355
query97	2423	2460	2397	2397
query98	200	199	193	193
query99	712	723	600	600
Total cold run time: 255048 ms
Total hot run time: 170306 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 19e6613119e1a17849264038f7635094ce31d960, data reload: false

query1	0.01	0.01	0.00
query2	0.10	0.06	0.05
query3	0.26	0.14	0.14
query4	1.61	0.14	0.14
query5	0.23	0.22	0.22
query6	1.16	0.78	0.82
query7	0.04	0.01	0.01
query8	0.06	0.04	0.04
query9	0.38	0.30	0.34
query10	0.57	0.58	0.55
query11	0.20	0.13	0.14
query12	0.18	0.14	0.13
query13	0.47	0.48	0.48
query14	1.02	0.99	0.99
query15	0.61	0.59	0.58
query16	0.33	0.33	0.33
query17	1.13	1.09	1.08
query18	0.22	0.20	0.20
query19	2.05	1.93	1.97
query20	0.02	0.02	0.01
query21	15.44	0.22	0.13
query22	4.74	0.06	0.05
query23	16.15	0.32	0.13
query24	2.97	0.42	0.33
query25	0.13	0.05	0.05
query26	0.73	0.22	0.14
query27	0.04	0.03	0.04
query28	3.55	0.79	0.35
query29	12.50	4.06	3.17
query30	0.28	0.15	0.16
query31	2.77	0.55	0.33
query32	3.23	0.58	0.49
query33	3.12	3.16	3.20
query34	15.52	3.92	3.29
query35	3.25	3.23	3.24
query36	0.55	0.43	0.40
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.16	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 96.13 s
Total hot run time: 23.89 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.27% (25709/43374)
Line Coverage 43.34% (258105/595509)
Region Coverage 39.04% (204689/524332)
Branch Coverage 40.38% (93381/231278)

### What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary: The percentile approx array regression test expected floating-point strings produced before floating-point serialization was unified. The unified shortest round-trip formatting changes the final digits for two approximate percentile values, causing the query result comparison to fail. Update the expected result to match the new formatting.

### Release note

None

### Check List (For Author)

- Test: Regression test attempted; the configured FE/BE cluster does not contain percentile_approx_array
- Behavior changed: No
- Does this need documentation: No
@jacktengg

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29296 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0fbff966af2628ce4bc91604960e9e52ec0a10bd, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17673	4092	4081	4081
q2	2011	318	214	214
q3	10282	1354	832	832
q4	4679	462	335	335
q5	7493	848	559	559
q6	180	181	142	142
q7	746	799	606	606
q8	9331	1581	1541	1541
q9	5237	4056	4049	4049
q10	6737	1650	1343	1343
q11	497	359	331	331
q12	761	568	440	440
q13	18091	3306	2745	2745
q14	271	264	250	250
q15	q16	735	729	662	662
q17	960	956	1032	956
q18	7079	5784	5438	5438
q19	1292	1209	1080	1080
q20	790	709	597	597
q21	6751	2802	2755	2755
q22	458	381	340	340
Total cold run time: 102054 ms
Total hot run time: 29296 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5147	4757	4779	4757
q2	294	332	215	215
q3	4887	5274	4689	4689
q4	2185	2272	1410	1410
q5	4466	4683	4446	4446
q6	244	194	136	136
q7	1832	1830	1475	1475
q8	2317	2114	2053	2053
q9	7269	6941	6685	6685
q10	4225	4263	3774	3774
q11	511	389	340	340
q12	732	710	491	491
q13	2976	3265	2814	2814
q14	272	275	246	246
q15	q16	680	682	603	603
q17	1235	1208	1205	1205
q18	7181	6983	6786	6786
q19	1088	1063	1057	1057
q20	2229	2188	1907	1907
q21	5293	4719	4610	4610
q22	518	457	409	409
Total cold run time: 55581 ms
Total hot run time: 50108 ms

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.27% (25709/43374)
Line Coverage 43.34% (258093/595509)
Region Coverage 39.01% (204568/524332)
Branch Coverage 40.37% (93378/231278)

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 171705 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0fbff966af2628ce4bc91604960e9e52ec0a10bd, data reload: false

query5	4335	653	506	506
query6	489	241	220	220
query7	4879	624	349	349
query8	366	211	190	190
query9	8814	4108	4089	4089
query10	496	364	327	327
query11	5888	2215	2062	2062
query12	165	106	111	106
query13	1323	626	465	465
query14	6152	4802	4480	4480
query14_1	3860	3905	3930	3905
query15	224	211	183	183
query16	1019	491	468	468
query17	995	760	610	610
query18	2490	502	385	385
query19	230	215	171	171
query20	114	110	108	108
query21	238	178	147	147
query22	13009	13004	12771	12771
query23	17335	16425	15982	15982
query23_1	16199	16116	16176	16116
query24	7497	1756	1295	1295
query24_1	1286	1263	1269	1263
query25	600	498	440	440
query26	1320	359	223	223
query27	2608	618	375	375
query28	4484	2094	2086	2086
query29	1131	679	521	521
query30	353	267	238	238
query31	1128	1098	986	986
query32	131	68	69	68
query33	594	352	286	286
query34	1180	1134	675	675
query35	766	785	651	651
query36	824	786	727	727
query37	159	107	102	102
query38	1846	1668	1623	1623
query39	861	814	803	803
query39_1	805	812	792	792
query40	257	172	165	165
query41	88	86	87	86
query42	110	100	101	100
query43	335	336	288	288
query44	1518	793	794	793
query45	204	184	174	174
query46	1037	1174	736	736
query47	1528	1531	1457	1457
query48	431	428	314	314
query49	612	418	313	313
query50	1037	445	360	360
query51	10578	10214	10377	10214
query52	88	91	79	79
query53	268	283	206	206
query54	310	262	240	240
query55	78	76	70	70
query56	320	305	311	305
query57	1042	983	940	940
query58	317	282	270	270
query59	1561	1673	1402	1402
query60	321	297	275	275
query61	194	184	189	184
query62	402	325	278	278
query63	245	213	208	208
query64	3112	1269	1075	1075
query65	3903	3829	3815	3815
query66	1879	506	401	401
query67	28181	28129	28045	28045
query68	3207	1629	982	982
query69	421	315	284	284
query70	899	778	779	778
query71	380	357	325	325
query72	3298	3000	2572	2572
query73	843	728	420	420
query74	4663	4533	4335	4335
query75	2397	2350	2013	2013
query76	2354	1148	776	776
query77	376	384	298	298
query78	11153	11206	10637	10637
query79	1438	1120	782	782
query80	1387	585	512	512
query81	568	339	286	286
query82	930	154	119	119
query83	428	351	313	313
query84	344	170	142	142
query85	1095	741	651	651
query86	440	242	229	229
query87	1805	1802	1705	1705
query88	3760	2870	2824	2824
query89	425	332	293	293
query90	1912	196	203	196
query91	227	212	191	191
query92	69	66	59	59
query93	1642	1616	935	935
query94	814	381	331	331
query95	816	599	500	500
query96	1097	792	334	334
query97	2448	2473	2349	2349
query98	213	204	199	199
query99	734	729	620	620
Total cold run time: 259060 ms
Total hot run time: 171705 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0fbff966af2628ce4bc91604960e9e52ec0a10bd, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.06	0.05
query3	0.26	0.14	0.13
query4	1.61	0.14	0.14
query5	0.24	0.22	0.22
query6	1.16	0.80	0.82
query7	0.03	0.01	0.01
query8	0.05	0.03	0.04
query9	0.38	0.31	0.30
query10	0.56	0.53	0.55
query11	0.18	0.14	0.14
query12	0.18	0.15	0.15
query13	0.47	0.48	0.46
query14	1.01	0.99	0.99
query15	0.60	0.59	0.58
query16	0.33	0.31	0.32
query17	1.11	1.09	1.15
query18	0.21	0.21	0.21
query19	2.11	1.97	1.96
query20	0.02	0.02	0.01
query21	15.49	0.21	0.13
query22	5.00	0.06	0.05
query23	16.14	0.31	0.12
query24	2.90	0.43	0.33
query25	0.11	0.06	0.04
query26	0.72	0.22	0.16
query27	0.05	0.03	0.04
query28	3.53	0.82	0.36
query29	12.62	3.99	3.19
query30	0.27	0.15	0.14
query31	2.78	0.57	0.32
query32	3.22	0.60	0.49
query33	3.17	3.19	3.20
query34	15.67	3.97	3.32
query35	3.25	3.21	3.24
query36	0.56	0.41	0.43
query37	0.11	0.06	0.06
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.16	0.15
query41	0.10	0.03	0.03
query42	0.04	0.03	0.02
query43	0.05	0.03	0.04
Total cold run time: 96.67 s
Total hot run time: 23.97 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 75.74% (32039/42300)
Line Coverage 60.42% (357413/591512)
Region Coverage 57.02% (299964/526090)
Branch Coverage 58.35% (134979/231312)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants