Skip to content

[improvement](be) Avoid eager formatting in hot paths - #66366

Open
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:opt-lazy-hot-path-formatting
Open

[improvement](be) Avoid eager formatting in hot paths#66366
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:opt-lazy-hot-path-formatting

Conversation

@Mryange

@Mryange Mryange commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Related PR: #66363

Problem Summary: Successful delete bitmap cache checks repeatedly traversed both bitmaps to construct an unused error message, and nullable LARGEINT Arrow serialization formatted values before checking whether rows were null. Reuse the computed bitmap cardinalities, construct the mismatch message only on failure, and format LARGEINT values only for non-null rows.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: None

Related PR: apache#66363

Problem Summary: Successful delete bitmap cache checks repeatedly traversed both bitmaps to construct an unused error message, and nullable LARGEINT Arrow serialization formatted values before checking whether rows were null. Reuse the computed bitmap cardinalities, construct the mismatch message only on failure, and format LARGEINT values only for non-null rows.

### Release note

None

### Check List (For Author)

- Test: No need to test (control flow-only optimization; clang-format and git diff checks completed)
- Behavior changed: No
- Does this need documentation: No
@Mryange
Mryange requested a review from gavinchou as a code owner August 3, 2026 02:17
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange

Mryange commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 20.00% (2/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.21% (25671/43357)
Line Coverage 43.29% (257721/595270)
Region Coverage 38.97% (204261/524102)
Branch Coverage 40.32% (93231/231206)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 50.00% (5/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 75.88% (32086/42283)
Line Coverage 60.52% (357850/591273)
Region Coverage 57.07% (300134/525860)
Branch Coverage 58.51% (135309/231240)

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29190 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f2399cb0ac897ae40d351b8e5074de435f99b5ea, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17606	4012	3947	3947
q2	2034	331	207	207
q3	10237	1389	842	842
q4	4688	468	335	335
q5	7486	858	557	557
q6	198	178	141	141
q7	771	796	612	612
q8	10165	1570	1642	1570
q9	5829	4054	4054	4054
q10	6759	1615	1370	1370
q11	514	373	320	320
q12	758	585	461	461
q13	18143	3475	2834	2834
q14	266	258	243	243
q15	q16	738	734	657	657
q17	957	996	1034	996
q18	6707	5703	5479	5479
q19	1164	1239	1087	1087
q20	782	705	576	576
q21	5814	2750	2581	2581
q22	450	383	321	321
Total cold run time: 102066 ms
Total hot run time: 29190 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4912	4743	4862	4743
q2	301	339	318	318
q3	4931	5143	4759	4759
q4	2203	2319	1438	1438
q5	4691	4594	4565	4565
q6	250	182	130	130
q7	1851	1724	1537	1537
q8	2348	1917	1906	1906
q9	6808	6755	6699	6699
q10	4247	4216	3862	3862
q11	510	376	336	336
q12	743	700	496	496
q13	3061	3418	2782	2782
q14	267	277	246	246
q15	q16	662	682	600	600
q17	1229	1222	1199	1199
q18	7218	6820	6669	6669
q19	1096	1091	1051	1051
q20	2195	2209	1890	1890
q21	5379	4755	4587	4587
q22	516	484	409	409
Total cold run time: 55418 ms
Total hot run time: 50222 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 169431 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f2399cb0ac897ae40d351b8e5074de435f99b5ea, data reload: false

query5	4343	618	468	468
query6	468	239	205	205
query7	4835	616	361	361
query8	328	186	169	169
query9	8799	4056	4049	4049
query10	468	357	289	289
query11	5697	2171	2024	2024
query12	151	99	96	96
query13	1253	585	423	423
query14	6045	4698	4437	4437
query14_1	3899	3874	3843	3843
query15	214	216	179	179
query16	1054	496	464	464
query17	1154	720	569	569
query18	2627	448	337	337
query19	200	184	139	139
query20	106	98	99	98
query21	233	158	135	135
query22	12979	12978	12860	12860
query23	17197	16445	16022	16022
query23_1	16102	16275	16122	16122
query24	7507	1724	1222	1222
query24_1	1260	1251	1247	1247
query25	541	453	364	364
query26	1344	342	212	212
query27	2581	582	365	365
query28	4442	2027	2015	2015
query29	1060	590	466	466
query30	348	262	227	227
query31	1110	1077	961	961
query32	107	61	58	58
query33	542	317	251	251
query34	1173	1145	631	631
query35	748	749	631	631
query36	778	781	706	706
query37	155	99	91	91
query38	1820	1666	1602	1602
query39	821	835	802	802
query39_1	787	796	785	785
query40	261	159	141	141
query41	63	62	64	62
query42	98	88	89	88
query43	321	327	274	274
query44	1479	783	764	764
query45	187	176	164	164
query46	1053	1211	718	718
query47	1536	1515	1453	1453
query48	421	420	309	309
query49	589	424	301	301
query50	1045	433	342	342
query51	10317	10579	10774	10579
query52	84	90	74	74
query53	275	275	199	199
query54	295	231	220	220
query55	74	69	65	65
query56	292	302	287	287
query57	1025	1026	930	930
query58	299	254	257	254
query59	1559	1567	1403	1403
query60	324	302	247	247
query61	154	151	160	151
query62	400	320	268	268
query63	234	199	196	196
query64	2818	1036	852	852
query65	3904	3816	3864	3816
query66	1805	468	364	364
query67	28171	28109	27930	27930
query68	3286	1507	982	982
query69	409	295	267	267
query70	904	795	812	795
query71	382	333	334	333
query72	3210	2828	2524	2524
query73	852	759	435	435
query74	4621	4492	4287	4287
query75	2404	2359	2019	2019
query76	2355	1157	762	762
query77	352	387	295	295
query78	11071	11105	10475	10475
query79	1355	1181	754	754
query80	694	567	487	487
query81	455	339	282	282
query82	569	153	117	117
query83	420	336	307	307
query84	289	168	138	138
query85	1035	639	523	523
query86	322	233	216	216
query87	1792	1792	1778	1778
query88	3763	2808	2798	2798
query89	411	325	288	288
query90	1963	209	206	206
query91	206	190	164	164
query92	62	59	54	54
query93	1577	1543	934	934
query94	554	358	312	312
query95	792	507	477	477
query96	996	817	344	344
query97	2453	2443	2324	2324
query98	203	194	190	190
query99	725	726	607	607
Total cold run time: 254974 ms
Total hot run time: 169431 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f2399cb0ac897ae40d351b8e5074de435f99b5ea, data reload: false

query1	0.01	0.01	0.00
query2	0.09	0.05	0.05
query3	0.26	0.13	0.13
query4	1.61	0.14	0.14
query5	0.24	0.22	0.22
query6	1.16	0.83	0.81
query7	0.04	0.01	0.01
query8	0.06	0.04	0.04
query9	0.37	0.33	0.34
query10	0.55	0.53	0.54
query11	0.19	0.14	0.13
query12	0.18	0.14	0.14
query13	0.47	0.46	0.47
query14	1.00	0.99	1.00
query15	0.60	0.59	0.58
query16	0.30	0.32	0.31
query17	1.11	1.07	1.10
query18	0.20	0.21	0.19
query19	1.98	2.01	1.95
query20	0.02	0.01	0.01
query21	15.43	0.22	0.13
query22	4.80	0.10	0.05
query23	16.08	0.30	0.12
query24	2.98	0.40	0.35
query25	0.12	0.05	0.04
query26	0.72	0.21	0.14
query27	0.04	0.03	0.03
query28	3.59	0.77	0.35
query29	12.49	4.02	3.19
query30	0.27	0.16	0.15
query31	2.76	0.54	0.31
query32	3.21	0.59	0.49
query33	3.11	3.20	3.24
query34	15.59	3.88	3.27
query35	3.26	3.21	3.22
query36	0.55	0.43	0.42
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.08	0.04	0.03
query42	0.04	0.02	0.03
query43	0.04	0.04	0.04
Total cold run time: 95.95 s
Total hot run time: 23.91 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants