Skip to content

perf: Merge Precision in-place#21219

Open
AdamGS wants to merge 1 commit intoapache:mainfrom
AdamGS:adamg/stats-in-place-min-max
Open

perf: Merge Precision in-place#21219
AdamGS wants to merge 1 commit intoapache:mainfrom
AdamGS:adamg/stats-in-place-min-max

Conversation

@AdamGS
Copy link
Copy Markdown
Contributor

@AdamGS AdamGS commented Mar 28, 2026

Which issue does this PR close?

Follow up of #20768.

Rationale for this change

Precision::min/max allocates a lot of new ScalarValues, and it can be done in place.
While running the sql_planner benchmark, it seems like for clickbench Statistics::try_merge_iter is a significant part of the runtime, and this PR improves that part by about 20-25% locally.

What changes are included in this PR?

Introduces a couple of of new internal functions to calculate the min/max of a Precision in-place.

Are these changes tested?

Existing general tests, and a few new unit tests.

Are there any user-facing changes?

None

@github-actions github-actions bot added the common Related to common crate label Mar 28, 2026
@AdamGS
Copy link
Copy Markdown
Contributor Author

AdamGS commented Mar 28, 2026

Locally, the perf impact on the stats_merge benchmark are:

stats_merge/try_merge_iter/10parts_1cols
                        time:   [261.85 ns 263.02 ns 264.27 ns]
                        change: [−20.578% −20.212% −19.831%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking stats_merge/try_merge_iter/10parts_5cols: Collecting 100 samples in estimatedstats_merge/try_merge_iter/10parts_5cols
                        time:   [1.1278 µs 1.1310 µs 1.1346 µs]
                        change: [−18.151% −17.782% −17.421%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
Benchmarking stats_merge/try_merge_iter/10parts_20cols: Collecting 100 samples in estimatestats_merge/try_merge_iter/10parts_20cols
                        time:   [4.1735 µs 4.1785 µs 4.1843 µs]
                        change: [−18.384% −18.204% −18.034%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
Benchmarking stats_merge/try_merge_iter/100parts_1cols: Collecting 100 samples in estimatestats_merge/try_merge_iter/100parts_1cols
                        time:   [2.2418 µs 2.2478 µs 2.2547 µs]
                        change: [−22.768% −22.464% −22.154%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe
Benchmarking stats_merge/try_merge_iter/100parts_5cols: Collecting 100 samples in estimatestats_merge/try_merge_iter/100parts_5cols
                        time:   [11.596 µs 11.628 µs 11.665 µs]
                        change: [−14.560% −14.268% −13.925%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
Benchmarking stats_merge/try_merge_iter/100parts_20cols: Collecting 100 samples in estimatstats_merge/try_merge_iter/100parts_20cols
                        time:   [43.707 µs 43.818 µs 43.949 µs]
                        change: [−16.318% −16.016% −15.691%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
Benchmarking stats_merge/try_merge_iter/500parts_1cols: Collecting 100 samples in estimatestats_merge/try_merge_iter/500parts_1cols
                        time:   [11.530 µs 11.559 µs 11.591 µs]
                        change: [−23.035% −22.749% −22.473%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
Benchmarking stats_merge/try_merge_iter/500parts_5cols: Collecting 100 samples in estimatestats_merge/try_merge_iter/500parts_5cols
                        time:   [57.852 µs 58.011 µs 58.178 µs]
                        change: [−15.960% −15.588% −15.191%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
Benchmarking stats_merge/try_merge_iter/500parts_20cols: Collecting 100 samples in estimatstats_merge/try_merge_iter/500parts_20cols
                        time:   [220.48 µs 221.23 µs 222.16 µs]
                        change: [−13.692% −13.092% −12.313%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
  

Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
@AdamGS AdamGS force-pushed the adamg/stats-in-place-min-max branch from 8324b4d to e4f4dba Compare March 28, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant