refactor: Consolidate async UDF handling in physical planner #19791
+124
−160
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replace scattered try_plan_async_exprs() calls with two focused helpers: - maybe_wrap_async_exec(): for plain expressions - maybe_wrap_async_exec_with_names(): for projections This simplifies Projection, Filter, and Aggregation handling, and fixes apache#18149 by using the correct input schema for aggregates. Removes PlannedExprResult, PlanAsyncExpr enums and try_plan_async_exprs().
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #18150
Fixes #18149
Rationale for this change
The previous implementation called
try_plan_async_exprs()at three separate locations (projection, filter, aggregation), each with its own match arms handlingPlanAsyncExpr::SyncvsPlanAsyncExpr::Async. This caused:AggregateExec, causing expression/schema mismatchWhat changes are included in this PR?
Introduces two helper methods that consolidate async UDF handling:
maybe_wrap_async_exec()— for plain expressions (filter, aggregation)maybe_wrap_async_exec_with_names()— for projection expressions with aliasesBoth methods:
AsyncFuncExecThis makes each call site a simple function call instead of a multi-arm match.
Removed code
try_plan_async_exprs()methodPlannedExprResultenumPlanAsyncExprenumHow this fixes #18149
The bug occurred because aggregation expressions were rewritten to reference
__async_fn_0@N, butAggregateExecwas created with the original schema (which didn't include async columns).The fix uses
input_exec.schema()after wrapping withAsyncFuncExec, ensuring the aggregate sees a schema that includes the async columns.Are these changes tested?
Existing
async_udf.slttests cover async UDF usage in projections, filters, and aggregations. Physical plan outputs remain unchanged.Are there any user-facing changes?
No. This is an internal refactor with no public API changes.