Skip to content

Move datafusion_array_function specific rewrite rules like to datafusion_functions_array crate #9519

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

As we port over functions out of the datafusion-physical-expr crate, we have discovered several parts of ExprSimplif that rely on semantics of the array functions themselves

For example, the rewrite from array1 @> array2 to array_has_all(array1, array2)in rewrite_array_has_all_operator_to_func`
([source link](https://github.com/apache/arrow-datafusion/blob/5537572820977b38719e2253f601a159deef5bc6/datafusion/optimizer/src/analyzer/rewrite_expr.rs#L137

As @jayzhan22 noted on #9496 #9496 (comment), this rewrite is not correct if users supply their own implementation of array_has_all as that implementation may not be equal to DataFusion's array_has_all function. Thus the rule only makes sense when using the datafusion implementation

Describe the solution you'd like

Move the array_function specific rules such as rewrite_array_has_all_operator_to_func to the dataufusion_array_functions

Describe alternatives you've considered

  1. Create a a User Defined Analyzer pass (defined in datafusion_functions_array) with the desired rewrite
  2. Register the datafusion_function_array specific rewrite when the array functions in datafusion_array_function are registered (somewhere near here

Note you can register an analyzer rule via SessionState::add_analyzer_rule

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions