ScalarValue::iter_to_array doesn't support Duration and Union types

[joroKr21]

Describe the bug

This is causing troubles for one of our user-defined aggregation functions. I don't know if there's an easier way to reproduce the issue. But it looks to me like it should be straightforward to support those data types:

• Duration is a primitive
• Union is similar to Struct and List - a single element array

To Reproduce

I am not sure. I could post some code with the UDAF reproduction but maybe someone knows a way to trigger the issue directly with a query?

Edit - failing query:

SELECT tag, array_agg(millis - arrow_cast(secs, 'Timestamp(Millisecond, None)')) FROM t GROUP BY tag ORDER BY tag;

Expected behavior

ScalarValue::iter_to_array to support all data types (maybe except the views) but I would settle for those two.

Additional context

I'm preparing a PR but I need ideas for testing

[jayzhan211]

For duration, I think this test is enough.

statement error
CREATE TABLE table as VALUES
          (arrow_cast(123456, 'Duration(Second)')),
          (arrow_cast(234567, 'Duration(Second)'))
----
DataFusion error: Internal error: Unsupported creation of Duration(Second) array from ScalarValue Some(DurationSecond("123456")).
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

https://github.com/apache/arrow-datafusion/blob/19356b26f515149f96f9b6296975a77ac7260149/datafusion/common/src/scalar/mod.rs#L1671-L1689

For Union, we need to support casting first before testing it in slt file (ideal in the long term). But, we can also test it in rust test for now.