query result empty when a struct field name and a regular field name is same

[anlihust]

Describe the bug

When a struct field name is the same as a regular field name, and the struct is declared before the regular field, it can cause issues when the field name in where clause

To Reproduce

construct a parquet file ,write 2 records as follow

    +---------------------+----+--------+
     | struct              | id | name   |
     +---------------------+----+--------+
     | {id: 1, name: aaa1} | 1  | test01 |
     | {id: 2, name: aaa2} | 2  | test02 |
     +---------------------+----+--------+

when execute sql select * from base_table where name='test01' ,will got empty result
here is the bug demo
https://github.com/anlihust/datafusion_demo/blob/main/src/main.rs

Expected behavior

when execute sql select * from base_table where name='test01' ,get the test01 record

    +---------------------+----+--------+
     | struct              | id | name   |
     +---------------------+----+--------+
     | {id: 1, name: aaa1} | 1  | test01 |
     +---------------------+----+--------+

Additional context

No response

[alamb]

Thank you for the report @anlihust (and the reproducer)
This sounds similar to #8335 which we fixed recently -- maybe additional code somewhere incorrectly maps column names to parquet columns.

In particular using parquet_column is needed to find the correct file index

/// Lookups up the parquet column by name
///
/// Returns the parquet column index and the corresponding arrow field
pub(crate) fn parquet_column<'a>(
    parquet_schema: &SchemaDescriptor,
    arrow_schema: &'a Schema,
    name: &str,
) -> Option<(usize, &'a FieldRef)> {
    let (root_idx, field) = arrow_schema.fields.find(name)?;
    if field.data_type().is_nested() {
        // Nested fields are not supported and require non-trivial logic
        // to correctly walk the parquet schema accounting for the
        // logical type rules - <https://github.com/apache/parquet-format/blob/master/LogicalTypes.md>
        //
        // For example a ListArray could correspond to anything from 1 to 3 levels
        // in the parquet schema
        return None;
    }

    // This could be made more efficient (#TBD)
    let parquet_idx = (0..parquet_schema.columns().len())
        .find(|x| parquet_schema.get_column_root_idx(*x) == root_idx)?;
    Some((parquet_idx, field))
}

[manoj-inukolunu]

Hello @alamb , I attempted a fix for this . row_group_metadata.columns() is returning the fields in structs as top level columns but with an additional path . So the predicate can be applied on either the struct field or the actual column field .I have added a filter to only consider top level columns rather than the ones in struct.

[alamb]

Thanks @manoj-inukolunu -- I will check out #8848 shortly