[Python] Allow to create field reference to nested field

[asfimport]

Currently in the Python layer we have ds.field(name) for referencing a field eg when creating a filter expression.

On the C++ side, the FieldRef has the ability to reference nested fields (eg a child of a struct), for example constructed with FieldRef::FromDotPath.

We should think about how to enable users to reference nested fields like this in the Python API.

Reporter: Joris Van den Bossche / @jorisvandenbossche
Assignee: Alenka Frim / @AlenkaF

Related issues:

• [Python] Selective projection of struct fields errors with use_legacy_dataset = False (blocks)
• [Python] parquet.read_table nested fields in columns does not work for use_legacy_dataset=False (blocks)
• [C++] Support nested FieldRefs (is blocked by)

PRs and other links:

• GitHub Pull Request #12863
• GitHub Pull Request #9799

_{Note: This issue was originally created as ARROW-11259. Please see the migration documentation for further details.}

[asfimport]

Ben Kietzman / @bkietz:
Related: currently field_refs can only extract a field from the scanned dataset. It'd be helpful if they could also extract a field from an Expression

[asfimport]

Ben Kietzman / @bkietz:
Note that this will require some added functionality for FieldRef, since some functions assume only top level fields will be materialized. For example IpcReadOptions::included_fields is a set of indices of top level fields. So we'll need to add functionality to extract the top level field from a nested FieldRef

[asfimport]

Neal Richardson / @nealrichardson:
Agree that it would be helpful (possibly necessary) to be able to extract a field from an Expression more generally.

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 12863
#12863

[asfimport]

Antoine Pitrou / @pitrou:
Oops, sorry, I hadn't checked the assignee field on this issue.

[asfimport]

Alenka Frim / @AlenkaF:
No problem at all! =)