Field Metadata Not Read

[treyhakanson]

Field metadata is not read when using @apache-arrow/ts@0.13.0. Example below also uses pyarrow==0.13.0

Steps to reproduce:

Adding metadata:

import pyarrow as pa
import pandas as pd

source = "sample.csv"
output = "sample.arrow"
df = pd.read_csv(source)
table = pa.Table.from_pandas(df)
schema = pa.schema([
    column.field.add_metadata({"foo": "bar"}))
    for column
    in table.columns
])
writer = pa.RecordBatchFileWriter(output, schema)
writer.write(table)
writer.close()

Reading field metadata using pyarrow:

source = "sample.arrow"
field = "foo"
reader = pa.RecordBatchFileReader(source)
reader.schema.field_by_name(field).metadata  # Correctly shows `{"foo": "bar"}`

Reading field metadata using @apache-arrow/ts:

import { Table, Field, Type } from "@apache-arrow/ts";

const url = "https://example.com/sample.arrow";
const buf = await fetch(url).then(res => res.arrayBuffer());
const table = Table.from([new Uint8Array(buf)]);
for (let field of table.schema.fields) {
    field.metadata;  // Incorrectly shows an empty map
}

[wesm]

Can you please open a JIRA issue?

[trxcllnt]

I checked in the debugger and it looks like flatbuffer's Field.customMetadataLength() call here returns zero on this file.

I know we haven't run flatc to generate the TS source from the format schemas in a while. Let me try that real quick to see if there's a mismatch there.

[trxcllnt]

@wesm @treyhakanson ah yes, regenerating the flatbuffers seems to have fixed the issue. I pushed a fix to the builders branch in PR #4476

[treyhakanson]

@trxcllnt @wesm thanks for checking this out guys! I filed a JIRA issue for the sake of staying organized; linked this issue there. Feel free to close this if that's the protocol.

[wesm]

Thanks