Skip to content

correctly build nullability information in values exec#7694

Closed
matthewgapp wants to merge 4 commits intoapache:mainfrom
matthewgapp:matt/fix/schema-mismatch-when-inserting
Closed

correctly build nullability information in values exec#7694
matthewgapp wants to merge 4 commits intoapache:mainfrom
matthewgapp:matt/fix/schema-mismatch-when-inserting

Conversation

@matthewgapp
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #7693.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added the logical-expr Logical plan and expressions label Sep 29, 2023
// we have this single row, null, typed batch as a placeholder to satisfy evaluation argument
let batch = RecordBatch::try_new(
schema.clone(),
// the schema we're using might have non-nullable fields, so we need to make them nullable
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the issue appears to be that the schema in the MemoryTable doesn't match the schema of all its batches, what would you think about making this change in MemoryTable insert logic itself.

For example, perhaps after insert it could replace all batches with its own schema (perhaps also ensuring the nullability is correct).

https://github.com/apache/arrow-datafusion/blob/2ffda2a9a893455e55cd773d9dd4f426a61d8cd3/datafusion/core/src/datasource/memory.rs#L262

@github-actions
Copy link
Copy Markdown

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.

@github-actions github-actions Bot added the Stale PR has not had any activity for some time label Apr 27, 2024
@github-actions github-actions Bot closed this May 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions Stale PR has not had any activity for some time

Projects

None yet

2 participants