Skip to content

DataSink additions#7778

Merged
Dandandan merged 8 commits intoapache:mainfrom
Dandandan:file_sink_updates
Oct 10, 2023
Merged

DataSink additions#7778
Dandandan merged 8 commits intoapache:mainfrom
Dandandan:file_sink_updates

Conversation

@Dandandan
Copy link
Copy Markdown
Contributor

@Dandandan Dandandan commented Oct 9, 2023

Which issue does this PR close?

Closes #7779

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added the core Core DataFusion crate label Oct 9, 2023
@Dandandan Dandandan changed the title File sink additions DataSink additions Oct 9, 2023
/// resolved) for all available partitions, the set of metrics
/// should be complete. If this function is called prior to
/// `execute()` new metrics may appear in subsequent calls.
fn metrics(&self) -> Option<MetricsSet>;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I wonder if there is value in making some sort of End to End example for this feature (so we don't accidentally break it in the future0

Comment thread datafusion/physical-plan/src/insert.rs Outdated
Comment thread datafusion/physical-plan/src/insert.rs Outdated
Dandandan and others added 4 commits October 10, 2023 09:29

/// Returns the metrics of the underlying [DataSink]
pub fn metrics(&self) -> Option<MetricsSet> {
self.sink.metrics()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh this was the wrong place, it should be in impl ExecutionPlan for FileSinkExec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add as_any and metrics to DataSink trait

3 participants