Skip to content

KPO write_logs does not need to use complicated pod logs reader#36228

Closed
dstandish wants to merge 1 commit into
apache:mainfrom
astronomer:simplify-write-logs-logic
Closed

KPO write_logs does not need to use complicated pod logs reader#36228
dstandish wants to merge 1 commit into
apache:mainfrom
astronomer:simplify-write-logs-logic

Conversation

@dstandish

Copy link
Copy Markdown
Contributor

Method write_logs was using pod manager method read_pod_logs which has some rather complicated logic that it doesn't need, because it does not "follow" the logs and because by the time we come out of deferral, the pod is already done.

By reducing usage of read_pod_logs, it makes it easier to hopefully refactor and simplify some of our KPO log fetching logic.

I also make the metheod explicitly private which it should have been anyway.

@boring-cyborg boring-cyborg Bot added area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels Dec 14, 2023
Comment on lines 706 to 654

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this considered a breaking change?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to consider it breaking. This is a method that realistically no one would ever override. It's used in one place. I think of it as "bugfix" that it wasn't marked private.

Really this also connects to something we've discussed before. That our operators should not be published as "base classes" to be extended. I think they should be end products whose behavior is governed by semver but not whose structure is governed by semver. Personally I think we ought to update our "public interface" doc to reflect this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool yeah I see the discussion in #36767

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the discussion in #36767, seems about right to me that no one "should" ever override the write_logs method, it is something that should not be changed. Seems like a non breaking change

@dstandish dstandish force-pushed the simplify-write-logs-logic branch from 886123e to e26679d Compare January 14, 2024 01:15

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, one nit/ask, rest is fine 👍🏽

Comment on lines 706 to 654

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the discussion in #36767, seems about right to me that no one "should" ever override the write_logs method, it is something that should not be changed. Seems like a non breaking change

"Set log level to DEBUG for traceback.",
e,
)
self.log.warning("Reading of logs interrupted with error %r;", e)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it retry? We should mention in the warning in that case.

@github-actions

github-actions Bot commented Mar 1, 2024

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Mar 1, 2024
@github-actions github-actions Bot closed this Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants