Skip to content

Add overwrite parameter to IMAP hook download_mail_attachments#66850

Closed
kyupark wants to merge 2 commits into
apache:mainfrom
kyupark:fix-imap-overwrite-65870
Closed

Add overwrite parameter to IMAP hook download_mail_attachments#66850
kyupark wants to merge 2 commits into
apache:mainfrom
kyupark:fix-imap-overwrite-65870

Conversation

@kyupark

@kyupark kyupark commented May 13, 2026

Copy link
Copy Markdown
Contributor

Add an overwrite parameter to the IMAP hook's download_mail_attachments() method.

When overwrite=True (default), duplicate filenames are overwritten (current behavior).
When overwrite=False, unique filenames are generated by appending a suffix (e.g. report_1.csv, report_2.csv).

The non-overwrite path uses exclusive create mode ("xb") to avoid TOCTOU races, and checks both in-memory state and the filesystem to handle files left over from previous runs.

closes: #65870


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines


Important

🛠️ Maintainer triage note for @kyupark · by @potiuk · 2026-07-02 17:46 UTC

Some review feedback from @shahar1 is waiting on you:

  • 2 unresolved review thread(s) from @shahar1 need a reply or a fix.

The ball is in your court — you've been assigned to this PR. Reply or push a fix in each thread, then mark them resolved.

Automated triage — may be imperfect; a maintainer takes the next look.

Comment thread providers/imap/tests/unit/imap/hooks/test_imap.py
@potiuk

potiuk commented May 18, 2026

Copy link
Copy Markdown
Member

@kyupark — There is 1 unresolved review thread on this PR from phanikumv, and you have engaged with it (post-review commits and/or in-thread replies). Could you confirm whether you believe the feedback is fully addressed and the PR is ready for maintainer review confirmation?

If yes, please mark the thread as resolved and ping the reviewer (phanikumv) for a final look. They will either label the PR ready for maintainer review or follow up with additional feedback.

If you are still working on the thread, please reply with what is outstanding so the threads stay unresolved on purpose.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

@kyupark

kyupark commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

@phanikumv I added the focused FileExistsError retry test you suggested: the first open(..., "xb") for report.csv raises FileExistsError, then the retry succeeds with report_1.csv.

Local checks passed and CI is green. I’ve marked the thread resolved. Could you take a final look when you have a chance?

@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 24, 2026

@phanikumv phanikumv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on the feedback, @kyupark . LGTM

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two nits

file_name = name
self._create_file(file_name, payload, local_output_directory)

def _unique_name(self, name: str, seen: dict[str, int], local_output_directory: str) -> str:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Names of methods should start with a verb (_generate_unique_name)

f.write(payload)
break
except FileExistsError:
seen[file_name] = 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be redundant considering the logic below

@eladkal

eladkal commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

@kyupark can you please address the open comments?

@potiuk potiuk removed the ready for maintainer review Set after triaging when all criteria pass. label Jun 22, 2026
@potiuk

potiuk commented Jun 25, 2026

Copy link
Copy Markdown
Member

@kyupark — this PR has 2 unresolved review thread(s) that still look like they need your attention. Once you've addressed them (push changes and/or reply in-thread), please resolve the threads or reply to confirm, and give the reviewer a nudge for another look. Thanks!

See the PR quality criteria.

Automated first-pass triage note drafted by an AI-assisted tool — may get things wrong; once addressed, a real Apache Airflow maintainer takes the next look. (why automated)


Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting

@vincbeck

vincbeck commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Fixed in #68838

@vincbeck vincbeck closed this Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IMAP hook silently overwrites attachments with identical filenames

6 participants