feat(inputs.directory_monitor): Allow to preserve timestamps when moving file

[highlyavailable]

Summary

When a processed file is moved to the finished or error directory, the plugin copies it to a new file, so the moved file gets the current time as its modification time and the original file history is lost. This is a problem for the auditing and tracking use cases described in #16885.

This adds an optional preserve_timestamps setting. When enabled, the source file's access and modification times are captured before the original is removed, then reapplied to the moved file with os.Chtimes. It defaults to false, so existing behavior is unchanged. Added a test covering both the enabled and disabled paths.

Checklist

• No AI generated code was used in this PR
• AI generated code used in this PR follows the InfluxData Policy on AI-Generated Code Contributions

Related issues

partially resolves #16885

[telegraf-tiger]

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

[highlyavailable]

!signed-cla

[srebhan]

Nice. Thanks @highlyavailable for your contribution! Just one comment from my side...

[Hipska]

Does the current state of the code actually sets the created date to the original file create date? According to the documentation, it would only set the access and modification time.

[srebhan]

@highlyavailable can you please run make fmt and make docs on the code and push the changes to make the CI pipelines happy?!

[srebhan]

AND please restore the PR description template, especially the AI checkboxes. Those items are there for a reason. :-)

[highlyavailable]

no, it only sets the access and modification time. The call is here:

telegraf/plugins/inputs/directory_monitor/directory_monitor.go

Line 453 in 1b95c71

if err := os.Chtimes(dstPath, srcTimes.AccessTime(), srcTimes.ModTime()); err != nil {

the sample.conf comment says "access and modification" so the docs match the behavior. do you thhink this should be changed?

[highlyavailable]

@srebhan I addressed both of your comments!

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[Hipska]

So this PR does not solve the mentioned Feature Request, were it states:

the system should preserve the file's original created and updated timestamps.

[highlyavailable]

So this PR does not solve the mentioned Feature Request, were it states:

the system should preserve the file's original created and updated timestamps.

to go back on my last response, the "updated" part of the issue's wording is modification time, which this PR actually does preserve. so the only remaining issue is not perserving creation time.

Go's stlib doesn't have a setter (that I know of) to set that creation/birth time. os.Chtimes covers access and mod times, but I believe birth time would need per-platform syscalls (macOS setattrlist, Windows SetFileTime, not sure about Linux but I could look into it)

I feel this is a reasonable (partial) fix given the intent of the issue, but I should have called out that it doesn't satisfy the full reqs in the feture request.

I could either update the docs to make clear only access+mod times are covered, or update the PR attempting to include per-platform creation-time support as a follow-up. any thoughts either way?

also, just wanted to say thank you @Hipska for the thorough review, its my first time contributing to this project and I appreciate it.

[Hipska]

Would the creation time be preserved when using os.Rename? If so, @srebhan would you agree to first using this and fallback to copy+chtimes as earlier suggested in #18921 (comment)

[highlyavailable]

Would the creation time be preserved when using os.Rename? If so, @srebhan would you agree to first using this and fallback to copy+chtimes as earlier suggested in #18921 (comment)

yes, on the same FS os.Rename (which wraps rename(2), source) just relinks the directory entry, so access+modification+ creation times would all stay intact.

And as we discussed, cross-FS it fails with EXDEV and we would need to fall back to copy + Chtimes, which only restores access and modification time

[srebhan]

@Hipska we cannot use os.Rename as there is no reliable way to know when it succeeds or fails. The behavior is highly OS-dependent and (on some OS) also depend on if you are using mount boundaries. We had this and moved away from it to the current code. So a clear "no" for going back.

@highlyavailable let us get this in as-is first. Please adapt your PR description and change "resolves" to "partially resolves"... In a second PR, implement platform dependent ways of restoring the creation/birth time. On Windows that's pretty easy (use SetFileTime) but I'm not sure that's even possible on e.g. Linux...

[highlyavailable]

@srebhan I just updated the description, thank you both for working on this with me

[srebhan]

Thanks @highlyavailable and sorry for the late response!

[skartikey]

@highlyavailable Thanks for the contribution!