Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support file source close when file not active for a long time #22733

Open
suikammd opened this issue Mar 27, 2025 · 0 comments
Open

Support file source close when file not active for a long time #22733

suikammd opened this issue Mar 27, 2025 · 0 comments
Labels
type: feature A value-adding code addition that introduce new functionality.

Comments

@suikammd
Copy link
Contributor

suikammd commented Mar 27, 2025

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

No response

Attempted Solutions

No response

Proposal

Hi Vector team,

Currently, the file source keeps file handles open indefinitely, even for files that haven't received new writes for a long time. This can lead to issues in environments like Kubernetes or systems relying on log rotation, where open file descriptors can block volume unmounting or prevent rotated log files from being deleted.

It would be very helpful if Vector could support a general-purpose mechanism to:

Close file handles automatically after a period of inactivity (no new writes),

Reopen the file when new data is detected.

This behavior would help reduce FD usage and avoid interfering with file lifecycle management, without the downsides of ignore_older (which can cause data loss by skipping files entirely).

Is there any plan to support this? If so, I'd love to help implement this feature.
Thanks for the great work!

References

No response

Version

No response

@suikammd suikammd added the type: feature A value-adding code addition that introduce new functionality. label Mar 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature A value-adding code addition that introduce new functionality.
Projects
None yet
Development

No branches or pull requests

1 participant