Skip to content

Conversation

sparrc
Copy link
Contributor

@sparrc sparrc commented Sep 11, 2025

  • Change DEFAULT_ROLLOVER_TYPE from 'date' to 'size'
  • Change DEFAULT_MAX_FILE_SIZE from 10MB to 5MB
  • Update README.md documentation for ECS_LOG_ROLLOVER_TYPE and ECS_LOG_MAX_FILE_SIZE_MB
  • Fix unit tests to expect size-based rollover instead of date-based
  • Add test cases for explicit hourly rollover to maintain backward compatibility

The reason for this change is to improve the situation where customers
are pulling logs more than 24 hours after an issue has occured, and we
are not able to troubleshoot it because there are no relevant logs
available.

Some metrics were pulled from our functional testing account to determine that this change should generally not negatively affect log retention in the case of a high-load instance, even in the case that debug logs are enabled.

With "debug" log level:

  • Median: 1.02 MB/hr
  • Average: 2.13 MB/hr

With "info" log level:

  • Median: 0.17 MB/hr
  • Average: 0.48 MB/hr

Testing

New tests cover the changes: no, modified existing unit tests

Description for the changelog

Enhancement: default to size-based log file rollover instead of hourly

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@sparrc sparrc requested a review from a team as a code owner September 11, 2025 23:42
@sparrc sparrc force-pushed the update-log-rollover branch 2 times, most recently from 1bfef12 to 050e301 Compare September 12, 2025 16:21
@sparrc sparrc changed the title [wip] Change ECS Agent log rollover defaults to size-based rotation Change ECS Agent log rollover defaults to size-based rotation Sep 12, 2025
- Change DEFAULT_ROLLOVER_TYPE from 'date' to 'size'
- Change DEFAULT_MAX_FILE_SIZE from 10MB to 5MB
- Update README.md documentation for ECS_LOG_ROLLOVER_TYPE and ECS_LOG_MAX_FILE_SIZE_MB
- Fix unit tests to expect size-based rollover instead of date-based
- Add test cases for explicit hourly rollover to maintain backward compatibility

The reason for this change is to improve the situation where customers
are pulling logs more than 24 hours after an issue has occured, and we
are not able to troubleshoot it because there are no relevant logs
available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants