Cache last updated location for a configurable number of seconds #6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Prevent annotation updates when the only update is the pod xlog location. This reduces the load on the K8s API for the replicas, as we don't have to update the pod every loop_wait interval.
Add prevent_xlog_position_only_pod_updates K8s parameter.
Instead of updating xlog location in K8s every loop_wait interval, use a newly introduced parameter "location_cache_ttl" to prevent location-only updates from happening oftener than the value of this parameter. It is set to 0 by default, so nothing will change until it is explicitly set. Also, the maximum value is set to 10min, to prevent xlog value to be too stale (although it is only used for monitoring endpoints).
Pull Request Test Coverage Report for Build 9485196974Details
💛 - Coveralls |
Kubernetes DCS is not consistent about config reloads: it sends the DCS-specific config during init, while global config is sent on reload. Make sure we read parameters from the kubernetes part of the configuration. Update xlog_cache_ttl during local configuration reload.
feikesteenbergen
approved these changes
Jun 3, 2024
alexeyklyukin
commented
Jun 3, 2024
Reflect we are running a fork
Also add XLOG_CACHE_TTL env variable.
We used to rely on cached_xlog_location and replaced_xlog_location when showing a debug message about pod update prevention, but that produced false positives, i.e. ``` 2024-06-06 11:34:36,308 DEBUG: prevented pod update, keeping cached xlog value for up to -1 seconds ``` Make sure we only show this message when we actually used cached xlog, no updates happen, but they should have happened, since cached value and received member value do not match.
Prevents bogus "keeping cahcned xlog value for up to 0 seconds" messages
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of updating xlog location in K8s every loop_wait interval,
use a newly introduced parameter "location_cache_ttl" to prevent
location-only updates from happening oftener than the value of
this parameter. It is set to 0 by default, so nothing will change
until it is explicitly set.
Also, the maximum value is set to 10min, to prevent xlog value
to be too stale (although it is only used for monitoring REST endpoints).