We offer a configuration option to tune how long we can use a throttle factor derived from a successful set of observations. So if the Admin client starts throwing exceptions or we encounter some inconsistencies in the observations, we can continue to use the last calculated throttle factor for some configurable time period.
So the user can say, "we are happy to use a successful observation from up to N seconds ago", after which time we would start using the fallback throttle factor (assuming that we are still failing to observe the cluster). The fallback can be configured by the user to be harsh, stopping message production, or lenient letting all traffic through.
We offer a configuration option to tune how long we can use a throttle factor derived from a successful set of observations. So if the Admin client starts throwing exceptions or we encounter some inconsistencies in the observations, we can continue to use the last calculated throttle factor for some configurable time period.
So the user can say, "we are happy to use a successful observation from up to N seconds ago", after which time we would start using the fallback throttle factor (assuming that we are still failing to observe the cluster). The fallback can be configured by the user to be harsh, stopping message production, or lenient letting all traffic through.