ZOOKEEPER-4951: Optimize the default Prometheus max queue size

[li4wang]

Setting the default max queue size to 10k instead of 1M to reduce the memory footprint and optimize GC overhead for enabling Prometheus metrics provider.

The read throughput was increased by 52% according to our internal perf comparison tests.

[li4wang]

@eolivelli Would you mind taking a look at this PR? Thanks.

[li4wang]

@kezhuw would you mind taking a look at it? Thanks.

[kezhuw]

Looks good to me. If system is overloading 10000, it will overloading 1000000 finally. Reducing the max queue size will let us know this earlier.

Setting the default max queue size to 10k instead of 1M to reduce the memory footprint and optimize GC overhead for enabling Prometheus metrics provider.

Does ArrayBlockingQueue help ?

[li4wang]

Thanks for reviewing the PR. @kezhuw

The ArrayBlockingQueue uses less bytes per element compared to LinkedBlockingQueue, however it allocates memory for all elements upfront. It could be beneficial if the maxQueueSize is set to some reasonable value.

We can look into the potential enhancement of conditionally use LinkedBlockingQueue vs ArrayBlockingQueue based on some perf test results.