The development have been moved to https://github.com/strimzi/kafka-quotas-plugin
This is a broker quota plugin for Apache Kafka to allow setting a per-broker limits statically in the broker configuration.
The default quota plugin in Apache Kafka will hand out a unique quota per client. This plugin will configure a total quota independent of the number of clients. For example, if you have configured a produce quota of 40 MB/second, and you have 10 producers running as fast as possible, they will be limited by 4 MB/second each.
The quota distribution across clients is not static. If you have a max of 40 MB/second, 2 producers, and one of them is producing messages at 10 MB/second, the second producer will be throttled at 30 MB/second.
To build the plugin:
./gradlew jar
Copy the resulting jar in build/libs/kafka-static-quota-plugin-0.1.jar into the Kafka classpath.
Alternatively, you can publish to your local maven repository with:
./gradlew publishMavenJavaPublicationToMavenLocal
Configure Kafka to load the plugin and some plugin properties:
client.quota.callback.class=org.apache.kafka.server.quota.StaticQuotaCallback
# The quota is given in bytes, and will translate to bytes/sec in total for your clients.
client.quota.callback.static.produce=1000000
client.quota.callback.static.fetch=1000000
# Storage quota settings in bytes. Clients will be throttled linearly between produce quota and 0
# after soft limit.
client.quota.callback.static.storage.soft=10000000
client.quota.callback.static.storage.hard=20000000
# Check storage usage every 5 seconds
client.quota.callback.static.storage.check-interval=5
Run it locally (make sure your server.properties enables the reporter):
CLASSPATH=/path/to/build/libs/kafka-static-quota-plugin-all.jar ./bin/kafka-server-start.sh server.properties