At least once strategy with failure handling for consumer groups

[cypriendker]

Hi,

We have a use case where we want to use consumer groups for high availability but make sure that all the messages of a topic have been successfully processed. The idea is to make sure that if a consumer in the group dies, its messages will be reprocessed.
From what I read, doing FetchMessage and then CommitMessage will not work as the offset is set up for the entire group.
I had multiple ideas but none of them are ideal or feasible :

• Having only one partition by topic: So that only one consumer at a time will be handling messages and we don't have the concurrency issue with Fetch and Commit. The downside is that we lose availability with a single partition in the Topic
• Having multiple topics but a custom failover assignment strategy (Only one consumer is assigned to all the partitions). This blog explains how to do it, but I couldn't find how we could generate custom groupBalancer
• Using Transactional producer, but it's not yet implemented

Any idea on how we can achieve that?

Thanks for the help