Fetching host/port from a Jedis instance (4.X)

[MSPigl]

Hello,

I'm currently upgrading my team's codebase from Jedis 2.X to 4.X. In our existing code, we use Jedis.getClient().getPort()/getHost(), which is used for logging/unit testing to verify that external configuration matches what the application's internal configuration expects.

However, 4.X switched Jedis.getClient() to just passthrough to getConnection(), so I'm not seeing how I would get the host and port for a Jedis instance after the upgrade. The logic that leverages getHost()/getPort() isn't application critical and could be reworked if there really is no way to get that data anymore, however I'd definitely prefer that the logic be kept.

Any thoughts on how to retrieve that data in 4.X?

[sazzad16]

@MSPigl Thank you for sharing your use case. Let's see if anyone else has anything to say and/or ideas to share.

The inspiration behind the removal of those methods is that the Connection may not be bound to traditional host and port anymore. For example we can mention UDS socket support. Also, host and port provided in constructor may not be the true ones at all due to the possible presence of HostAndPortMapper.

[YNedderhoff]

Essentially we have some batching logic to reduce writes. We use a ShardedJedis instance to get the shard for a given key, so that we know where it belongs. We batch by using a map with shards as key, and a LinkedBlockingQueue of increment operations as value. We define a shard by host and port as a unique identifier.

Essentially an abstraction layer where for example incrementing will either just add it to the queue, or also flush the queue. A queue exists by shard.

public void incrementBy(IncrementOperation incrementOperation) {
    String shard;
    try (ShardedJedis jedis = shardInfoPool.getResource()) {
        Client client = jedis.getShard(getKey(incrementOperation)).getClient();
        shard = client.getHost() + ":" + client.getPort();
    }

    batchedUpdates.putIfAbsent(shard, new BatchedItem(DateTime.now(), new LinkedBlockingQueue<>()));

    BlockingQueue<IncrementOperation> queue = batchedUpdates.get(shard).getQueue();
    queue.add(incrementOperation);

    maybeFlushQueue(shard, queue);
}

So if we could replace host and port with another unique identifier of the shard, I think we'd be fine.

[sazzad16]

@YNedderhoff Just to understand correctly, are you describing how you are using Jedis 3.x and asking for a leeway in Jedis 4.x?

[YNedderhoff]

Yes. Sorry, could have been clearer.

[sazzad16]

@MSPigl @YNedderhoff PTAL at #3137 and let me whether it fulfills your purposes. You'd be able to do Jedis.getConnection().getSocketString().

[sazzad16]

Am I right that this now means that I should replace ShardedJedisPool with ShardedPipeline in Jedis 4.x?

@YNedderhoff It seems you should be replacing almost everything with ShardedPipeline.

ShardedPipeline shardedPipeline;

public void incrementBy(IncrementOperation incrementOperation) {
    insertInPipeline(shardedPipeline, incrementOperation);
}

// shardedPipeline.sync(); // close() calls sync() anyway
shardedPipeline.close();

[YNedderhoff]

Yes, I thought the same, but, how do I solve my use case as outlined above? The one you're proposing that PR for?

[sazzad16]

You won't need that anymore. It'll be handled internally in ShardedPipeline.

[YNedderhoff]

I am not following I think. The batched updates I have posted above are handled internally?

[sazzad16]

Yes. Each close() would complete one batched updates. We may improve it to one batch per sync() (#3207) but this may take time.

IMO, ATM a better approach for you could be having one ShardedConnectionProvider and use it in multiple pipeline (if possible). This way connections will be better reused.

ShardedConnectionProvider provider;
//Pattern pattern; // if you want

// start batch -->
ShardedPipeline pipeline = new ShardedPipeline(provider); // or, new ShardedPipeline(provider, pattern);

//
// add commands/operations in pipeline
//

pipeline.close(); // <-- complete batch

[sazzad16]

#3745 is just merged which could be useful in OP's use case.
Note: This is targeted for Jedis version 5.2.0.