HDDS-10479. Support Ozone to run ratis local related commands

[sarvekshayr]

What changes were proposed in this pull request?

Added support to ozone to run ratis related commands.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10479

How was this patch tested?

Added unit tests in TestOzoneRatis.

Tested the CLI on a docker cluster.

bash-4.4$ ozone ratis
Usage: ratis sh [generic options]
         [election [transfer] [stepDown] [pause] [resume]]         
         [group [info] [list]]                                     
         [local [raftMetaConf]]                                    
         [peer [add] [remove] [setPriority]]                       
         [snapshot [create]] 

bash-4.4$ ozone ratis local raftMetaConf -peers "peer1|localhost:8080,peer2|localhost:8081" -path ../.././data/metadata/ratis/bf265839-605b-3f16-9796-c5ba1605619e/current/
Index in the original file is: 0
Generate new LogEntryProto info is:
index: 1
configurationEntry {
  peers {
    id: "peer1"
    address: "localhost:8080"
    startupRole: FOLLOWER
  }
  peers {
    id: "peer2"
    address: "localhost:8081"
    startupRole: FOLLOWER
  }
}

bash-4.4$ ozone ratis local raftMetaConf
Failed to parse args for raftMetaConf: Missing required options: peers, path
Usage: raftMetaConf -peers <[P0_ID|]P0_HOST:P0_PORT,[P1_ID|]P1_HOST:P1_PORT,[P2_ID|]P2_HOST:P2_PORT> -path <PARENT_PATH_OF_RAFT_META_CONF>
Generate a new raft-meta.conf file based on original raft-meta.conf and new peers.

bash-4.4$ ozone ratis local raftMetaConf -path ../.././data/metadata/ratis/bf265839-605b-3f16-9796-c5ba1605619e/current/
Failed to parse args for raftMetaConf: Missing required option: peers
Usage: raftMetaConf -peers <[P0_ID|]P0_HOST:P0_PORT,[P1_ID|]P1_HOST:P1_PORT,[P2_ID|]P2_HOST:P2_PORT> -path <PARENT_PATH_OF_RAFT_META_CONF>
Generate a new raft-meta.conf file based on original raft-meta.conf and new peers.

bash-4.4$ ozone ratis local raftMetaConf -peers localhost:8080
Failed to parse args for raftMetaConf: Missing required option: path
Usage: raftMetaConf -peers <[P0_ID|]P0_HOST:P0_PORT,[P1_ID|]P1_HOST:P1_PORT,[P2_ID|]P2_HOST:P2_PORT> -path <PARENT_PATH_OF_RAFT_META_CONF>
Generate a new raft-meta.conf file based on original raft-meta.conf and new peers.

bash-4.4$ ozone ratis local raftMetaConf -peers "localhost8080" -path ../.././data/metadata/ratis/bf265839-605b-3f16-9796-c5ba1605619e/current/
Failed to parse the server address parameter "localhost8080".

bash-4.4$ ozone ratis local raftMetaConf -peers "peer1|localhost:8081,peer2|localhost:8081" -path ../.././data/metadata/ratis/bf265839-605b-3f16-9796-c5
ba1605619e/current/
Found duplicated address: localhost:8081. Please make sure the address of peer have no duplicated value.

bash-4.4$ ozone ratis local raftMetaConf -peers "peer1|localhost:8080,peer1|localhost:8081" -path ../.././data/metadata/ratis/bf265839-605b-3f16-9796-c5ba1605619e/current/
Found duplicated ID: peer1. Please make sure the ID of peer have no duplicated value.

[sumitagrawl]

@sarvekshayr Thanks for working over this, have few review comment.

[errose28]

Hi @sarvekshayr, I have a few questions about this PR.

• What is the intended use case for this command?
• Can we leave this to the Ratis CLI without manually copying over ratis sh commands to Ozone? If we need Ratis commands it would probably be better to ship Ratis shell directly with our Ozone release.
• Everything under ozone admin should run over the network. If this needs to be run from a specific host to modify the server's files it should go under ozone repair.

[sumitagrawl]

• What is the intended use case for this command?

@errose28
This command is used for recovery of ratis qourum. i.e. one of node goes down and unable to retrieve, this till will help in regenerate raftMetaConf with new peers.

This case is observed when one of SCM metada gots deleted in some of recovery.

[kerneltime]

cc @hemantk-12 This might be useful for manually backing up and restoring OM.

[errose28]

We had some discussion on this and came up with the following points:

• It will be useful to have a command ozone ratis that works the same as the ratis CLI, but automatically populates the TLS information required to communicate with a secure Ozone cluster.
  • Currently ratis invoked from the command line does not support TLS and will fail.
• We should implement this in a way where Ozone can just forward arguments to Ratis. We should not manually create wrappers of the same names for every Ratis CLI, like how this change is currently implemented.
• One possible solution proposed by @szetszwo was a Ratis change that exposes RatisShell and allows passing a builder for TLS configuration. Then from the ozone ratis subcommand we could do something like this:

RatisShell shell = new Builder().setTlsConf(tls).build();
shell.run(args)

[szetszwo]

Filed RATIS-2155 "Add a builder for RatisShell".

[szetszwo]

For the local raftMetaConf command, we don't have to wait for RATIS-2155 since it is a "local" command which does not require TLS.

[szetszwo]

@sarvekshayr , thanks for the update! The change looks good. Just some minor comments inlined.

[szetszwo]

+1 the change looks good.

[sumitagrawl]

LGTM