HDDS-11418. leader execution flow

[sumitagrawl]

What changes were proposed in this pull request?

New flow for leader execution for performance.

1. Enable flag for new flow ozone.om.leader.executor.enable
2. TransactionInfo content format change including another index for Leader side execution (do not depend on ratis index)
3. LeaderExecutionFlow

• retrive DB changes using RDBBatchOperation cache
• Intrroduce PersistDb request to update db at all nodes via ratis - OmPersistDbRequest
• Simplified StateMachine - OMBasicStateMachine
• New classes for Leader execution flow -- OMGateway, LeaderRequestExecutor, FollowerRequestExecutor, RequestContext and PoolExecutor
• Switch to use new flow with minimal code to call new flow
• support for calling existing request at leader and follower (like snapshot, security cache - s3Secret) using OMGateway

4. Upgrade Prepare request switch for new flow to support prepare
5. Disable testcases that depends on further PR like Granular locking, No caching, ...

Pending:

1. Granular Locking, no-cache, quota resource,
2. metrics revisit
3. cleanup of old code (depends on merge),
4. re-factor existing requests to have proper fit for new flow

Parent JIRA:
https://issues.apache.org/jira/browse/HDDS-11415

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11418

How was this patch tested?

• Executed current test cases for success

Below will be handled with further story.

• Pending to have new testcases for new flow of code
• Few test cases - failing for cache at follower, locking supressed using @unhealthy("HDDS-11415 ***") tag
• 3 acceptance test case failing due to follower cache for snapshot, ...

1. Acceptance (misc)

Prepare Ozone Manager                                                 | FAIL |
255 != 0
--
Test write operation fails                                            | FAIL |
0 != 255
--
Om-Prepare :: Smoke test to test preparing OMs in an OM HA cluster.   | FAIL |
3 tests, 1 passed, 2 failed

2. Acceptance (compact)

Prepare Ozone Manager                                                 | FAIL |
255 != 0
--
1.4.0-1.5.0-SNAPSHOT-2-pre-finalized-prepare :: Prepares OMs          | FAIL |
1 test, 0 passed, 1 failed

3. Acceptance (HA-secure)

Create 2 actual keys with prefix 'sn', key contents the same as th... | FAIL |
255 != 0
--
Create snapshot 'snap2'                                               | FAIL |
Test timeout 5 minutes exceeded.
--
Data-Creation-Before-Om-Bootstrap :: Smoke test for creating data ... | FAIL |
5 tests, 3 passed, 2 failed

Performance test:

Env: M1 Mac, Docker-compose - om-HA
Freon:
ozone freon ockrw -n 1000000 -t 10 --percentage-read 0 --size 0 -r 10000 -v voltest -b buckettest -p performanceTest

Total execution time (sec): 839 ==> approx 14 min
Failures: 0
Successful executions: 1000000

Upstream Performance (Base) -- with WAL disabled (Without WAL disable, its not able to complete):

Total execution time (sec): 2041 ==> 34min
Failures: 0
Successful executions: 1000000