HAproxy based load balancer customization

[ravening]

Add new feature for load balancers based on haproxy

code ported from #4141

Description

As discussed in mailing list, create this PR for haproxy config customization.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/VR+haproxy+customization+in+CloudStack

Fixes #3789

It supports (or will support)

Basic haproxy configurations:
haproxy statistics uri, auth, enable/disable
global maxconn and maxpipes
Basic configurations on rule
timeout connection, client ,server globally and per rule
http, httpalive per rule
maxconn, fullconn per rule
maxconn, minconn, maxqueue per server in rule
advanced features
transparent load balancer
SSL offloading
http2 support
Variable SSL configurations

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

Adding networks level haproxy configs

Adding load balancer level configs

How Has This Been Tested?

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2139

[nvazquez]

Hi @ravening is this PR ready for review? Can you please fix the conflicts?

[ravening]

Hi @ravening is this PR ready for review? Can you please fix the conflicts?

@nvazquez done

[nvazquez]

Thanks @ravening
@blueorangutan package

[blueorangutan]

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2544

[nvazquez]

@blueorangutan test

[blueorangutan]

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-3273)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35254 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5799-t3273-kvm-centos7.zip
Smoke tests completed. 86 look OK, 6 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	Error	0.52	test_primary_storage.py
test_01_primary_storage_nfs	Error	0.10	test_primary_storage.py
ContextSuite context=TestStorageTags>:setup	Error	0.19	test_primary_storage.py
test_01_lb_usage	Error	0.00	test_usage.py
test_03_deploy_and_scale_kubernetes_cluster	Failure	32.87	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	Failure	63.49	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	Failure	33.86	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	Failure	29.80	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	Error	119.62	test_kubernetes_clusters.py
test_01_secure_vm_migration	Error	150.13	test_vm_life_cycle.py
test_02_unsecure_vm_migration	Error	300.21	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	Error	140.81	test_vm_life_cycle.py
test_08_migrate_vm	Error	43.71	test_vm_life_cycle.py
test_02_list_snapshots_with_removed_data_store	Error	9.43	test_snapshots.py
test_02_list_snapshots_with_removed_data_store	Error	9.43	test_snapshots.py
test_hostha_enable_ha_when_host_in_maintenance	Error	302.80	test_hostha_kvm.py

[nvazquez]

Sorry @ravening there are some conflicts again, can you please fix?

[github-actions]

Hi @${author}, your pull request has merge conflicts. Can you fix the conflicts and sync your branch with the base branch?

[ravening]

Sorry @ravening there are some conflicts again, can you please fix?

@nvazquez fixed it

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✖️ el7 ✖️ el8 ✖️ debian ✖️ suse15. SL-JID 3172

[blueorangutan]

Packaging result: ✖️ el7 ✖️ el8 ✖️ debian ✖️ suse15. SL-JID 3173

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[soreana]

@weizhouapache I have a suggestion for #5799. We can test it gradually, like what we did when we developed it. We can have a branch named HAproxy-LoadBalancer in the community. I will port all the changes from #5799 to HAproxy-LoadBalancer one by one. When satisfied with its quality, we can merge it to the main branch. That way, it would be far easier for community members to test it.
This feature could be part of it, too.

@soreana Each PR can be merged only if there are at least 2 approvals and all smoke tests pass. Most users use official release. If someone want to test a feature, they can build and test the specific branch, it is not needed to merge it into main.

@weizhouapache, Let's discuss it here and not spamming their pr :)

I completely agree with you, but the problem is that this pr didn't get the enough attention. I thought with that approach I can get the community members to review and test +300 line code changes rather than a scary +3,949 −86 😄