Commit signoff gdinhof (#71)

* Create LICENSE

Signed-off-by: Gerhard Dinhof [email protected]

* docker-compose setup to start Issuer-Verifier and Holder AcaPys as well as Postgres DBs

* Clean up docker-compose setup

* Add postman based load generator for testing if the local setup works as expected

* choreinitialized kotlin spring boot project

* chore: added copyright

* added maven to git ignore

* docs: removed generated help md

* chore!: updated to jdk 17

* docs: added copyright

* feat: added boilerplate apis

* feat: added Dockerfile

* docs: added README

* Add submodule + script for VON-network

* restructuring the files

* Automatically register public DID on the ledger

* code cleanup

* build: Switch to Java 11 as the built target for brought compatibility

* feat: Integration of an automatically generated SwaggerUI

* Update README.md

* Update README.md

* feat(AcaPy Integration): Integration of an AcaPy Java client library for calling the AcaPy Admin API as well as receiving Webhook calls from AcaPy

* Working state with permission error in tails

* tails server working

* added extended logging capabilities

* removed gitignore

* remove ignored files

* chor(Postman): Restructure files

* chor(.idea): Update .idea files

* chor(Setup): Let user know that it will take 30 seconds to register DID

* chor(Setup): Automatically load and update git submodules

* build(Java Version): Switch to Java Version 11

* Added README for the setup

* Fixed typo in environment files

* chore: Rename WebhookController to AcaPyWebhookController and LoadGeneratorController to InfoController

* refactor: Structure of controllers and Swagger UI

* feat: Integrate simple webhook logging

* feat(TestRunner): application.yml is used to choose and configure the test case

* feat(FullProcessRunner): Write schema and credential definition to ledger during setup

* feat(FullProcessRunner): Connect to Issuer/Verifier as well as Holder AcaPy

* feat(FullProcessRunner): Establish connection between Issuer/Verifier and Holder

* build(Restart Policy): Restart each container always

* feat(FullProcessRunner): Issue credential to holder and request credential from holder via proof request

* refactor(Spring Configs): Move all Spring Config files into a config package

* refactor(Aries Client): Build a wrapper around the AcaPyAriesClient library to make it easier replaceable by a different AriesClient library

1. to make it easier to time the HTTP requests the AriesClient
2. to potentially be able to use this load generators to test other agents too

* refactor(Logger): Move logger into their own package

* refactor(Aries Event Handler): Build a wrapper around the AcaPyEventHandler to make it easier replaceable by a different AriesClient library

To potentially be able to use this load generators to test other agents too.

* refactor(AriesClient and AriesObserver): Move service interfaces into the domain layer

* chore: Renaming of package

* chore: Clean up SwaggerConfig

* feat(Readme): Update the Readmes

* feat(Wallet Logger): Log the size of the issuer/holder wallet database periodically

* refactor(Runner): Move termination condition into own method

* feat(logging): added Loki as log collector

* feat(Log HTTP Requests): Log the durations and Status Codes of HTTP requests going out to the AcaPy

* chore(Renaming): Rename AriesEventLogger types

* chore: Remove unused volume from docker-compose

* chore: Clean WalletLogger log message

* refactor(Agent Implementations): Move all AcaPy agent specific code into one folder

* chore: Adopt manage.sh logging

* chore: Adopt manage.sh logging

* feat(AcaPy Http Request Timeout): Make it possible to configure the timeout for HTTP requests to the AcaPy clients

* chore(Name Containers): Name the Grafana and Loki container

* chore(Logging): Remove unused Logback configuration

* chore(Logging): Adopt Logback output format

* feat(Provision Grafana): Add Loki as the default datasource to Grafana and deactivate authentication for Grafana

* fix(Wallet Logger): remove comma from log message

* feat(Dashboard): Configured a default Grafana dashboard that is automatically provisioned on startup of Grafana

* feat(Dashboard): Add overviews about Connections, Credentials, and Proof Requests to the dashboard

* feat(Dashboard): Visualize the HTTP Response Codes and HTTP Response Times in Grafana

* feat(Dashboard): Visualize the rate in which connections are being activated, credentials are being issued, and proof requests are being verified in Grafana

* fix(AcaPy Event Publisher): Fix string date to milliseconds conversion

* fix(Loki Config): Add custom Loki config to avoid the "Too many outstanding requests." issue

* feat(Dashboard): Add total number of connection, credential issuance, and proof request count

* refactor(Dashboard): Rename the keys that are being logged

* chore(.env): Remove .env file from git

* chore(Dashboard): Update HTTP Response Times metric

* chore(Dashboard): Fix labeling rate is not per Second but per Minute

* chore(Dashboard): Provide the wallet DB size in MB instead of bytes

* chore(Dashboard): Outline the number of test iterations started per minute on the dashboard to visualize the load the load generator puts on the Agents

* feat(FullProcessRunner): Add a new test runner to start new full process iterations on a constant "iteration per minute" base

* chore(Dashboard): Reorder dashboard panels

* feat(Export PDF Report): Grafana dashboards can now be exported as a PDF

* fix(Dashboard Deployment): Put dashboard and logging containers into a separate docker network to avoid error when shutting everything down

* refactor(Full Process Runner): Avoid code duplication across the FullProcessRunners

* feat(Full Process Runner): Schedule FullProcessConstantLoadRunner at a fixed rate leveraging a thread pool

* refactor(Aries Agent): Extract connectionId from ProofRequestDo and CredentialDo in preparation for connectionless proof requests

* feat(Connectionless Proof Requests): Started implementing support for connectionless proof requests - not finished yet

* feat(Scale AcaPy using Nginx): Scale AcaPy horizontally by using a Nginx container as a load balancer

* feat(Scale AcaPy using Nginx): Scale Issuer/Verifier as well as Holder AcaPy to 10 instances each by using a Nginx container as a load balancer

* feat(AcaPy Wallet Type): Make AcaPy Wallet Type configurable via the .env file

* fix(.env): Fix .env file

* feat(Handle Exception on Connection Invitation Receive): Start a new iteration if the holder is not able to accept the connection invitation

* feat(Dashbaord): Show number of failes "POST/connections/receive-invitation"

* chore(Holder AcaPy): Increase number of Holder AcaPys to 20

* chore(Dashboard Setup): Define the containers Grafana depends on

* chore(Dashboard Setup): Define the containers Grafana depends on

* chore(AcaPy): Increase default AcaPy Version

* fix: Circular dependency in dashboard/logging deployment

* refactor(TestRunners and TestFlows): Differentiate between TestRunners and TestFlows to make it easier to implement differen TestFlows while reusing the existing TestRunners

* chore(Readme and Dashboard): Adopt the Readme and Dashboard to meet the new setup

* feat(Issuer Flow): Added a new test flow to simulate an issuer who is establishing connections to holders and issues credentials to them afterwards

* chore(Application Config): Add new application configuration settings and rename existing ones

* chore(Application Config): Add new application configuration settings and rename existing ones

* chore: Readme

* feat(Revocable Credentials): Configure if a credential should be revocable, the size of the revocation registry, and if a proof request should include a non revoked check

* chore(OOB Credential Offer): Preparations to support OOB Credential Offers

* feat(Wallet DB Connections): Configure the number of connections to the Wallet DBs

* feat(Connection Flow): Test Flow that only established connections with the holder

* feat(Credential Issuance Flow): Test Flow that only issues credentials to the holder

* feat(Proof Request Flow): Test Flow that only requests proofs from the holder

* feat(Increasing Load Test Runner): Added a new test runner that increases the peak load in predefined steps

* fix: Count of iterations in the Increasing-Load-Test-Runner

* feat(AcaPy): Make Storage Type used by AcaPys configurable

* feat(ErrorLogger): Introduction of an error logger to log unexpected error that occurred during testing

* feat: Start Load Generator as Docker Container using "./manage.sh start" - to not start it via docker use "./manage.sh startWithoutLoadGenerator"

* fix: .env.example

* feat(Distribute load Across many Holders): Instead of having one horizontally scaled holder agent we now start many independent holder agents and the load generator takes care of distributing the load across all the holders

* feat: Increase number of stand-alone holders to 10

* feat: Increase number of issuer/verifier AcaPys to 10

* feat: Add known error to the dashboard

* feat: Rework manage.sh to ensure everything starts in the correct order

* feat: Offer a "restart" command via the manage.sh

* fix: AcaPy provisioning

* fix: AcaPy provisioning

* docs: Update Readme

* Prometheus set-up (#43)

* #32 Initial setup (macOS only)

* #32 adding cadvisor support to be docker and platform agnostic

* #32 Adding node-export

* #32 Adding prometheus data source provisioning config

* #32 Adding prometheus / node-exporter dashboard

* Remove unused redis container

* Increase platform agnostic-ness

* fix: remove "redis" dependency

* Removed unused prometheus scraper

* Added Prometheus and installation guide for node-exporter

Co-authored-by: PaulWen <[email protected]>

Signed-off-by: Gerhard Dinhof [email protected]

* fix(manage.sh): Rebuild load-generator docker image on every start to ensure that the latest application.yaml is loaded

* feat(Docker Container Metrics): Visualize the docker container metrics in Grafana that have been collected by cAdvisor

* feat(Linux Localhost): Avoid having to switch between "host.docker.internal" and "172.17.0.1" depending on the host OS

* feat(Postgres Metrics): Collect Postgres metrics using postgres_exporter and visualize them in Grafana

* Fixed cAdvisor volume mapping for macOS #44


Signed-off-by: Gerhard Dinhof [email protected]

* fix: configure postgres username and password via env-variables

* feat(Configure Load Tests): The load test configuration can now be done via the .env file instead of the application.yml

* feat(Configure Load Tests): The load test configuration can now be done via the .env file instead of the application.yml

* chore(Dashboards): Improve description of graphics

* chore(Dashboards): Add a download link to each dashboard

* chore(Dashboards): Update test result dashboard

* fix(Exception Handling): Catch and log exceptions thrown in the context of ScheduledThreadPools

* refactor(Error Logging): Rework the error logging to differentiate between errors and provide more information for easier debugging

* feat(Summary Dashboard): Introduce a new dashboard which lists the most interesting graphs in one single dashboard

* fix(Grafana PDF Exporter): Build grafana-pdf-exporter docker image to have the latest bug fixes and features included

* chore: Specify ID for Grafana folder to ensure a static URL

* fix: Improve error logging for failed HTTP requests to AcaPy

* fix(Test Results Dashboard): Improve the error logging on the test results dashboard

* feat(Test Results Dashboard): Log started iterations as well as pending and failed iterations

* fix(Constant and Increasing Load Runners): Previously, the scheduler used to generate the load was not running more than one task at a time.
This caused the load generator not to generate the desired iterations per minute.

* chore: Fix wording on test result dashboard

* chore: Update default Revocation Registry Size to 3000

* refactor: Combine test result and summary dashboards into one dashboard

* chore: Add CPU and memory usage of all containers to the test result dashboard

* chore: Adjust tooltips

* fix: Increasing Load Test Runner

* chore: Improve test result dashboard

* chore: Improve test result dashboard

* Revert "fix(Grafana PDF Exporter): Build grafana-pdf-exporter docker image to have the latest bug fixes and features included"

This reverts commit cfe7f21.

* chore: Improve test result dashboard

* chore: Restructure the manage.sh

* fix: Cancel schedulers properly once the tests are over

* chore: Update test results

* docs: restructured and added architecture diagram

* chore: deleted unneeded readme

* docs: added nginx and small cleanups

* Implement Postgres Cluster as wallet DB (#49)

Issue: #48
Author: Ilie Circiumaru
Reviewer: @PaulWen 
Description: Implement 3 instances Postgres Cluster for wallet DB for the issuer-verifier. Split docker-compose on more files to enable launch of the isolated components

* chore: Update Test-Results Dashboard

* chore: Remove unused grafana-pdf-exporter reference

* refactor: Cleanup the setup procedure

* fix: Gitmodules

* fix: Gitmodules

* fix: Gitmodules

* fix: nginx-issuer-verifier.conf

* fix: manage.sh

* chore: remove unused profiles from docker-compose-agents.yml

* feat(Activate Postgres Cluster via .env): Switch from many start commands to just one and configure what is started via the .env

* feat(Configure Number of Issuer/Verifier Agents): Depending on the number of requested issuer/verifier AcaPy instances a different nginx.conf is loaded that matches the desired number

* feat(Configure Number of Holder Agents): The number of requested holder AcaPy instances can be changed via the .env

* chore: restructure .env

* feat(Log Revocation Registry Metadata): Track the usage of revocation registries. In particular look out if any RevReg/Index combination has been assigned to multiple credentials.

* Implement declarative deployment configuration (#58)

Issue: #56
Co-authored-by: Ilie Circiumaru <[email protected]>
Co-authored-by: PaulWen <[email protected]>
Description:
1. Implementation of declarative deployment configuration
2. Setup metrics for Postgres Cluster

* fix(Revocation Registry Logging): Do not log any revocation registry metadata if not revocation registry is used

* feat(Postgres Configuration): make the "statement_timeout" configurable

* fix: network handling with docker-compose

* feat(Postgres configuration): Configure "idle_in_transaction_session" and "lock_timeout" via the .env

* fix(Postgres configuration): "idle_in_transaction_session_timeout"

* fix: Test Results Dashboard

* chore: Update Test-Results Dashboard

* feat(Multitenancy): Run tests with AcaPy in multitenancy mode

* chore: remove unnecessary host.docker.internal

* docs: Split Readme into Developer and User Readme to avoid any confusion

* docs: Improve the README.md

* docs: Improve the README.md

* fix: add host.docker.internal reference to load-generator so that he is able to connect to the VON Network to register a public DID

* chore: Update test-results.json

* docs: define test cases to verify the load generator still behaves as expected

* docs: Add a troubleshooting section to the README.md

* Implement monitoring of the Spring Boot app (#62)

Author: Ilie Circiumaru <[email protected]>
Issue: //issues/45
Reviewer Paul Wen

* docs: Update Troubleshooting section

* docs: Fix spelling

* Added configuration option to enable Postgres debug logging (#60)

* Added flags to enable postgres deub logging

* Removing postgres debug settings from clustered setup

* added comprehensive description for POSTGRES_LOG_DEBUG

* alphabetical order

Signed-off-by: Gerhard Dinhof [email protected]

* fix(Configuration): Fix the default configuration to be compatible with for the debugging case where the Load Generator is run locally outside a docker-container

* fix(Configuration): Fix the default configuration to be compatible with for the debugging case where the Load Generator is run locally outside a docker-container

* #63 debug acapy (#64)

feat(Debug AcaPy): Running "manage.sh debug" it is now possible to attach a debugger to the issuer-verifier-acapy.

* Feat 59: credential revocation (#65)

feat(Full Flow): Support for credential revocation with configurable revocation batch size

* Fix 66 macos compsev2 compatibility (#67)

fix: Switches Postgres from port 5000 to port 5432 as MacOS Monterey already uses the post 5000 itself and extends the README to outline that Compose V2 is not supported.

* feat(Debugging): Configure via the .env if the load-generator should be started when running "manage.sh debug"

* feat(Debugging): Configure via the .env if the load-generator should be started when running "manage.sh debug"

* feat(Debugging): Configure via the .env if the load-generator should be started when running "manage.sh debug"

* fix(Debugging): Load generator needs to be started manually after the debugger has been attached to the issuer-verifier-acapy.

* docs: Improve the troubleshooting section

* fix(Database Logger): Fix the configuration of the database exporter to the metrics of the wallet DB

* chore(Multitenancy): Be able to run load tests with 9 tenants

* feat(Problem Reports): Log problem reports and show them on the dashboard

* chore(README): Add list of required dependencies

* feat(Dashboards): Update the dashboards and add a dashboard to evaluate AcaPy logs

* fix(Aries Client Logging): Ensure that aries client errors can be parsed by Loki

* feat(manage.sh): Wait for the ledger to start until the DID was registered successfully to the ledger

* feat(Apple M1 Chip): Update the dockerfile to be able to build the Load Generator Docker image on an Apple M1 Mac

* feat(.env): Update the default .env values

* chore(OOB): Remove out-of-band (OOB) placeholder code

---------

Signed-off-by: Gerhard Dinhof [email protected]
Co-authored-by: PaulWen <[email protected]>
Co-authored-by: Richard Holzeis <[email protected]>
Co-authored-by: Lukas Willburger <[email protected]>
Co-authored-by: Paul Wenzel <[email protected]>
Co-authored-by: iliecirciumaru <[email protected]>