Develop a Synthetic Monitoring Platform Build Project

With a projected compound annual growth rate of 15.3% from 2023 to 2028, synthetic monitoring is a rapidly growing global industry. That figure reflects an age where incidents such as the July 2024 worldwide outage of Windows systems caused by a faulty and recklessly-released CrowdStrike sensor configuration update can cause billions in uninsured damages. As such, in our world today, active monitoring is a high-stakes discipline of difficult-to-understate importance. Over this 8-week "Build Project" associated with the Open Avenues Build Fellowship and led by Build Fellow Sonu Gupta, I developed a synthetic monitoring platform that can monitor the performance of an application or system by pinging servers for time-series data from the command line or Flask UI and relaying them through Prometheus to a Grafana dashboard for visualization.

System Components:

YAML Configuration File Parser

• Python module which reads & parses YAML configuration files to return as a Python dictionary.
• Detects top-level keys & common configuration types based on YAML structure when ran independently.
• Handles various error scenarios such as file not found, invalid YAML syntax, etc.

Network Connectivity Monitor

• Python module which leverages 'pingparsing' library to monitor network connectivity.
• Takes targets, probes, and interval from user input, pings server(s) after given interval, parses results.
• Initializes Prometheus metrics for each target server and each latency/packet loss metric to expose.
  • Metrics Exposed to Prometheus:
  • Packet Transmit/Receive/Loss/Duplicate Count
  • Round Trip Time Average/Best/Worst
  • Round Trip Time Mean Deviation (Jitter)
  • Packet Loss Rate
  • Packet Duplicate Rate
• Extracts & present key metrics on console with confirmation of updating Prometheus metrics.
  • Metrics Displayed to CLI/GUI: RTT Average/Best/Worst, Packet Loss/Duplicate Rate, Jitter

Integrated Modules @ main.py

• Main Python script which integrates the YAML file parser and network connectivity monitor modules.
• Starts HTTP server (in prometheus.yml) on port 8989.
• Takes command line argument for YAML file path (else default provided).
• Reads YAML file and fetches a list of servers and pings each target server after given interval.
• Displays ping metrics on console while setting Prometheus metrics with ping_monitor.display_and_expose_results().

Flask Frontend Entry Point & Output Stream

• Python module which integrates Flask microframework to create a frontend for the monitoring platform.
• Accepts YAML path and starts program while redirecting output to a persistent log accessed from homepage.
• Clear previous messages and run on new config with one click.

Prometheus & Grafana

• Prometheus: Open source monitoring system & alerting tool configured to scrape metrics from the ping monitor.
• Grafana: Open source observability platform & visualization tool with configured time-series data dashboards.

Example Dashboard Panels:

Flask UI:

Getting Started

Pre-requisites: Prometheus & Grafana binaries, Python 3.8+

• Clone the repository.
• [Optional] Add your YAML configuration file to configs/ directory.
• Add HTTP server port to Prometheus configuration.
• Navigate to Prometheus directory & run Prometheus: ./prometheus --config.file=prometheus.yml
• Navigate to Grafana directory & run Grafana: ./bin/grafana-server
• Open and login to Grafana at http://localhost:3000 with a browser.
• Add Prometheus http://localhost:9090 as a data source in Grafana.
• Navigate to root directory of cloned repository.
• Install project dependencies:

  pip install -r requirements.txt

• CLI: Run main script in main.py & pass YAML file path with target servers (default provided if no config argument):
  • Note: In certain environments use python3

  python src/main.py configs/example.yml

• Flask UI: Run frontend.py in flask_frontend directory & navigate to the development server at http://127.0.0.1:5000 in a browser.

  flask --app src/flask_frontend/frontend.py run

• If providing custom YAML file (not default provided variety.yaml), format as:

global_settings:
  probes: 4
  interval: 1
  port: 8989

targets:
  - 8.8.8.8
  - example.com
  - another.target.io