Skip to content

Conversation

Copy link

Copilot AI commented Sep 22, 2025

This PR implements a comprehensive Python script to interact with the HPCC Platform GetGlobalMetrics service in ws_machines, based on the existing cppunit tests in ws_machineService.cpp.

Overview

The new tools/get_global_metrics.py script provides a command-line interface for querying global metrics from HPCC Platform with support for:

  • Time range filtering: Query metrics for specific date/time ranges
  • Category filtering: Filter by specific metric categories or get all categories
  • Dimension filtering: Filter by dimension name/value pairs (user, cluster, etc.)
  • Statistics selection: Extract only specified statistics from results
  • Multiple output formats: JSON, human-readable table, and CSV

Implementation Details

The script creates XML requests that match the format expected by the GetGlobalMetrics service:

<GetGlobalMetricsRequest>
    <Category>categoryOne</Category>
    <Dimensions>
        <Dimension><Name>user</Name><Value>alice</Value></Dimension>
    </Dimensions>
    <DateTimeRange>
        <Start>1999-01-01T00:00:00</Start>
        <End>2099-01-01T00:00:00</End>
    </DateTimeRange>
</GetGlobalMetricsRequest>

And parses the XML responses into structured Python data for easy processing.

Usage Examples

# Get all metrics with time range filtering
python3 get_global_metrics.py --host localhost --port 8010 \
    --start "2023-01-01T00:00:00" --end "2023-12-31T23:59:59"

# Filter by category and dimensions
python3 get_global_metrics.py --host localhost --port 8010 \
    --category "categoryOne" --dimension user:alice

# Extract specific statistics as JSON
python3 get_global_metrics.py --host localhost --port 8010 \
    --stats TimeLocalExecute CostExecute --format json

Testing and Quality Assurance

The implementation includes comprehensive unit tests (testing/unittests/test_get_global_metrics.py) that verify:

  • XML request generation with all parameter combinations
  • XML response parsing using actual test data from the cppunit tests
  • Statistics filtering and output formatting
  • Error handling for edge cases

All tests pass and validate the script works correctly with the expected XML formats.

Future Extensibility

The script is designed to be easily extended with aggregation functionality in the next round of changes. An example aggregation script (tools/example_aggregation.py) demonstrates how to build on this foundation to:

  • Aggregate statistics by category or dimension
  • Identify top performers by specific metrics
  • Perform basic time series analysis
  • Generate summary reports

Files Added

  • tools/get_global_metrics.py - Main Python client script
  • tools/example_aggregation.py - Example showing future aggregation capabilities
  • tools/README.md - Comprehensive documentation and usage examples
  • testing/unittests/test_get_global_metrics.py - Unit tests

The implementation provides a solid foundation for the planned aggregation features while being immediately useful for querying and analyzing HPCC Platform global metrics.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Based on the new GetGlobalMetricsResponse service call in ws_machines, and based on example usage in new cppunit tests in ws_machineService.cpp, create a python script that interacts with this service and pull select stats for a time range. NB: the fun... Add Python client for GetGlobalMetrics service with time range and statistics filtering Sep 22, 2025
Copilot AI requested a review from jakesmith September 22, 2025 11:53
Copilot finished work on behalf of jakesmith September 22, 2025 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants