Nutanix Prism Prometheus Exporter

About

The Nutanix Exporter is a Go application that fetches live data from any number of Prism Element servers and presents it in a format ingestible by Prometheus. It runs as a docker container, automatically fetching all PE clusters from a Prism Central instance and exporting metrics from multiple APIv2 endpoints (VMs, Hosts, Clusters, etc.).

Features

• YAML config files define which metrics to collect
• Hashicorp Vault support for fetching cluster credentials
• Refreshes credentials from Vault on 4xx errors
• Support for reading cluster credentials from environment variables
• Parent Exporter class that can be extended for any APIv2 endpoint
• Per cluster metrics exposed at /metrics/cluster-name
• Optional filtering by cluster name prefix

Getting Started

Prerequisites

• Nutanix Prism Central 2023.4 or later
• When using HashiCorp Vault: A Vault server with KVv2 Secrets Engine enabled
  • Secrets Engine name: defined in VAULT_ENGINE_NAME environment variable
  • Secret name: defined in PE_TASK_ACCOUNT and PC_TASK_ACCOUNT environment variables
  • Namespace: Optional, but can be defined in VAULT_NAMESPACE environment variable
  • Fields: username, secret
• When using environment variables for providing the credentials:

  • The VAULT_ADDR environment variable must not be defined

  • PC_USERNAME and PC_PASSWORD environment variables must be defined with Prism Central credentials

  • For each cluster the PE_USERNAME_<CLUSTERNAME> and PE_PASSWORD_<CLUSTERNAME> environment variables have to be defined with Prism Element credentials

  • The cluster specific environment variable names can only contain letters A-Z, numbers 0-9 and underscores (_). Lower case letters should be converted to upper case and all other characters to underscores.

    Example
    If your cluster name is cluster.name or cluster-name, then the environment variables must be written as:

    • PE_USERNAME_CLUSTER_NAME
    • PE_PASSWORD_CLUSTER_NAME

Metrics Configuration

Metrics are collected from the Prism Element v2.0 APIs. Currently, the exporter supports the following endpoints:

• Clusters
• Hosts
• VMs
• Storage Containers

Additionally, the v1 VM API is supported for gathering additional stats not available in the v2 API. The config file for this endpoint is vm_v1.yaml.

The response from the API contains a list of entities, each with a set of key-value pairs. The exporter will flatten these key-value pairs and expose them as Prometheus metrics.

/config contains a YAML configuration file for each exporter. This is where the metrics to be collected are defined. Any value in the API response can be collected; however, the exporter will only collect metrics that are defined in the configuration file. It is important to note that nested fields in the API response are flattened and exposed like "parent_child", e.g. "stats_num_iops".

Each entry must have the following fields:

• name: The name of the metric key in the API response
• help: User defined description of the metric

- name: memory_mb
  help: Memory in MB.
- name: power_state
  help: Power state of the VM.
- name: vcpu_reservation_hz
  help: vCPU reservation in Hz.
- name: stats_num_iops
  help: Number of IOPS. (Example of a nested key where stats is the parent in the response)

Default configuration files are provided for each APIv2 endpoint. These can be overwritten when running the exporter by mounting a new configuration file into the container as seen in the deployment section.

Running the Exporter

While the exporter is designed to run in a containerized environment, it can also be run natively on a host. The following instructions will guide you through both methods. For production environments, the exporter should always be run in a container. However, for development and testing, running the Go binary natively is generally easier.

For both methods, start by cloning the repository and navigating to the repo root in your terminal.

To build and run the Go binary natively:

1. Download and install the Go Programming Language
2. Export all necessary environment variables
3. go run cmd/nutanix_exporter/main.go
4. The exporter will now be running on localhost:9408

To build and run in a container:

1. Download and install Docker from here
2. docker build -t nutanix_exporter .
3. docker run -p 9408:9408 --env-file configs/exporter.env nutanix_exporter
4. The exporter will now be running on localhost:9408

Example exporter.env:

### Common options
PC_CLUSTER_NAME='Prism Central' (Required, can be any value, letters a-z,A-Z, numbers 0-9, underscores (_), space ( ) and dash (-))
PC_CLUSTER_URL=https://your-pc-cluster.yourdomain.com:9440 (Required, the full URL to Prism Central)
PC_API_VERSION=v3 (Optional, defaults to v4. Supports v3, v4b1, v4)
CLUSTER_REFRESH_INTERVAL=1800 (Optional, defaults to 30 minutes, value is in seconds)
CLUSTER_PREFIX=optional-cluster-prefix (Optional, prefix to filter cluster names)
CONFIG_PATH=/etc/prometheus-nutanix-exporter/ (Optional, defaults to `./configs`)

### For HashiCorp Vault only
VAULT_ADDR=https://your-vault-server.yourdomain.com
VAULT_NAMESPACE=production
VAULT_ENGINE_NAME=NutanixKV2
VAULT_ROLE_ID=12345678-1234-5678-1234-567812345678
VAULT_SECRET_ID=12345678-1234-5678-1234-567812345678
VAULT_REFRESH_INTERVAL=1500 (Seconds. Optional, defaults to 30 minutes)
PE_TASK_ACCOUNT=PETaskAccount
PC_TASK_ACCOUNT=PCTaskAccount

### For environment variable credential provider only
PE_USERNAME_<CLUSTERNAME_ONE>=cluster1-user-name
PE_PASSWORD_<CLUSTERNAME_ONE>=cluster1-user-password
PE_USERNAME_<CLUSTERNAME_TWO_>=cluster2-user-name
PE_PASSWORD_<CLUSTERNAME_TWO>=cluster2-user-password

Deployment

Example docker-compose.yml:

services:
  NutanixExporter:
    image: "your_container_registry/nutanix_exporter:latest"
    restart: unless-stopped
    volumes:
      - /path/to/your/configs/cluster.yaml:configs/cluster.yaml:z
      - /path/to/your/configs/storage_container.yaml:configs/storage_container.yaml:z
      - /path/to/your/configs/host.yaml:configs/host.yaml:z
      - /path/to/your/configs/vm.yaml:configs/vm.yaml:z
    env_file:
      - /path/to/your/configs/exporter.env
    ports:
      - '9408:9408'

Built With

• Go - Programming language
• Go Prometheus Client - Prometheus client library for Go
• Go Hashicorp Vault Client - Hashicorp Vault client library for Go
• Docker - Containerization
• GitHub Actions - CI/CD pipeline

Community

We do not have any dedicated Slack channel for this project, however the GitHub Discussions feature is enabled for this repository. Feel free to ask questions, share ideas, or discuss the project.

Contributing

Please read CONTRIBUTING for more details about making a contribution to this open source project and ensure that you follow our CODE_OF_CONDUCT.

Contact

If you have any other issues or questions regarding this project, feel free to contact one of the CODEOWNERS for a more in-depth discussion.

Licence

This project is open source under the "GPLv2 License", read the LICENCE terms for more details.