documentation/hugo/content/en/containers/monitoring/kubernetes_explorer.md at 35beb1c251496bbb81c04f9f4746db0058e47955 · DataDog/documentation

title

Kubernetes Explorer

description

Using Datadog's Kubernetes Explorer page to monitor your Kubernetes resources, such as pods and deployments.

aliases

/infrastructure/containers/orchestrator_explorer

further_reading

link	tag	text
https://www.datadoghq.com/blog/kubernetes-operator-performance	Blog	Monitor your Kubernetes operators to keep applications running smoothly

link	tag	text
https://learn.datadoghq.com/courses/getting-started-k8s	Learning Center	Getting Started with Kubernetes Observability

{{< img src="infrastructure/livecontainers/orch_ex.png" alt="Kubernetes Explorer, showing Kubernetes Pods." style="width:80%;">}}

Datadog's Kubernetes Explorer allows you to monitor the state of pods, deployments, and other Kubernetes resources. You can also view resource specifications for failed pods within a deployment, correlate node activity with related logs, track resource utilization, automatically scale workloads, and remediate errors.

When using the Datadog Agent, Kubernetes Explorer requires Agent 7.27.0+ and Cluster Agent 1.11.0+. If you are using Kubernetes 1.25+, then Cluster Agent 7.40.0+ is required.

Configuration

Enable Kubernetes Explorer

Kubernetes Explorer is enabled by default for most Datadog Agent installations.

When you install the Datadog Agent by using the Datadog Operator, Kubernetes Explorer is enabled by default.

To verify that Kubernetes Explorer is enabled, ensure that the features.orchestratorExplorer.enabled parameter is set to true in your datadog-agent.yaml:

apiVersion: datadoghq.com/v2alpha1
kind: DatadogAgent
metadata:
  name: datadog
spec:
  global:
    clusterName: <CLUSTER_NAME>
    credentials:
      apiKey: <DATADOG_API_KEY>
      appKey: <DATADOG_APP_KEY>
  features:
    orchestratorExplorer:
      enabled: true

When you install the Datadog Agent by using the official Helm chart, Kubernetes Explorer is enabled by default.

To verify that Kubernetes Explorer is enabled, ensure that the orchestratorExplorer.enabled parameter is set to true in your datadog-values.yaml file:

datadog:
  clusterName: <CLUSTER_NAME>
  # (...)
  processAgent:
    enabled: true
  orchestratorExplorer:
    enabled: true

Then, upgrade your Helm chart.

{{% /tab %}} {{% tab "Manual" %}} For manual setup, see Set up Kubernetes Explorer with a DaemonSet.

You can populate the Kubernetes Explorer using a native OpenTelemetry pipeline instead of the Datadog Agent. This setup uses the k8sobjects receiver to collect Kubernetes resource data and forwards it through the Datadog Exporter's orchestrator explorer functionality.

The following steps enable Explorer's resource views without collecting the metrics used by related dashboards. To collect those metrics and populate Explorer, follow Kubernetes Metrics with OpenTelemetry instead.

This feature is not available for {{< region-param key="dd_site_name" >}}.

Prerequisites

OpenTelemetry Collector Contrib v0.154.0 or later.
OpenTelemetry Collector Helm chart v0.156.2 or later.

Limitations

The open source k8sobjects receiver can place significant load on a cluster's Kubernetes API server.

Recommendations:

Use Kubernetes 1.33 or later, which includes streaming list improvements that reduce API server impact.
Start with smaller clusters. Limit the number of objects per resource type to fewer than 5,000 as a starting point, and scale up gradually while monitoring cluster health.

1. Create a Datadog API key secret

Create a Kubernetes secret to store your Datadog API key. These steps use the default namespace for both the secret and the Collector:

export DD_API_KEY="<YOUR_DATADOG_API_KEY>"
kubectl create secret generic datadog-secret \
  --namespace default \
  --from-literal="api-key=$DD_API_KEY"

2. Configure the cluster collector

Create deployment-collector.yaml with the following complete Helm values. Replace <YOUR_DATADOG_SITE> with your Datadog site and <YOUR_CLUSTER_NAME> with your cluster name.

This configuration runs one Collector as a Deployment. It sets the cluster name explicitly, so cloud-provider detection is not required.

mode: deployment
replicaCount: 1

image:
  repository: otel/opentelemetry-collector-contrib
  tag: 0.154.0
  pullPolicy: IfNotPresent

extraEnvs:
  - name: DD_API_KEY
    valueFrom:
      secretKeyRef:
        name: datadog-secret
        key: api-key

presets:
  kubernetesObjects:
    enabled: true
    watch: true

config:
  receivers:
    k8sobjects:
      interval: 3m

  processors:
    resourcedetection:
      detectors: [k8s_api]
      override: false
    resource/add-cluster-name:
      attributes:
        - key: k8s.cluster.name
          value: "<YOUR_CLUSTER_NAME>"
          action: upsert

  exporters:
    datadog:
      api:
        site: "<YOUR_DATADOG_SITE>"
        key: ${env:DD_API_KEY}
      orchestrator_explorer:
        enabled: true

  service:
    pipelines:
      logs:
        receivers: [k8sobjects]
        processors: [resourcedetection, resource/add-cluster-name]
        exporters: [datadog]

The kubernetesObjects preset configures the receiver, service account, and RBAC permissions. Keep the 3m collection interval and the k8s_api detector, which identifies the cluster UID. The logs pipeline sends Kubernetes resource objects to Explorer; it does not collect application logs.

Automatic cluster name detection (optional)

If you prefer automatic cluster name detection, make these changes in deployment-collector.yaml before deploying:

Add your provider's detector to resourcedetection.detectors, keeping k8s_api. Follow the configuration and permissions guidance for EKS, AKS, or GKE, including enabling the k8s.cluster.name resource attribute.
Remove resource/add-cluster-name from both config.processors and the logs pipeline's processors list.

3. Deploy with Helm

Install the OpenTelemetry Collector using your configuration file:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

helm install deployment-collector open-telemetry/opentelemetry-collector \
  --namespace default \
  --values ./deployment-collector.yaml

4. Verify the installation

Open the Kubernetes Explorer and filter by your OpenTelemetry cluster name. All core Kubernetes resource sections should populate, along with Custom Resources > CRD. The Custom Resources > Resources section is not supported with this setup.

5. Correlate logs, metrics, and traces with Kubernetes Explorer (optional)

This step applies to Collectors that receive application telemetry, not the Explorer-only Collector above. To correlate that telemetry with Kubernetes resources, add the k8sattributes and resourcedetection processors to those Collectors' pipelines. Use the same cluster name as the Explorer Collector.

The following fragment shows the processor configuration. Keep the existing receivers, exporters, and processors in your application-telemetry pipelines; replace ... with the other processors in each pipeline.

processors:
  k8sattributes:
    auth_type: "serviceAccount"
    extract:
      metadata:
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.deployment.name
        - k8s.namespace.name
        - k8s.node.name
        - k8s.replicaset.name
        - k8s.statefulset.name
        - k8s.daemonset.name
        - k8s.cronjob.name
        - k8s.job.name
        - k8s.container.name
    pod_association:
      - sources:
          - from: resource_attribute
            name: k8s.pod.uid
      - sources:
          - from: resource_attribute
            name: k8s.pod.ip
      - sources:
          - from: resource_attribute
            name: k8s.pod.name
          - from: resource_attribute
            name: k8s.namespace.name
      - sources:
          - from: connection

service:
  pipelines:
    logs:
      processors: [k8sattributes, resourcedetection, ...]
    metrics:
      processors: [k8sattributes, resourcedetection, ...]
    traces:
      processors: [k8sattributes, resourcedetection, ...]

For a complete application-telemetry Collector example, see the DaemonSet collector configuration.

You can populate the Kubernetes Explorer using the opentelemetry-kube-stack Helm chart instead of the Datadog Agent.

The opentelemetry-kube-stack Helm chart installs the OpenTelemetry Operator and manages collectors as OpenTelemetryCollector custom resources (CRs). Datadog maintains a reference values.yaml that configures two collectors:

cluster (Deployment): Scrapes kube-state-metrics, watches Kubernetes objects, and enables orchestrator_explorer to populate Kubernetes Explorer.
daemon (DaemonSet): Collects host and kubelet metrics, and exposes an OTLP endpoint for application telemetry data.

This feature is not available for {{< region-param key="dd_site_name" >}}.

Prerequisites

OpenTelemetry Kube Stack Helm chart 0.20.1 or later.
OpenTelemetry Collector Contrib v0.154.0 or later (pinned by the reference values file).
cert-manager, which is required for the operator's admission webhook.

Limitations

The open source k8sobjects receiver can place significant load on a cluster's Kubernetes API server.

Recommendations:

Use Kubernetes 1.33 or later, which includes streaming list improvements that reduce API server impact.
Start with smaller clusters. Limit the number of objects per resource type to fewer than 5,000 as a starting point, and scale up gradually while monitoring cluster health.

Quickstart (interactive installer)

The opentelemetry-examples repository ships an interactive installer that handles all of the steps below. From guides/kubernetes/configuration/opentelemetry-kube-stack/:

./install

The installer prompts for your Datadog API key, Datadog site, Kubernetes platform, and deployment environment. For EKS, GKE, and AKS it enables the matching resource-detection preset. For other platforms it prompts for the cluster name. It then creates the opentelemetry-operator-system namespace and datadog-secret, installs cert-manager if needed, and installs or upgrades the chart.

Install with values files

If you did not use the interactive installer above, follow the steps below to install manually.

1. Install cert-manager (if not already present)

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

2. Create the Datadog secret

Set DD_SITE to your Datadog site (defaults to datadoghq.com):

export DD_API_KEY="<YOUR_DATADOG_API_KEY>"
export DD_SITE="datadoghq.com"  # for example us3.datadoghq.com, datadoghq.eu

kubectl create namespace opentelemetry-operator-system \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret generic datadog-secret \
  --namespace opentelemetry-operator-system \
  --from-literal="api-key=$DD_API_KEY" \
  --from-literal="dd-site=$DD_SITE" \
  --dry-run=client -o yaml | kubectl apply -f -

3. Create a deployment overlay

The reference values.yaml is the base; deployment-specific settings (cluster platform, environment, cluster name) live in an overlay file. From guides/kubernetes/configuration/opentelemetry-kube-stack/, copy the example that matches your platform:

mkdir -p deployment

# EKS, GKE, or AKS (resource detector auto-populates k8s.cluster.name):
cp examples/eks-deployment/values.yaml deployment/values.yaml
cp examples/gcp-deployment/values.yaml deployment/values.yaml
cp examples/aks-deployment/values.yaml deployment/values.yaml

# Other platforms (set the cluster name manually):
cp examples/manually-set-k8s-cluster-name/values.yaml deployment/values.yaml

For non-EKS/GKE/AKS platforms, edit deployment/values.yaml and replace my_k8s_cluster and production with your cluster name and deployment environment.

4. Deploy the reference collectors

Install or upgrade the chart with both the base values.yaml and your overlay:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

helm upgrade --install opentelemetry-kube-stack \
  open-telemetry/opentelemetry-kube-stack \
  --namespace opentelemetry-operator-system \
  --values ./values.yaml \
  --values ./deployment/values.yaml

Both collectors default to limits of 500m CPU and 1Gi memory, and requests of 200m CPU and 500Mi memory. Scale up for large clusters.

Verify the installation

Open the Kubernetes Explorer and filter by your cluster name. All core Kubernetes resource sections should populate, along with Custom Resources > CRD. The Custom Resources > Resources section is not supported with this setup.

Add custom tags to resources

To ease filtering, you can add custom tags to your Kubernetes resources through the DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS environment variable. These tags only appear in Kubernetes Explorer.

Set the DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS environment variable twice in datadog-agent.yaml:

In agents.containers.processAgent.env
In clusterAgent.env

apiVersion: datadoghq.com/v2alpha1
kind: DatadogAgent
metadata:
  name: datadog
spec:
  global:
    credentials:
      apiKey: <DATADOG_API_KEY>
      appKey: <DATADOG_APP_KEY>
  features:
    liveContainerCollection:
      enabled: true
    orchestratorExplorer:
      enabled: true
  override:
    agents:
      containers:
        processAgent:
          env:
            - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
              value: "tag1:value1 tag2:value2"
    clusterAgent:
      env:
        - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
          value: "tag1:value1 tag2:value2"

Then, apply the new configuration:

kubectl apply -n $DD_NAMESPACE -f datadog-agent.yaml

Set the DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS environment variable twice in datadog-agent.yaml:

In processAgent.env
In clusterAgent.env

agents:
  containers:
    processAgent:
      env:
        - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
          value: "tag1:value1 tag2:value2"
clusterAgent:
  env:
    - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
      value: "tag1:value1 tag2:value2"

Then, upgrade your Helm chart.

Set the environment variable on both the Process Agent and Cluster Agent containers:

- name: DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS
  value: "tag1:value1 tag2:value2"

Usage

Views

Toggle among the {{< ui >}}Pods{{< /ui >}}, {{< ui >}}Clusters{{< /ui >}}, {{< ui >}}Namespaces{{< /ui >}}, and other Kubernetes resources in the {{< ui >}}Select Resources{{< /ui >}} dropdown menu in the top left corner of the page.

Each of these views includes a data table to help you better organize your data by field such as status, name, and Kubernetes labels, and a detailed Cluster Map to give you a bigger picture of your pods and Kubernetes clusters.

See Query filter details for more details on how to filter these views.

{{< img src="infrastructure/livecontainers/orch_ex_replicasets.png" alt="Orchestrator Explorer opened to show Workloads > Replica Sets, in Summary mode" style="width:80%;">}}

Group by functionality and facets

Group pods by tags, Kubernetes labels, or Kubernetes annotations to get an aggregated view which allows you to find information quicker. You can perform a group by using the "Group by" bar on the top right of the page or by clicking on a particular tag or label and locating the group by function in the context menu as shown below.

{{< img src="infrastructure/livecontainers/orch_ex_groupby.png" alt="An example of grouping by team" style="width:80%;">}}

You can also use facets on the left hand side of the page to group resources or filter for resources you care most about, such as pods with a CrashLoopBackOff pod status.

{{< img src="infrastructure/livecontainers/crashloopbackoff.mp4" alt="An example of grouping the CrashLoopBackOff pod status" video=true style="width:80%;">}}

Cluster map

A cluster map gives you a bigger picture of your pods and Kubernetes clusters. You can see all of your resources together on one screen with customized groups and filters, and choose which metrics to fill the color of the nodes.

Examine resources from cluster maps by clicking on any circle or group to populate a detailed panel.

{{< img src="infrastructure/livecontainers/cluster-map.mp4" alt="A cluster map with customized groups and filters" video=true style="width:80%;">}}

Information panel

Click on any row in the table or on any object in a Cluster Map to view information about a specific resource in a side panel.

{{< img src="infrastructure/livecontainers/orch_ex_panel.png" alt="A view of resources in the side panel, opened to processes." style="width:80%;">}}

The side panel's {{< ui >}}YAML{{< /ui >}} tab shows the full resource definition. Starting in Agent version 7.44.0, it also includes seven days of definition history. You can compare what changed over time and across different versions. The time indicated is approximately when the changes were applied to the resource.

To prevent displaying a large number of irrelevant changes, updates affecting only the following fields are ignored:

metadata.resourceVersion
metadata.managedFields
metadata.generation
metadata.annotations["kubernetes.io/config.seen"]
status

{{< img src="infrastructure/livecontainers/orch_ex_manifest_history.png" alt="A view of resources in the side panel, showing the yaml history feature" style="width:80%;">}}

The other tabs show more information for troubleshooting the selected resource:

Logs: View logs from your container or resource. Click on any log to view related logs in the Log Explorer.
APM: View traces from your container or resource, including the date, service, duration, method, and status code of a trace.
Metrics: View live metrics for your container or resource. You can view any graph full screen, share a snapshot of it, or export it from this tab.
{{< ui >}}Processes{{< /ui >}}: View all processes running in the container of this resource.
{{< ui >}}Network{{< /ui >}}: View a container or resource's network performance, including source, destination, sent and received volume, and throughput fields. Use the {{< ui >}}Destination{{< /ui >}} field to search by tags like DNS or ip_type, or use the {{< ui >}}Group by{{< /ui >}} filter in this view to group network data by tags, like pod_name or service.
Events: View all Kubernetes events for your resource.
{{< ui >}}Monitors{{< /ui >}}: View monitors tagged, scoped, or grouped for this resource.

For a detailed dashboard of this resource, click the View Dashboard in the top right corner of this panel.

{{< img src="infrastructure/livecontainers/view-pod-dashboard.png" alt="A link to a pod dashboard from Live Containers overview" style="width:80%;">}}

Resource utilization

For the Resource Utilization page, see Resource Utilization.

Within the Kubernetes Explorer tab, you can explore a selection of resource utilization metrics.

{{< img src="infrastructure/livecontainers/orch_ex_resource_utilization.png" alt="Container Resource Utilization" style="width:80%;">}}

All of these columns support sorting, which helps you to pinpoint individual workloads based on their resource utilization.

{{< img src="infrastructure/livecontainers/orch_ex_resource_utilization_sorted_column.png" alt="Container Resource Utilization Sorted Columns" style="width:50%;">}}

Query filter details

You can narrow down the displayed resources by supplying a query within the "Filter by" search bar on the top left of the page.

Syntax

A query filter is composed of terms and operators. Example:

{{< img src="infrastructure/livecontainers/orch_syntax.png" alt="Orchestrator Explorer query filter syntax." style="width:80%;">}}

Terms

There are multiple types of terms available:

Type	Examples
Tags: Attached to resources by the agent collecting them. There are also additional tags that Datadog generates for Kubernetes resources.	`datacenter:staging`, `tag#datacenter:staging` (the `tag#` is optional)
Labels: Extracted from a resource's metadata. They are typically used to organize your cluster and target specific resources with selectors.	`label#chart_version:2.1.0`
Annotations: Extracted from a resource's metadata. They are generally used to support tooling that aid in cluster management.	`annotation#checksum/configmap:a1bc23d4`
Metrics: Added to workload resources (pods, deployments, etc.). You can find resources based on their utilization. To see what metrics are supported, see Resource Utilization Filters.	`metric#cpu_usage_pct_limits_avg15:>80%`
String matching: Supported by some specific resource attributes, see below. Note: string matching does not use the key-value format, and you cannot specify the attribute to match on.	`"10.132.6.23"` (IP), `"9cb4b43f-8dc1-4a0e"` (UID), `web-api-3` (Name)
Fields: Extracted from a resource's metadata or from custom resources' indexed fields.	`field#metadata.creationTimestamp:>=4wk`, `field#metadata.deletionTimestamp:<=1hr`, `field#status.currentReplicas:3`, `field#status.conditions.Active.status:True`

Note: You might find the same key-value pairs as both a tag and label (or annotation) - this is dependent on how your cluster is configured.

The following resource attributes are supported in arbitrary String Matching:

metadata.name
metadata.uid
IP Addresses found in:
- Pods
- Nodes (internal and external)
- Services (cluster, external, and load balancer IPs)

You do not need to specify a key to search for a resource by name, or IP. Quotes are not required unless your string search includes certain special characters.

Comparators

All terms support the : equality operator. Metric value terms support numeric comparisons as well:

:> Greater than (for example, metric#cpu_usage_avg15:>0.9)
:>= Greater than or equal
:< Less than
:<= Less than or equal

Operators

To combine multiple terms into a complex query, you can use any of the following case sensitive boolean operators:

Operator	Description	Example
`AND`	Intersection: Both terms are in the selected events (if nothing is added, AND is taken by default)	`a AND b`
`OR`	Union: Either term is contained in the selected events	`a OR b`
`NOT` / `-`	Exclusion: The following term is NOT in the event (apply to each individual raw text search)	`a AND NOT b` or `a AND -b`
`( )`	Grouping: Specify how to group terms logically.	`a AND (b OR c)` or `(a AND b) or c`

`OR` value shorthand

Multiple terms sharing the same key can be combined into a single term if they all use the OR operator. For example, this query:

app_name:web-server OR app_name:database OR app_name:event-consumer

Can be reduced to:

app_name:(web-server OR database OR event-consumer)

Wildcards

You can use * wildcards as part of a term to filter by partial matches, both for values and keys. Some examples:

kube_job:stats-*: Find all resources with a kube_deployment tag value starting with stats-.
pod_name:*canary: Find all resources with a pod_name value ending in canary.
label#release:*: Find all resources with a release label, regardless of its value.
-label#*.datadoghq.com/*: Find resources that do not have any Datadog scoped labels.
kube_*:*stats*canary: Find resources that have related resource tags (kube_*), with stats in the middle of the value, also ending with canary.

Extracted tags

In addition to the tags you have configured within your Datadog agent, Datadog injects generated tags based on resource attributes that can help your searching and grouping needs. These tags are added to resources conditionally, when they are relevant.

All resources

All resources have the kube_cluster_name tag and all namespaced resources have the kube_namespace tag added to them.

Additionally, resources contain a kube_<api_kind>:<metadata.name> tag. For example, a deployment named web-server-2 would have the kube_deployment:web-server-2 tag automatically added to it.

Note: There are some exceptions to this pattern:

Pods use pod_name instead.

VPAs: verticalpodautoscaler.

HPAs: horizontalpodautoscaler.

Persistent Volume Claims: persistentvolumeclaim.

Based on the labels attached to the resource, the following tags will also be extracted:

Tag	Source Label
`kube_app_name`	`app.kubernetes.io/name`
`kube_app_instance`	`app.kubernetes.io/instance`
`kube_app_version`	`app.kubernetes.io/version`
`kube_app_component`	`app.kubernetes.io/component`
`kube_app_part_of`	`app.kubernetes.io/part-of`
`kube_app_managed_by`	`app.kubernetes.io/managed-by`
`env`	`tags.datadoghq.com/env`
`version`	`tags.datadoghq.com/version`
`service`	`tags.datadoghq.com/service`

Relationships

Related Resources will be tagged with each other. Some examples:

A pod that is part of the "XYZ" deployment will have a kube_deployment:xyz tag.
An ingress that points at service "A" will have a kube_service:a tag.

Resources that are spawned from "parent" resources will have the kube_ownerref_kind and kube_ownerref_name tags (such as pods and jobs).

Tip: Utilize the filter query autocomplete feature to discover what related resource tags are available. Type kube_ and see what results are suggested.

Pods

Pods are given the following tags:

pod_name
pod_phase (extracted from the manifest)
pod_status (calculated similarly to kubectl)

Workloads

Workload resources (pods, deployments, stateful sets, etc.) will have the following tags, indicating their support within the Resources Utilization page:

resource_utilization (supported or unsupported)
missing_cpu_requests
missing_cpu_limits
missing_memory_requests
missing_memory_limits

Conditions

Some conditions, for some resources, are extracted as tags. For example, you can find the kube_condition_available tag on deployments. The tag format is always kube_condition_<name> with a true or false value.

Tip: Use the autocomplete feature to discover what conditions are available on a given resource type by entering kube_condition and reviewing the results.

Resource specific tags

Some resources have specific tags that are extracted based on your cluster's environment. The following tags are available in addition to the shared tags above.

Resource	Extracted Tags
Cluster	`api_server_version` `kubelet_version`
Custom Resource Definitions & Custom Resources	`kube_crd_kind` `kube_crd_group` `kube_crd_version` `kube_crd_scope` `kube_crd_resource`
Namespace	`phase`
Node	`kube_node_unschedulable` `kube_node_kubelet_version` `kube_node_kernel_version` `kube_node_runtime_version` `eks_fargate_node` `node_schedulable` `node_status`
Persistent Volume	`kube_reclaim_policy` `kube_storage_class_name` `pv_type` `pv_phase`
Persistent Volume Claim	`pvc_phase` `kube_storage_class_name`
Pod	`pod_name` (instead of `kube_pod`) `pod_phase` (extracted from the Manifest) `pod_status` (calculated similarly to `kubectl`)
Service	`kube_service_type` `kube_service_port`

Resource Utilization Filters

The following workload resources are enriched with resource utilization metrics:

Clusters
Nodes
Pods

These metrics are calculated at the time of collection, based on the average values over the last 15 minutes. You can filter by metric values like so: metric#<metric_name><comparator><numeric_value>.

metric_name is an available metric (see below)
comparator is a supported comparator
and numeric_value is a floating point value.

For Pods, the following metric names are available:

CPU	Memory
`cpu_limits_avg15`	`mem_limits_avg15`
`cpu_requests_avg15`	`mem_requests_avg15`
`cpu_usage_avg15`	`mem_usage_avg15`
`cpu_usage_pct_limits_avg15`	`mem_usage_pct_limits_avg15`
`cpu_usage_pct_requests_avg15`	`mem_usage_pct_requests_avg15`
`cpu_waste_avg15`	`mem_waste_avg15`

In addition, clusters, and nodes have the following metrics available to them:

cpu_usage_pct_alloc_avg15
cpu_requests_pct_alloc_avg15
mem_usage_pct_alloc_avg15
mem_requests_pct_alloc_avg15

Metric units

CPU metrics are stored as a number of cores.

Memory metrics are stored as bytes.

Percents (*_pct_*) are stored as floats, where 0.0 is 0%, and 1.0 is 100%. The value is the ratio of the two indicated metrics - for example cpu_usage_pct_limits_avg15 is the value of usage / limits. Metric values can be above 100%, such as Percentage CPU Usage of Requests.

Notes and known issues

Data is updated automatically in constant intervals.
In clusters with 1000+ Deployments or ReplicaSets you may notice elevated CPU usage from the Cluster Agent. There is an option to disable container scrubbing in the Helm chart. See the Helm Chart repo for more details.

Uh oh!

FilesExpand file tree

kubernetes_explorer.md

Latest commit

History

kubernetes_explorer.md

File metadata and controls

Configuration

Enable Kubernetes Explorer

Prerequisites

Limitations

1. Create a Datadog API key secret

2. Configure the cluster collector

Automatic cluster name detection (optional)

3. Deploy with Helm

4. Verify the installation

5. Correlate logs, metrics, and traces with Kubernetes Explorer (optional)

Prerequisites

Limitations

Quickstart (interactive installer)

Install with values files

1. Install cert-manager (if not already present)

2. Create the Datadog secret

3. Create a deployment overlay

4. Deploy the reference collectors

Verify the installation

Add custom tags to resources

Usage

Views

Group by functionality and facets

Cluster map

Information panel

Resource utilization

Query filter details

Syntax

Terms

Comparators

Operators

OR value shorthand

Wildcards

Extracted tags

All resources

Relationships

Pods

Workloads

Conditions

Resource specific tags

Resource Utilization Filters

Metric units

Notes and known issues

Further reading

`OR` value shorthand