[feat] Network Costs Test

[Manas23601]

Description

Network Costs

• Receive Bytes
• Transfer Bytes

Prometheus Query

sum(increase(container_network_receive_bytes_total{pod!=""}[24h:5m])) by (pod, namespace)`

sum(increase(container_network_transmit_bytes_total{pod!="", %s}[%s:%dm])) by (pod_name, pod, namespace, %s)

Testing

Network Costs

md@Manas:/mnt/c/Manas/opencost-integration-tests$ go test -v ./test/integration/prometheus/network_costs_test.go
=== RUN   TestNetworkCosts
    network_costs_test.go:34: testCases: [{Yesterday 24h namespace false}]
=== RUN   TestNetworkCosts/Yesterday
    network_costs_test.go:180: Namespace argo
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 2113274703.39
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 3601434433.67
    network_costs_test.go:180: Namespace gpu-operator
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 3951830782.66
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 3303167969.23
    network_costs_test.go:180: Namespace cert-manager
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 51280593.62
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 35189496.75
    network_costs_test.go:180: Namespace ingress-nginx
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 77230121.14
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 69102378.82
    network_costs_test.go:180: Namespace load-generator
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 7790042667.26
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 20252712121.09
    network_costs_test.go:180: Namespace folding-at-home
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 389527427.79
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 591013869.49
    network_costs_test.go:180: Namespace sealed-secrets
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 9175452.21
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 8833045.30
    network_costs_test.go:180: Namespace prometheus-system
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 29482772937.37
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 48800928242.51
    network_costs_test.go:180: Namespace kube-system
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 104585511444.96
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 164773304552.70
    network_costs_test.go:180: Namespace opencost
    network_costs_test.go:185:   - NetworkTransferBytes[Pass]: ~ 174612718.16
    network_costs_test.go:190:   - NetworkReceiveBytes[Pass]: ~ 463782783.89
--- PASS: TestNetworkCosts (2.09s)
    --- PASS: TestNetworkCosts/Yesterday (2.09s)
PASS
ok      command-line-arguments  2.098s

Network Internet Costs

=== RUN   TestNetworkInternetCosts
    network_internet_costs_test.go:33: testCases: [{Yesterday 24h pod false}]
=== RUN   TestNetworkInternetCosts/Yesterday
    network_internet_costs_test.go:167: Pod test-install-speedtest-tracker-56944bbf5b-9lpd4
    network_internet_costs_test.go:172:   - NetworkInternetCost[Pass]: ~ 0.52375
--- PASS: TestNetworkInternetCosts (2.78s)
    --- PASS: TestNetworkInternetCosts/Yesterday (2.78s)
PASS
ok      command-line-arguments  2.785s

[mbolt35]

Just wanted to drop a quick note about network costing. When we calculate network costs, we do not use the NetworkTransferBytes or NetworkReceiveBytes -- It's not a bad test to run for comparison, but as it relates to cost, we do not use these statistics for costs because they include all sub-categories of traffic (ie: local traffic, which is free).

I'm not sure if the network-costs pods are installed on this cluster, so we should definitely ensure that gets done as soon as possible.

[mbolt35]

The actual costs and usage data we use is here: https://github.com/opencost/opencost/blob/ef740759c88732487bffa644509480b070ca77c5/modules/prometheus-source/pkg/prom/metricsquerier.go#L907-L1021

But note that this requires a network-costs daemonset be running on the cluster (not sure if this is currently running on that cluster - if not, these metrics won't show up)

[ameijer]

validCostsSeen := false

[Manas23601]

@ameijer , if network zone and region tests look good, I'll go ahead anc comment those out in test.bats

[ameijer]

@Manas23601 looks like the tests for zone and region are failing? for those, it is ok if we don't see any valid values

[ameijer]

why is this commented?

[Manas23601]

I wanted the option to drill down to a pod level if needed. do you think we need to go down further down?

[Manas23601]

@Manas23601 looks like the tests for zone and region are failing? for those, it is ok if we don't see any valid values

[Manas23601]

@Manas23601 looks like the tests for zone and region are failing? for those, it is ok if we don't see any valid values

These are meant to fail because there is no valid zone and region costs. We should either not fail the tests if there are no valid costs or not run this test at all