Skip to content

[Bug] BE IO metrics are missing when storing data on nvme devices #48350

@xiacongling

Description

@xiacongling

Search before asking

  • I had searched in the issues and found no similar issues.

Version

2.1.7

What's Wrong?

When deploying Doris BE on machines with NVME disks, I/O metrics like doris_be_max_disk_io_util_percent or doris_be_disk_io_time_ms are all 0's.
After debugging, it is found that device labels in the metrics are like nvmeXnYp.

$ curl http://localhost:8040/metrics | grep "doris_be_disk_io_time_ms"
# TYPE doris_be_disk_io_time_ms counter
doris_be_disk_io_time_ms{device="nvme1n1p"} 0
doris_be_disk_io_time_ms{device="nvme5n1p"} 0
doris_be_disk_io_time_ms{device="nvme0n1p"} 0
doris_be_disk_io_time_ms{device="nvme3n1p"} 0
doris_be_disk_io_time_ms{device="nvme6n1p"} 0
doris_be_disk_io_time_ms{device="nvme7n1p"} 0
doris_be_disk_io_time_ms{device="nvme4n1p"} 0
doris_be_disk_io_time_ms{device="nvme2n1p"} 0

while the real device names are like nvmeXnY or nvmeXnYpZ.

$ cat /proc/partitions
major minor  #blocks  name

   8        0  468320256 sda
   8        1     488264 sda1
   8        2  467830784 sda2
 253        0   20971520 dm-0
 253        1   12582912 dm-1
 253        2  434274304 dm-2
 259        0 1875374424 nvme0n1
 259        1 1875373056 nvme0n1p1
 259        2 1875374424 nvme1n1
 259        3 1875373056 nvme1n1p1
 259        4 1875374424 nvme7n1
 259        5 1875373056 nvme7n1p1
 259        6 1875374424 nvme6n1
 259        7 1875373056 nvme6n1p1
 259        8 1875374424 nvme5n1
 259       12 1875373056 nvme5n1p1
 259        9 1875374424 nvme4n1
 259       13 1875373056 nvme4n1p1
 259       10 1875374424 nvme3n1
 259       14 1875373056 nvme3n1p1
 259       11 1875374424 nvme2n1
 259       15 1875373056 nvme2n1p1

What You Expected?

expose I/O metrics correctly for NVME devices

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions