Skip to content

[Bug] BE IO metrics are missing when storing data on nvme devices #48350

Open
@xiacongling

Description

@xiacongling

Search before asking

  • I had searched in the issues and found no similar issues.

Version

2.1.7

What's Wrong?

When deploying Doris BE on machines with NVME disks, I/O metrics like doris_be_max_disk_io_util_percent or doris_be_disk_io_time_ms are all 0's.
After debugging, it is found that device labels in the metrics are like nvmeXnYp.

$ curl http://localhost:8040/metrics | grep "doris_be_disk_io_time_ms"
# TYPE doris_be_disk_io_time_ms counter
doris_be_disk_io_time_ms{device="nvme1n1p"} 0
doris_be_disk_io_time_ms{device="nvme5n1p"} 0
doris_be_disk_io_time_ms{device="nvme0n1p"} 0
doris_be_disk_io_time_ms{device="nvme3n1p"} 0
doris_be_disk_io_time_ms{device="nvme6n1p"} 0
doris_be_disk_io_time_ms{device="nvme7n1p"} 0
doris_be_disk_io_time_ms{device="nvme4n1p"} 0
doris_be_disk_io_time_ms{device="nvme2n1p"} 0

while the real device names are like nvmeXnY or nvmeXnYpZ.

$ cat /proc/partitions
major minor  #blocks  name

   8        0  468320256 sda
   8        1     488264 sda1
   8        2  467830784 sda2
 253        0   20971520 dm-0
 253        1   12582912 dm-1
 253        2  434274304 dm-2
 259        0 1875374424 nvme0n1
 259        1 1875373056 nvme0n1p1
 259        2 1875374424 nvme1n1
 259        3 1875373056 nvme1n1p1
 259        4 1875374424 nvme7n1
 259        5 1875373056 nvme7n1p1
 259        6 1875374424 nvme6n1
 259        7 1875373056 nvme6n1p1
 259        8 1875374424 nvme5n1
 259       12 1875373056 nvme5n1p1
 259        9 1875374424 nvme4n1
 259       13 1875373056 nvme4n1p1
 259       10 1875374424 nvme3n1
 259       14 1875373056 nvme3n1p1
 259       11 1875374424 nvme2n1
 259       15 1875373056 nvme2n1p1

What You Expected?

expose I/O metrics correctly for NVME devices

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions