Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature pulsar monitoring #11339

Merged
merged 22 commits into from
Oct 31, 2023
Merged

Feature pulsar monitoring #11339

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
ec1f450
draft
liangyepianzhou Sep 14, 2023
162eb07
test
liangyepianzhou Oct 4, 2023
ada8018
test
liangyepianzhou Oct 6, 2023
6cf69d6
test
liangyepianzhou Oct 7, 2023
538de1d
mysql
liangyepianzhou Oct 9, 2023
1b41304
rebase
liangyepianzhou Oct 29, 2023
9070343
service
liangyepianzhou Oct 29, 2023
4a70a4e
port
liangyepianzhou Oct 29, 2023
e9c1767
note
liangyepianzhou Oct 29, 2023
fe3f45e
Merge branch 'master' into pulsar
wu-sheng Oct 29, 2023
e2c4be8
address some comments
liangyepianzhou Oct 29, 2023
c6b8040
address some comments
liangyepianzhou Oct 30, 2023
54a7a2c
unit
liangyepianzhou Oct 30, 2023
0ed1da6
Merge branch 'master' into pulsar
wu-sheng Oct 30, 2023
3d62f94
fix
liangyepianzhou Oct 30, 2023
1a57d8b
Merge remote-tracking branch 'xiangying/pulsar' into pulsar
liangyepianzhou Oct 30, 2023
b8a3d52
fix
liangyepianzhou Oct 30, 2023
2acf377
Update docs/en/changes/changes.md
wu-sheng Oct 30, 2023
b7a61d6
fix
liangyepianzhou Oct 30, 2023
3393224
fix
liangyepianzhou Oct 31, 2023
bfb532c
fix
liangyepianzhou Oct 31, 2023
4ca25bc
fix expect file
liangyepianzhou Oct 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/en/setup/backend/backend-bookkeeper-monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ the OAP. In each cluster, the nodes are represented as `Instance`.
| Bookie Ledger Writable Dirs | meter_bookkeeper_bookie_ledger_writable_dirs | The number of writable directories in the bookie. | Bookkeeper Cluster |
| Bookie Ledger Dir Usage | meter_bookkeeper_bookie_ledger_dir_data_bookkeeper_ledgers_usage | The number of successfully created connections. | Bookkeeper Cluster |
| Bookie Entries Count | meter_bookkeeper_bookie_entries_count | The number of the bookie write entries. | Bookkeeper Cluster |
| Bookie Write Cache Size | meter_bookkeeper_bookie_write_cache_size | The size of the bookie write cache. | Bookkeeper Cluster |
| Bookie Write Cache Size | meter_bookkeeper_bookie_write_cache_size | The size of the bookie write cache (MB). | Bookkeeper Cluster |
| Bookie Write Cache Entry Count | meter_bookkeeper_bookie_write_cache_count | The entry count in the bookie write cache. | Bookkeeper Cluster |
| Bookie Read Cache Size | meter_bookkeeper_bookie_read_cache_size | The size of the bookie read cache. | Bookkeeper Cluster |
| Bookie Read Cache Size | meter_bookkeeper_bookie_read_cache_size | The size of the bookie read cache (MB). | Bookkeeper Cluster |
| Bookie Read Cache Entry Count | meter_bookkeeper_bookie_read_cache_count | The entry count in the bookie read cache. | Bookkeeper Cluster |
| Bookie Read Rate | meter_bookkeeper_bookie_read_rate | The bookie read rate. | Bookkeeper Cluster |
| Bookie Write Rate | meter_bookkeeper_bookie_write_rate | The bookie write rate. | Bookkeeper Cluster |
| Bookie Read Rate | meter_bookkeeper_bookie_read_rate | The bookie read rate (MB/min). | Bookkeeper Cluster |
| Bookie Write Rate | meter_bookkeeper_bookie_write_rate | The bookie write rate (MB/min). | Bookkeeper Cluster |
Copy link
Member

@wankai123 wankai123 Oct 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MAL rate means: rate(Duration): Calculates the per-second average rate of increase in the time range.
The rate('PT1M') does not mean per-min. It's per-second increase in 1 min


### BookKeeper Node Supported Metrics

Expand All @@ -48,7 +48,7 @@ the OAP. In each cluster, the nodes are represented as `Instance`.
| JVM Threads | meter_bookkeeper_node_jvm_threads_current <br /> meter_bookkeeper_node_jvm_threads_daemon <br /> meter_bookkeeper_node_jvm_threads_peak <br /> meter_bookkeeper_node_jvm_threads_deadlocked | The count of the jvm threads. | Bookkeeper Bookie |
| GC Time | meter_bookkeeper_node_jvm_gc_collection_seconds_sum | Time spent in a given JVM garbage collector in seconds. | Bookkeeper Bookie |
| GC Count | meter_bookkeeper_node_jvm_gc_collection_seconds_count | The count of a given JVM garbage. | Bookkeeper Bookie |
| Thread Executor | meter_bookkeeper_node_thread_executor_completed | The count of the executor thread. | Bookkeeper Bookie |
| Thread Executor Completed | meter_bookkeeper_node_thread_executor_completed | The count of the executor thread. | Bookkeeper Bookie |
| Thread Executor Tasks | meter_bookkeeper_node_thread_executor_tasks_completed <br /> meter_bookkeeper_node_thread_executor_tasks_rejected <br /> meter_bookkeeper_node_thread_executor_tasks_failed | The count of the executor tasks. | Bookkeeper Bookie |
| Pooled Threads | meter_bookkeeper_node_high_priority_threads <br /> meter_bookkeeper_node_read_thread_pool_threads | The count of the pooled thread. | Bookkeeper Bookie |
| Pooled Threads Max Queue Size | meter_bookkeeper_node_high_priority_thread_max_queue_size <br /> meter_bookkeeper_node_read_thread_pool_max_queue_size | The count of the pooled threads max queue size. | Bookkeeper Bookie |
Expand Down
8 changes: 4 additions & 4 deletions ...er-starter/src/main/resources/ui-initialized-templates/bookkeeper/bookkeeper-cluster.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_bookkeeper_bookie_read_rate"
"meter_bookkeeper_bookie_read_rate/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
Expand All @@ -314,7 +314,7 @@
},
"widget": {
"name": "bookie_read_rate",
"title": "Bookie Read Rate"
"title": "Bookie Read Rate(MB/min)"
}
},
{
Expand All @@ -330,7 +330,7 @@
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_bookkeeper_bookie_write_rate"
"meter_bookkeeper_bookie_write_rate/1024/1024"
],
"metricConfig": [],
"graph": {
Expand All @@ -346,7 +346,7 @@
},
"widget": {
"name": "bookie_write_rate",
"title": "Bookie Write Rate"
"title": "Bookie Write Rate(MB/min)"
},
"typesOfMQE": [
"TIME_SERIES_VALUES"
Expand Down
4 changes: 2 additions & 2 deletions ...erver-starter/src/main/resources/ui-initialized-templates/bookkeeper/bookkeeper-node.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@
},
"widget": {
"name": "jvm_gc_collection_seconds_sum",
"title": "GC Time (ms/min)"
"title": "GC Time (s)"
}
},
{
Expand Down Expand Up @@ -225,7 +225,7 @@
],
"widget": {
"name": "thread_executor_completed",
"title": "Thread Executor"
"title": "Thread Executor Completed"
}
},
{
Expand Down
3 changes: 1 addition & 2 deletions ...erver-starter/src/main/resources/ui-initialized-templates/bookkeeper/bookkeeper-root.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"children": [
{
"x": 0,
"y": 0,
"y": 3,
"w": 24,
"h": 29,
"i": "0",
Expand All @@ -17,7 +17,6 @@
"metrics": [
""
],
"moved": false,
"graph": {
"type": "ServiceList",
"dashboardName": "BookKeeper-Cluster",
Expand Down
112 changes: 50 additions & 62 deletions ...ver/server-starter/src/main/resources/ui-initialized-templates/pulsar/pulsar-cluster.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,17 +34,15 @@
"tips": "Total number of topics.",
"name": "total_topics"
},
"expressions": [],
"expressions": [
"meter_pulsar_total_topics"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-7",
"moved": false,
"metrics": [
"meter_pulsar_total_topics"
],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [],
"metricTypes": [],
"metricConfig": []
},
{
Expand All @@ -67,18 +65,16 @@
"tips": "Total number of subscriptions.",
"name": "total_subscriptions"
},
"expressions": [],
"expressions": [
"meter_pulsar_total_subscriptions"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-8",
"moved": false,
"metricConfig": [],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [
"meter_pulsar_total_subscriptions"
]
"metricTypes": [],
"metrics": []
},
{
"x": 12,
Expand All @@ -100,18 +96,16 @@
"tips": "Total number of producers.",
"name": "total_producers"
},
"expressions": [],
"expressions": [
"meter_pulsar_total_producers"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-9",
"moved": false,
"metricConfig": [],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [
"meter_pulsar_total_producers"
]
"metricTypes": [],
"metrics": []
},
{
"x": 18,
Expand All @@ -133,18 +127,16 @@
"tips": "Total number of consumers.",
"name": "total_consumers"
},
"expressions": [],
"expressions": [
"meter_pulsar_total_consumers"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-10",
"moved": false,
"metricConfig": [],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [
"meter_pulsar_total_consumers"
]
"metricTypes": [],
"metrics": []
},
{
"x": 0,
Expand All @@ -166,9 +158,11 @@
"tips": "Incoming message rate.",
"name": "message_rate_in"
},
"expressions": [],
"expressions": [
"meter_pulsar_message_rate_in"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-11",
"moved": false,
"metricConfig": [],
Expand Down Expand Up @@ -199,18 +193,16 @@
"tips": "Outgoing message rate.",
"name": "message_rate_out"
},
"expressions": [],
"expressions": [
"meter_pulsar_message_rate_in"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-12",
"moved": false,
"metricConfig": [],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [
"meter_pulsar_message_rate_in"
]
"metricTypes": [],
"metrics": []
},
{
"x": 0,
Expand All @@ -232,18 +224,16 @@
"tips": "Incoming throughput.",
"name": "throughput_in"
},
"expressions": [],
"expressions": [
"meter_pulsar_throughput_in"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-13",
"moved": false,
"metricConfig": [],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [
"meter_pulsar_throughput_in"
]
"metricTypes": [],
"metrics": []
},
{
"x": 12,
Expand All @@ -265,18 +255,16 @@
"tips": "Outgoing throughput.",
"name": "throughput_out"
},
"expressions": [],
"expressions": [
"meter_pulsar_throughput_in"
],
"typesOfMQE": [],
"metricMode": "General",
"metricMode": "Expression",
"id": "16-0-14",
"moved": false,
"metricConfig": [],
"metricTypes": [
"readLabeledMetricsValues"
],
"metrics": [
"meter_pulsar_throughput_in"
]
"metricTypes": [],
"metrics": []
},
{
"x": 12,
Expand Down Expand Up @@ -306,7 +294,7 @@
},
"widget": {
"name": "storage_read_rate",
"title": "Storage Read Rate"
"title": "Storage Read Rate (messages per second)"
}
},
{
Expand Down Expand Up @@ -337,7 +325,7 @@
},
"widget": {
"name": "storage_logical_size",
"title": "Storage Logical Size"
"title": "Storage Logical Size (MB)"
}
},
{
Expand Down Expand Up @@ -368,7 +356,7 @@
},
"widget": {
"name": "storage_write_rate",
"title": "Storage Write Rate"
"title": "Storage Write Rate (messages per second)"
}
},
{
Expand Down Expand Up @@ -402,7 +390,7 @@
},
"widget": {
"name": "storage_size",
"title": "Storage Size"
"title": "Storage Size (MB)"
},
"typesOfMQE": []
}
Expand Down
3 changes: 1 addition & 2 deletions ...server/server-starter/src/main/resources/ui-initialized-templates/pulsar/pulsar-root.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"children": [
{
"x": 0,
"y": 0,
"y": 3,
"w": 24,
"h": 29,
"i": "0",
Expand All @@ -17,7 +17,6 @@
"metrics": [
""
],
"moved": false,
"graph": {
"type": "ServiceList",
"dashboardName": "Pulsar-Cluster",
Expand Down
Loading