Add ActiveMQ classic monitoring #12109

CzyerChen · 2024-04-15T16:56:46Z

#12063

Update the documentation to include this new feature.
Tests(including UT, IT, E2E) are added to verify the new feature.
ActiveMQ single-node/ 2 single-nodes/ shared-file-master-slave have been tested.
If it's UI related, attach the screenshots below.
Update the CHANGES log.

For ActiveMQ Classic cluster:

System Load: range in [0, 100].
Thread Count: the number of threads currently used by the JVM.
Heap Memory: capacity of heap memory.
GC: memory of ActiveMQ is managed by Java's garbage collection (GC) process. Compatible with JDK 1.6-17 with ActiveMQ classic(5.10.x ~ 6.1.0)
Enqueue/Dequeue/Dispatch/Expired Rate: growth rate of messages in different states.
Average/Max Enqueue Time: time taken to join the queue.

For ActiveMQ Classic broker:

Uptime: duration of the node.
State: 1 = slave node, 0 = master node.
Current Connentions: number of connections.
Current Producer/Consumer Count: number of current producers/consumers.
Increased Producer/Consumer Count: number of increased producers/consumers.
Enqueue/Dequeue Count: number of enqueue and dequeue.
Enqueue/Dequeue Rate: rate of enqueue and dequeue.
Memory Percent Usage: amount of memory space used by undelivered messages.
Store Percent Usage: space used by pending persistent messages.
Temp Percent Usage: space used by non-persistent messages.
Average/Max Message Size: number of messages.
Queue Size: number of messages in the queue.

For ActiveMQ Classic destination:

Produser/Consumer Count: number of producers/Consumers.
Queue Size: unacknowledged messages of the queue.
Memory usage: usage of memory.
Enqueue/Dequeue/Dispatch/Expired/Inflight Count: number of messages in different states.
Average/Max Message Size: number of messages.
Average/Max Enqueue Time: time taken to join the queue.

Due to the shortcomings of the activemq-5.x agent plugin, it can not suit hierarchy definition now.
I have to fix the agent part later, then push the hierarchy definition.

...erver-starter/src/main/resources/ui-initialized-templates/activemq/activemq-destination.json

wu-sheng · 2024-04-16T01:44:34Z

The UI submodule seems to be changed unexpectedly?

...er/server-starter/src/main/resources/ui-initialized-templates/activemq/activemq-cluster.json

wu-sheng · 2024-04-16T01:45:38Z

Another question is, what is the status of service hierarchy about this new active MQ monitoring?

CzyerChen · 2024-04-16T02:32:47Z

Another question is, what is the status of service hierarchy about this new active MQ monitoring?

now:
virtual_mq service name: 192.168.208.2:61616
agent peer: 192.168.208.2:61616
dashboard service name: activemq.skywaling-showcase.svc.local

Currently, it is not compatible with building service hierarchy by using servicename.

wu-sheng · 2024-04-16T02:56:39Z

That rule should base on k8s deployment. Not VM.

CzyerChen · 2024-04-16T03:46:55Z

The UI submodule seems to be changed unexpectedly?

Reverted.

test/e2e-v2/cases/activemq/docker-compose.yml

CzyerChen · 2024-04-16T13:32:23Z

That rule should base on k8s deployment. Not VM.

ActiveMQ agent side set Tags.MQ_BROKER = raw IP address:port, rather than hostname:port

in K8S:
ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port ) : support service hierarchy
VIRTUAL_MQ( ip:port ) -> ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) : can not support service hierarchy
ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) -> K8S_SERVICE(skywalking-showcase::activemq.skywalking-showcase): support service hierarchy

wu-sheng · 2024-04-16T13:33:55Z

@wankai123 Do we have IP-service name resolver to support in hierarchy detection? I think that would be useful.

wu-sheng · 2024-04-16T13:39:44Z

That rule should base on k8s deployment. Not VM.

ActiveMQ agent side set Tags.MQ_BROKER = raw IP address:port, rather than hostname:port

in K8S: ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port ) : support service hierarchy VIRTUAL_MQ( ip:port ) -> ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) : can not support service hierarchy ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) -> K8S_SERVICE(skywalking-showcase::activemq.skywalking-showcase): support service hierarchy

So, the name of the ActiveMQ service is hard coded only?

wu-sheng · 2024-04-16T13:42:08Z

@CzyerChen As when ActiveMQ is deployed in k8s, OTEL collector should be able to use sidecar mode to get service name aligned with target service's name, and instance name as pod name.

@weixiang1862 Do you know how to set up sidecar?

…mq-monitoring

wu-sheng · 2024-04-16T13:55:11Z

That rule should base on k8s deployment. Not VM.

ActiveMQ agent side set Tags.MQ_BROKER = raw IP address:port, rather than hostname:port

in K8S: ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port ) : support service hierarchy VIRTUAL_MQ( ip:port ) -> ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) : can not support service hierarchy ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) -> K8S_SERVICE(skywalking-showcase::activemq.skywalking-showcase): support service hierarchy

About this, in k8s,

VIRTUAL_MQ( ip:port ) could be service name, is the activeMQ plugin(Java agent) resolving the Domain name and gets the IP?
ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) should be k8s service name, right?

wankai123 · 2024-04-16T13:57:03Z

@CzyerChen

ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port )

Where is the ACTIVEMQ-SERVER(ip:port) data from？

CzyerChen · 2024-04-16T13:59:28Z

That rule should base on k8s deployment. Not VM.

ActiveMQ agent side set Tags.MQ_BROKER = raw IP address:port, rather than hostname:port
in K8S: ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port ) : support service hierarchy VIRTUAL_MQ( ip:port ) -> ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) : can not support service hierarchy ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) -> K8S_SERVICE(skywalking-showcase::activemq.skywalking-showcase): support service hierarchy

About this, in k8s,

VIRTUAL_MQ( ip:port ) could be service name, is the activeMQ plugin(Java agent) resolving the Domain name and gets the IP?

ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) should be k8s service name, right?

yes

CzyerChen · 2024-04-16T14:00:41Z

@CzyerChen

ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port )

Where is the ACTIVEMQ-SERVER(ip:port) data from？

Data reported by activeMQ plugin(Java agent).

wu-sheng · 2024-04-16T14:09:21Z

That rule should base on k8s deployment. Not VM.

ActiveMQ agent side set Tags.MQ_BROKER = raw IP address:port, rather than hostname:port
in K8S: ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port ) : support service hierarchy VIRTUAL_MQ( ip:port ) -> ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) : can not support service hierarchy ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) -> K8S_SERVICE(skywalking-showcase::activemq.skywalking-showcase): support service hierarchy

About this, in k8s,

VIRTUAL_MQ( ip:port ) could be service name, is the activeMQ plugin(Java agent) resolving the Domain name and gets the IP?

ACTIVEMQ cluster monitoring(activemq.skywalking-showcase) should be k8s service name, right?

yes

Let's separate this in two ways.

Service hierarchy is still working if you are building ActiveMQ server monitoring with K8s service monitoring. The hierarchy between those two isn't affected by the agent use case.
About the agent is reporting the service IP, is this a good practice? I am thinking this because usually virtual MQ should be a logic concept, if the MQ is restarted by k8s, the IP could change, but DNS name would not. Nowadays, k8s deployment is very popular, and it is the key use case of the hierarchy feature, we only need to focus on that. How about add remote.ip tag for the actual IP, and use peer for domain name?

wankai123 · 2024-04-16T14:15:55Z

@CzyerChen

ACTIVEMQ-SERVER(ip:port) -> VIRTUAL_MQ( ip:port )

Where is the ACTIVEMQ-SERVER(ip:port) data from？

Data reported by activeMQ plugin(Java agent).

So, the ACTIVEMQ-SERVER should be an instance rather than a service？ If not, who does this IP belong to, a k8s service?

CzyerChen · 2024-04-16T15:46:36Z

part2 agent side needs to upgrade

I think we could put the rules here once for all? As currently, the agent side has a totally different release period. The next(9.3) agent release could be 3 months later.

- VIRTUAL_MQ -> ACTIVEMQ may have to check cluster mode, rather than single node. if client connected by failover:(tcp://amq1:61616,tcp://amq2:61617), it may hard to continue service hierarchy.

wu-sheng · 2024-04-16T15:51:24Z

- VIRTUAL_MQ -> ACTIVEMQ may have to check cluster mode, rather than single node. if client connected by failover:(tcp://amq1:61616,tcp://amq2:61617), it may hard to continue service hierarchy.

How could this be in k8s? I think K8s should be DNS oriented right? Even with a cluster, client should use service address like activemq-svr.namespace.cluster.

VM deployment is complex, we are hard to make service hierarchy working in that deployment out-of-box.

docs/en/swip/readme.md

wu-sheng · 2024-04-17T01:14:09Z

@CzyerChen How about the latest question?

CzyerChen · 2024-04-17T03:44:21Z

- VIRTUAL_MQ -> ACTIVEMQ may have to check cluster mode, rather than single node. if client connected by failover:(tcp://amq1:61616,tcp://amq2:61617), it may hard to continue service hierarchy.

How could this be in k8s? I think K8s should be DNS oriented right? Even with a cluster, client should use service address like activemq-svr.namespace.cluster.

VM deployment is complex, we are hard to make service hierarchy working in that deployment out-of-box.

Normally we use failover transport to connect server node on client side, like failover:(tcp://broker1.namespace.cluster:61616,tcp://broker2.namespace.cluster:61616,tcp://broker3.namespace.cluster:61616)?transportOptions in K8S.

In Master-Slave or Broker cluster mode, If just configure one node of Broker on client side, the failover will not work automatically.

wu-sheng · 2024-04-17T03:50:27Z

Are you saying this is recommended deployment for ActiveMQ on k8s?
No unified domain namr for its cloud native deployment by adopting k8s DNS resolving mechanism?

wu-sheng · 2024-04-17T03:53:08Z

Typically in k8s, two or more deployment could be as one service, and failover could be done internally and natively.

CzyerChen · 2024-04-17T05:40:14Z

Are you saying this is recommended deployment for ActiveMQ on k8s? No unified domain namr for its cloud native deployment by adopting k8s DNS resolving mechanism?

It needs to list the addresses of all brokers in the cluster now in ActiveMQ Classic(no specific description for K8S deployment). But after introducing JGroups auto-discovery from Artemis, the client no longer needs to list the addresses of all brokers in the cluster, only need the logical address of the cluster.

For the Classic, there is no nameserver to provide a unified connection portal.

wu-sheng · 2024-04-17T06:40:42Z

OK, @wankai123 Let us skip that.

docs/en/setup/backend/backend-activemq-monitoring.md

…mq-monitoring

This reverts commit d9238c9.

...ver/server-starter/src/main/resources/ui-initialized-templates/activemq/activemq-broker.json

wankai123 · 2024-04-18T03:35:03Z

The others are LGTM.

# Conflicts: # docs/en/changes/changes.md

…mq-monitoring # Conflicts: # docs/en/changes/changes.md

wu-sheng

lgtm

wankai123 · 2024-04-19T09:02:57Z

@CzyerChen, Hi, I found some expressions didi not add service_instance_id in activemq-cluster.yaml, is that expected?

CzyerChen · 2024-04-19T09:48:21Z

@CzyerChen, Hi, I found some expressions didi not add service_instance_id in activemq-cluster.yaml, is that expected?

In Master-Slave mode, cluster metrics only report from Master node, but JVM metrics will report from all nodes. So JVM metrics add service_instance_id, cluster metrics not.

wu-sheng requested review from wankai123 and weixiang1862 April 16, 2024 00:31

wu-sheng added this to the 10.0.0 milestone Apr 16, 2024

wu-sheng added backend OAP backend related. feature New feature labels Apr 16, 2024

wankai123 reviewed Apr 16, 2024

View reviewed changes

...erver-starter/src/main/resources/ui-initialized-templates/activemq/activemq-destination.json Outdated Show resolved Hide resolved

wankai123 reviewed Apr 16, 2024

View reviewed changes

...er/server-starter/src/main/resources/ui-initialized-templates/activemq/activemq-cluster.json Outdated Show resolved Hide resolved

revert skywalking-ui.

b28d8dd

CzyerChen force-pushed the activemq-monitoring branch from f5a0c50 to b28d8dd Compare April 16, 2024 03:44

Merge branch 'master' into activemq-monitoring

a8ba1ce

Fine0830 mentioned this pull request Apr 16, 2024

Fix the Table widget apache/skywalking-booster-ui#389

Merged

weixiang1862 reviewed Apr 16, 2024

View reviewed changes

test/e2e-v2/cases/activemq/docker-compose.yml Show resolved Hide resolved

Merge branch 'master' into activemq-monitoring

cf943cb

CzyerChen added 2 commits April 16, 2024 21:42

update backend-activemq-monitoring.md

bfe6942

Merge remote-tracking branch 'origin/activemq-monitoring' into active…

78ffe4f

…mq-monitoring

wu-sheng reviewed Apr 16, 2024

View reviewed changes

docs/en/swip/readme.md Outdated Show resolved Hide resolved

update SWIP

8f7f3bc

Merge branch 'master' into activemq-monitoring

50eac50

wu-sheng reviewed Apr 18, 2024

View reviewed changes

docs/en/setup/backend/backend-activemq-monitoring.md Outdated Show resolved Hide resolved

CzyerChen added 3 commits April 18, 2024 10:40

Merge remote-tracking branch 'upstream/master' into activemq-monitoring

95617bc

update backend-activemq-monitoring.md

3223307

Merge remote-tracking branch 'origin/activemq-monitoring' into active…

c48e94e

…mq-monitoring

wu-sheng requested a review from wankai123 April 18, 2024 02:54

CzyerChen added 2 commits April 18, 2024 10:55

update backend-activemq-monitoring.md

d9238c9

Revert "update backend-activemq-monitoring.md"

b4a1b16

This reverts commit d9238c9.

wankai123 reviewed Apr 18, 2024

View reviewed changes

...ver/server-starter/src/main/resources/ui-initialized-templates/activemq/activemq-broker.json Outdated Show resolved Hide resolved

Merge branch 'master' into activemq-monitoring

955064b

CzyerChen added 3 commits April 18, 2024 11:37

update activemq-broker.json

67673bd

Merge remote-tracking branch 'upstream/master' into activemq-monitoring

0de06dc

# Conflicts: # docs/en/changes/changes.md

Merge remote-tracking branch 'origin/activemq-monitoring' into active…

64cd0f1

…mq-monitoring # Conflicts: # docs/en/changes/changes.md

wu-sheng approved these changes Apr 18, 2024

View reviewed changes

wankai123 approved these changes Apr 18, 2024

View reviewed changes

weixiang1862 approved these changes Apr 18, 2024

View reviewed changes

wu-sheng merged commit 7e97826 into apache:master Apr 18, 2024
153 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ActiveMQ classic monitoring #12109

Add ActiveMQ classic monitoring #12109

CzyerChen commented Apr 15, 2024 •

edited

Loading

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024 •

edited

Loading

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wankai123 commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wankai123 commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 17, 2024

CzyerChen commented Apr 17, 2024

wu-sheng commented Apr 17, 2024

wu-sheng commented Apr 17, 2024

CzyerChen commented Apr 17, 2024

wu-sheng commented Apr 17, 2024

wankai123 commented Apr 18, 2024

wu-sheng left a comment

wankai123 commented Apr 19, 2024

CzyerChen commented Apr 19, 2024

Add ActiveMQ classic monitoring #12109

Add ActiveMQ classic monitoring #12109

Conversation

CzyerChen commented Apr 15, 2024 • edited Loading

#12063

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024 • edited Loading

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wankai123 commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wankai123 commented Apr 16, 2024

CzyerChen commented Apr 16, 2024

wu-sheng commented Apr 16, 2024

wu-sheng commented Apr 17, 2024

CzyerChen commented Apr 17, 2024

wu-sheng commented Apr 17, 2024

wu-sheng commented Apr 17, 2024

CzyerChen commented Apr 17, 2024

wu-sheng commented Apr 17, 2024

wankai123 commented Apr 18, 2024

wu-sheng left a comment

Choose a reason for hiding this comment

wankai123 commented Apr 19, 2024

CzyerChen commented Apr 19, 2024

CzyerChen commented Apr 15, 2024 •

edited

Loading

wu-sheng commented Apr 16, 2024 •

edited

Loading