Flow-Control And Global Parallelism Metrics

[sancar]

Currently it is hard for our users to understand what is going on with Flow-Control/Global Parallellism.

Did it hit the limit?
How many messages are waiting because of the limit?
What is the affective Flow-Control Rate/Parallelism/Period ?

We are revising the Flow-Control implementation. The plan is to improve the get flow-control endpoint as a start to help users more

GET /v2/flow-control

Current version:

{
  "flowControlKey": "<string>",
  "waitlistSize": 123,
}

Improved version:

{
  "flowControlKey": "<string>", //  "$" for the messages without flow control. 
  "waitlistSize": 123,
  "parallelism": 10, // If not exists, it means not limited by parallelism
  "parallelismCount": 5, // 5 tasks are running in parallel at the moment
  "rateMax": 10,       // If not exists, it means not limited by rate
  "rateCount" : 4,  //  4 tasks are started within the last period
  "ratePeriod": "1d", 
  "ratePeriodStart": 1708000000 // Unix timestamp when the current rate period started
}

To access new fields via SDK -> https://upstash.com/docs/qstash/features/flowcontrol#get-a-single-flow-control-key
There is a special $ wait list on this list that reports the waitlist size for the messages that doesn't have any related flow control key and just waiting for global parallelism.
To access global parallelism info via SDK -> https://upstash.com/docs/qstash/features/flowcontrol#get-global-parallelism

You can access all these via rest api as well here: https://upstash.com/docs/qstash/api-refence/flow-control/list-flow-control-keys

Flow Control tab of QStash Console also shows all these data together with the historic global parallelism graph. And there is more detailed version of global parallelism graph on the Usage tab of the QStash Console.

[ctrlaltdylan]

Yes real time is essential for diagnosing any current delays.

Historic metrics are critical for understanding how current parallelism (global or user defined) are affecting perf.

Without this users are in the dark about how parallelism is affecting their perf.

Please make sure this applies to the "global" parallel limit as well, the API example above seems to rely on a flow control key. Not sure if that needs to be omitted to see the global queue, or if there's a global reserved name for the global parallel limit.

[sancar]

We will probably have a second endpoint to check the global but with similar metrics. For global, I wasn't planning to keep a rate info but just parallelism, because that is the only limit we have on global.

/v2/globalParallelism (the endpoint name can change)

{
  "waitlistSize": 123,
  "parallelism": 10, // account level parallelism decided by the plan. 
  "parallelismCounter": 5, // 5 tasks are running in parallel at the moment
}

[ctrlaltdylan]

Got it.

Those three metrics should be enough to at least get a "balance sheet" perspective of the accounts current perf.

[ctrlaltdylan]

Is there any update on this?

[ctrlaltdylan]

Any updates on this?

[sancar]

Also if you go to Flow Control tab of QStash Console, you can see all this data as well together with the historic global parallelism graph. And there is more detailed version of global parallelism graph on the Usage tab of the QStash Console.

[ctrlaltdylan]

Thanks @sancar that's great news. I didn't see an announcement about this.

I'll try out the API, but the dashboard metrics don't seem to make sense. It's showing no data.

I'll start a support ticket to share the more detailed information, but this screenshot is what I'm seeing (I'm looking at the correct region too).

[sancar]

| Thanks @sancar that's great news. I didn't see an announcement about this.

This is pretty new, we are still furnishing docs. Announcement will be done soon.

| I'll start a support ticket to share the more detailed information,

Sure, access via support so that we can look into details of your account.

[ctrlaltdylan]

I think I see the issue, this Flow Control UI doesn't support global parallelism tracking.

I expected it to also show historic metrics on the global, since that's the core issue.

[sancar]

It tracks the count of concurrently running tasks globally. You can see if you hit the global limit for past month or not.
There is no historic data for total wait list size for tasks hitting the global limit.
You can also see the current wait list size of for each flow control key or for the tasks that has no flow control associated with them ($(unbound)).

Are these enough for your use case ?
Do you need to historic data of waitListSize for the tasks that hit the global parallelism ?