Rate Limiter

[jamesdoc]

BullMQ offers the ability to set a rate limiter on a worker. Eg:

const worker = new Worker(
  "getFromApiQueue",
  async job => processJob(job),
  {
    connection: redisOpts,
    // call no more than once a minute
    limiter: { 
      max: 1,
      duration: 60000,
    },
  }
);

From the documentation here I'm not clear how to pass this configuration onto BullMQ. Is this possible at the moment?

[RomainLanz]

Hey @jamesdoc! 👋🏻

You can define this configuration inside the config/queue.ts file.

defineConfig({
  worker: {
    limiter: {
      // ...
    },
  },
})

[jamesdoc]

Ahh, that makes sense and works well. Thank you @RomainLanz, however that appears to apply globally across all workers.

Is it then possible to apply different limiters to different workers? (eg for different API calls with different rate limits)

Eg - this simple proof of concept works as I expected it to outside of Adonis

const worker30s = new Worker(
  "oneInThirty",
  async job => processJob(job),
  {
    connection: redisOpts,
    // One job every 30 seconds
    limiter: {
      max: 1,
      duration: 30000,
    },
  }
);

const worker10s = new Worker(
  "oneInTen",
  async job => processJob(job),
  {
    connection: redisOpts,
    // One job every 10 seconds
    limiter: {
      max: 1,
      duration: 10000,
    },
  }
);

[RomainLanz]

It is not doable at the moment, I may add the possibility to define queue worker directly in the configuration, allowing you to configure then from here.

I need to think about it.

[jamesdoc]

Thanks Romain. I appreciate the thought and response.