Skip to content

Enumerating partitions has 1ms fixed cost #1595

@fbac

Description

@fbac

Scanning a partition has plan fixed cost of 1ms, and in many the queries, such as SelectGatewayEnvelopesByTopics, the planner has to consider all partitions. Scales linearly and becomes unbearable, as the system reaches >~ 500 partitions easily.

This issue becomes a bottleneck specially in SelectGatewayEnvelopesByTopics - selecting by topic, as we can't predict the originator id where an specific topic will fall, so no partition pruning applies.
Which is aggravated by the fact libxmtp most used backend query is QueryEnvelopes by topic.

Consider strategies:

  • materialized view
  • application side cache
  • prepared statements with plan_cache_mode

Metadata

Metadata

Assignees

No one assigned

    Labels

    service/databaseItems related to the xmtpd database

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions