Skip to content

[feat] Free sync memory after a run and cut sync CPU #1154

Description

@cenodude

Problem to solve

Idle memory stays high after a sync.
At 25k items the container sits at 278 MiB after syncing.
390 MiB once the Analyzer and dashboard pages have been opened, against 83 MiB at startup.
It is not a leak, but the memory never comes back.

Three causes:

  • The run context is injected as a module global into the provider modules and never cleared.
    • It keeps the snapshot cache (all source and target items) and the whole orchestrator alive after the run ends.
  • Analyzer, wall and insight caches keep a full copy of every item and are never dropped.
  • Python does not hand freed memory back to the OS on its own.

Proposal

  • Add cw_platform/memory_release.py. Caches register a clear function there.
  • Restore the provider ctx after a run and clear the snapshot cache.
  • Cache the master key, the cipher and the normalized config, keyed on mtime and size.
  • id_map: skip absent keys, return numeric and tt ids without a regex, dict fast path.
  • Batch the statistics insert.

Area

  • Watchlist
  • Ratings
  • History
  • Playlists
  • Scrobble watcher
  • Scrobble webhook
  • Scheduler
  • Analyzer, Events, Exporter
  • Web UI
  • Command line interface
  • Config/API

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions