Community Plugin architecture! 🎁

[kyr0]

Hi,

let's please discuss a community plugin architecture.

Motivation:
I'd like to implement my "Let's build a custom Alexa clone with 3D talking head, TTS, ASR and agentic workflows 100% local" directly in oMLX. However, parts of this architecture would use ANE (Apple Neural Engine) including specific Swift native components. It is common sense that something like that does not belong into the core of oMLX - but it would be a great synergy anyway.

If oMLX had a plugin API for the admin interface, I could simply provide a nice web interface and a few custom HTTP server endpoints plus add it in the menu bar. This would greatly simplify deploying my idea (everything is ready and fast) - likewise oMLX would become a decent deployment platform and millions of users would start using it with decent plugins being released.

Idea:

• We allow to customize oMLX by loading plugins from a specific, defined location. e.g. .~/omlx/plugins/*
  • Each plugin has to have a manifest file specifying the entrypoint file for:
    • Backend (Python)
      • With an API to hook the omlx/api (add routes)
    • Frontend
      • HTML template
      • JavaScript main file
    • Menu entry (for admin/templates/dashboard/_navbar.html dynamic integration)

This would probably already suffice for 99% of use-cases. With the possibility to override routes we get infinite possibilities and with the possibility to render any HTML template + JS anything is possible on the loaded page (DOM).

What do you think?

I believe we could try this idea out - and then, later on, build a community marketplace, so users can mix and match cool plugins they love!