[Design] AgentOS Component System (ACS) and Registry (Formerly ACR - AgentOS Component Registry)

[nickjalbert]

This discussion thread is about the design of AgentOS component system (ACS) and Registry. Originally, both ideas were combined in the design proposal for AgentOS Component Registry, which has since been replaced by the two separate sub-systems that work together.

Design effort leader: @nickjalbert

Design Doc Pull Requests:

• PR #1: Proposal for issue 68: AgentOS registry
• PR #2: Avoid merging requirements on component install
• PR #4: Harmonize ACR design with core abstraction updates

Versions of the design doc:

• v1
• v2
• v3
• v4
• v5
• v6

[andyk]

The following review is for v6 of the design doc.

• The first time you mention agent.ini, add a link to the Core Abstractions design doc Agent Definition File section which details the design of the agent.ini file.

• "As you let your agent run, you'll get periodic updates on its performance improvement as it gains more experience playing 2048 and learns via the SARSA algorithm." If this is true and the agent learn as it runs, why do you need to call agentos train agent.ini 1000 before running it? The next paragraph explains that the developer can pick along the spectrum of batch vs online learning, but it still seems a little confusing in the demo code to first call train, and then call run which runs+trains.

• "If you have more than one component installed for a particular role (e.g. two complementary environments) then you can access each component via their name in the acr module:". Instead of this, what about just adding more members to the agent class, with the name of the components, e.g. self.enviroment.2048 or self.2048 or self.evironments["2048"].

• Would it also make sense to support registering Agents? If not in V0, then maybe we want to the Long Term Plans section?

• "Developers have an easy way to register their local custom components with acr so it can be accessed via the acr module in other parts of their agent." Can we add example code showing how this would look?

• From the FAQ: "Each component maintains exposes a configuration in its agent.ini entry. This allows for both manual tweaking of hyperparameters as well as programmatic exploration and tuning."

  • Typo "maintains exposes"
  • By "exposes a configuration" do you mean has the name of a config text file? We might want to say a little bit more, or even give a code example, of using a specific popular hyperparam search tool (maybe sk-learn?)

[nickjalbert]

@andyk @trush good discussion and feedback! Let me know if I've gotten anything wrong here.

Notes from our meeting on Feb 12 2021:

• Design doc feedback: it'd be useful to clearly define what the thing being
  designed is at the top of the doc .

  • Maybe also a glossary of terms at the end of the doc

  • What is a component? A shareable, swappable package that an agent can
    incorporate that adheres to a specific API. Agent is (probably)
    different than a component.

  • How can I tell if a piece of code is a component?

• What is ACR?

  • a subsystem of AgentOS that integrates with the CLI and a registry that
    allows for structuring of agents.

  • do we have an "import acr"? We may not even want to provide runtime
    access to ACR (as currently proposed) until we have a clear use case.

• A reimagined version of ACR:

  • Package management functionality:

    • agentos install
    • agentos search
    • agentos uninstall
    • agentos status - new command to show what's available in your agent

  • Git-like mapping tools

    • commands to map particular versions of installed components to
      particular members available in the agent class.

• More related work investigation: Couch + Acme + RLGraph + others

  • Acme docs are good
  • Big players are in this space. What will we compete on?
    • Registry seems somewhat unique. Potentially could be the glue that
      binds all these frameworks. "The platform for building RL using
      libraries like acme, couch, etc"

• How does component versioning apply to the mutable part of a policy

  • Maybe this is a separate component
  • A policy evolves over time as an agent gains experience. Exposing this
    to the developer might be useful.
    • Again, could be git-like (or tagging like docker)
    • MLFlow already does a lot of things in this ballpark, potentially
      reusable

• Some next steps:

  • Nick: looking at design doc feedback; propose answers to Qs if possible
  • Nick: clean up "new" abstraction branch and issue a PR
  • Andy: try to port an example agent over using the new abstractions
  • Beth: continue ramp up

[andyk]

Good notes!

[andyk]

Also, today I called the Intel framework Couch, but it is actually Coach https://github.com/IntelLabs/coach

[andyk]

One other thing:

Currently, component type in the registry entry spec might be under-specified because agentos APIs will change and so we might want to add agentos version to the component spec.

E.g.:

...
type: policy
version: 0.1.0
...

[andyk]

I made a few mockups of what how an end user might view the ACR via a web interface: https://docs.google.com/presentation/d/1k0RNr1HBJV8Dvc58bz_xWPTDqCKdNLzcvt3Op0IPuFU/edit?usp=sharing

[andyk]

We've recently agreed to separate the idea of the AgentOS Component System (ACS) from the AgentOS Registry. Previously the ideas were combined in the original design of the AgentOS Component Registry (ACR) -- i.e., this discussion & it's associated design doc & GitHub Issues.

[andyk]

Also, the MVP of what used to be called ACR (now ACS plus the Registry) has been merged into Master and the plan is to release it in its initial form in v0.1.0.