Skip to content

mlgo AOT: serialize weights separately #199044

@mtrofin

Description

@mtrofin

The weights are currently exported as literals in the EmitC-generated code. If we, instead, exported them to a separate file, we could:

  • have much, much smaller generated C++ files to inspect. This is valuable if we want to reason about the performance of the generated code (i.e. compiler performance)
  • allow loading weights as a command line flag - as long as the model architecture is the same, weights could be dynamically loaded, as opposed to requiring a rebuild of the compiler. This can offer a training-time alternative, as well as a light-weight alternative to multi-model support. This can further enable shipping a "sufficiently good" model architecture out of the box in llvm, which could then be trained with black box techniques like Evolutionary Strategy, which have no dependency on any specific ML framework. Power users would still want to have the full flexibility of model choice and training infrastructure choice, but this would make using and training / tuning ML models accessible to more.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions