Skip to content

RNN formulation #15

@mtanghu

Description

@mtanghu

The math for an RNN based LEAP is already shown. It just needs to be integrated with the huggingface generate API.

Also if relative embeddings are used (see #14) and constant memory gradient calculation is implemented (see #12) this would enable truly infinite context.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions