Added OLMo(E) v1

[jonasrohw]

Description

Adds support for OLMo v1 model family and OLMoE. Transformers>3.40 will let numpy do a major upgrade; pyproject.toml prevents this now.

OLMO v2 will require dropping python3.8 support because the required Transformers version also drops it. It will be added in a separate PR based on TransformerLens 3.
This also completes PR: #718

Fixes # (issue)

• [Proposal] Compatibility for OLMo and OLMo2? #804 (partially)

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• [] I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

[taziksh]

Hey @jonasrohw, looks like you've got this feature pretty much ready to go - just seeing type check failures blocking it. I'd be interested in taking a stab at fixing those type issues if you're not actively working on it.

[jonasrohw]

@taziksh Yeah, I didn't have time to fix some of the type issues. Go ahead!

[taziksh]

@jonasrohw
I've added type checking fixes to complete the OLMo implementation in #1081. Happy to collaborate however works best!