[fun] llama.triton

[ByronHsu]

🚀 The feature, motivation and pitch

@thomwolf and i have an idea to implement llama from scratch in pure triton, inspired by karpathy. liger kernel already contains most of the kernels except matmul. We would love to call out for any interested! It can be added under our example/ folder!

Alternatives

No response

Additional context

No response

[ziliangpeng]

omw

[thevasudevgupta]

i implemented gpt-2 in triton few days back. Ig llama would be similar- just need to implement some specific layers.

sharing if someone wants the starting code!

[vigneshbp]

@thevasudevgupta Could you please share the specific code so that I can directly look into it ?

[thevasudevgupta]

ohh; I forgot to link it. sorry;

https://github.com/thevasudevgupta/gpt-triton

[kerthcet]

Do you guys think a triton based inference engine would be a good path?

[ByronHsu]

@kerthcet no we want to do training here. triton based inference already has too many options like vllm

[ghostway0]

about the mm kernel, I wrote something like that, if that interests anyone

if the todos there are fixed, I think a pr would make sense?