[Feat]: llama.cpp vs mnn-llm speed

[bi4key]

llama.cpp vs mnn-llm speed

---

https://www.reddit.com/r/LocalLLaMA/s/tDh1l0cvVe

.

https://github.com/alibaba/MNN/blob/master/project/android/apps/MnnLlmApp/README.md

.

https://github.com/alibaba/MNN

[a-ghorbani]

Interesting. The multimodality aspect sounds great, but the LLM inference speed doesn’t seem to be significantly different on phones, from what folks report:

• https://www.reddit.com/r/LocalLLaMA/comments/1ia7v0x/comment/m9akra7/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
• https://www.reddit.com/r/LocalLLaMA/comments/1ia7v0x/comment/m9a9i0l/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[Mcxiaocaibug]

I think there's still a big difference

[Mcxiaocaibug]

pocketpal 5tokens/s
mnn 50token/s
DeepSeek r1 1.5B
There is a big difference between them and hopefully it can be improved
@a-ghorbani

[HiWill]

MNN beat llama.cpp

pocketpal 5tokens/s mnn 50token/s DeepSeek r1 1.5B There is a big difference between them and hopefully it can be improved @a-ghorbani

[HiWill]

llama.cpp 易用性要好，希望能提升性能