Can we use it along with ollama ?

[ChanduKaranam]

Can we combine this with ollama to run models using ollama?

If we can would appreciate knowing how to do that .

[Jackson57279]

No, they work differently. Ollama loads the full model into memory, while AirLLM loads layers one at a time. They're optimized for different use cases.

[mayank64ce]

@Jackson57279, what do you mean by different use cases? Both are for model serving right?

[Jackson57279]

@Jackson57279, what do you mean by different use cases? Both are for model serving right?

They both serve models yes, but the use case for airllm is to run large models on weaker gpus. Ollamas use case is just run the model without any performance tweaks

[ahmedalseify33-a11y]

So i can't use it for openclaw; right?

[Jackson57279]

@ahmedalseify33-a11y you might, I think openclaw has a openai compatible so if the model is running and you point the URL to that, it should work. But I dont recommend local models for openclaw especially if there small.