Skip to content

Latest commit

 

History

History
35 lines (25 loc) · 659 Bytes

README.md

File metadata and controls

35 lines (25 loc) · 659 Bytes

Llama Fast API Server

Installation

FastAPI

Windows

pip install fastapi

macOS

pip3 install fastapi

Llama-cpp-python

Windows + CUDA

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121

Windows + CPU

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

macOS + Metal

pip3 install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal