I went through a bit of trouble trying to find and setup a working container so I threw this together. This is based on this example from the llama-cpp-python repo.
I went through a bit of trouble trying to find and setup a working container so I threw this together. This is based on this example from the llama-cpp-python repo.