diff --git a/README.md b/README.md index cbba2cd..4b71251 100644 --- a/README.md +++ b/README.md @@ -95,6 +95,17 @@ Here's the steps to set up the project locally: 7. Fill in the models name and check the "active" checkbox in red boxes in the image bellow and click "Save" (pay attention to the LLM and Embeddings model name placement): ![local_models_admin_fields](docs/assets/local_models_admin_fields.png) +### Using DeepSeek R1 with nomic: + +1. `make up` +2. `make createuser` +3. `make ollama model=nomic-embed-text:latest` +4. `make ollama model=deepseek-r1` | `deepseek-coder` | `deepseek-coder-v2`, coder-v2 is the best if you have a good machine. +5. Go to [http://localhost:8000/admin](http://localhost:8000/admin) in internet browser and login (admin/admin) +6. Select "Models" option in the menu on the left +7. Fill in the models name and check the "active" checkbox in red boxes in the image below and click "Save" (pay attention to the LLM and Embeddings model name placement): + ![local_models_admin_fields](docs/assets/local_models_admin_fields.png) + ## Prompt Examples There are some prompt examples in [here](docs/prompt_examples.md). These are sorted by complexity and are a way for us to measure the quality of the LLM's response. diff --git a/labs/llm/ollama.py b/labs/llm/ollama.py index 12eca43..4a2fb57 100644 --- a/labs/llm/ollama.py +++ b/labs/llm/ollama.py @@ -12,6 +12,7 @@ def completion_without_proxy(self, messages, *args, **kwargs): model=self._model_name, messages=messages, format="json", + options={"num_ctx": 8192}, *args, **kwargs, )