Skip to content

Commit e81ec68

Browse files
committed
Update Vault documentation to include support for Ollama SDK and clarify provider specifications
1 parent 5e78f4e commit e81ec68

File tree

2 files changed

+150
-216
lines changed

2 files changed

+150
-216
lines changed

docs/agent-studio/key-concepts/vault.mdx

Lines changed: 66 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -85,46 +85,101 @@ To add a provider, open **Vault → Add Provider** and paste your credentials.
8585
## Custom Models
8686

8787
You can now add **custom models** to the SmythOS platform directly through the Vault.
88-
This feature lets you connect your own model servers or third-party hosted APIs, as long as they expose endpoints compatible with the **OpenAI SDK**.
88+
This feature lets you connect your own model servers or third-party hosted APIs, supporting both **Ollama SDK** and **OpenAI-compatible SDK** endpoints.
8989

9090
### How it works
9191

9292
1. Go to **Vault → Custom Models**
9393
2. Click **Add Custom Model**
9494
3. Provide the required details:
9595
- **Name**: The display name shown in the model list
96-
- **Model ID**: Copy from your supported model (e.g., Grok, Together AI, etc.)
97-
- **Base URL**: The API base endpoint (must be public, not localhost or private IP)
98-
- **Provider**: Select **OpenAI-compatible**
99-
- **API Key**: Add the provider key (e.g., from Grok)
100-
- **Context Window**: Optional limit on context tokens
96+
- **Model ID**: Copy from your supported model (e.g., `grok-1`, `ollama-mistral-7b`, etc.)
97+
- **Base URL**: The API base endpoint (for SaaS: must be public, not localhost or private IP)
98+
- **Provider**: Select **Ollama** or **OpenAI**
99+
- **API Key**: Add the provider key if required
100+
- **Context Window**: Maximum tokens for input + output
101+
- **Max Output Tokens**: Maximum tokens for model response
101102
- **Fallback Model**: Used when the custom model is unreachable
102103

103-
<WarningCallout title="Important Security Note">
104+
### Feature Toggles
105+
106+
- **Text Completion** *(enabled by default)* — Makes model available in all LLM components
107+
- **Function Calling / Tool Use** *(disabled by default)* — Enables model for Agents with skill calling
108+
109+
<WarningCallout title="Important Security Note (SaaS Environment)">
104110
Local or private IP addresses (like `localhost`, `127.0.0.1`, `10.x.x.x`, or `172.x.x.x`) are **not allowed** for base URLs in the SaaS environment for security reasons.
105111
Always use a public or hosted endpoint instead (e.g., Grok, Together AI, or a remote Colab server).
112+
113+
**Note:** If you're running SmythOS on your own infrastructure, local endpoints are supported.
106114
</WarningCallout>
107115

108116
Once saved, reload your **Builder** page to see the newly added custom model in your model list.
109117

110-
### Example: Adding a Grok model
118+
### Provider Specifications
111119

112-
```json
120+
#### Ollama
121+
- **Base URL**: `http://your-hosted-model` your hosted Ollama endpoint
122+
- **Supported Models**: Mistral, Llama 2, Neural Chat, Wizard Coder, and others available in Ollama's model library
123+
- **Authentication**: Usually not required for local instances; include API key if your hosted instance requires it
124+
125+
#### OpenAI-Compatible
126+
- **Base URL**: `https://api.openai.com/v1` (OpenAI) or provider-specific endpoint (Grok, Together AI, etc.)
127+
- **Supported Providers**: OpenAI, Azure OpenAI, Together AI, Replicate, Fireworks, xAI (Grok)
128+
- **Authentication**: API key if required for OpenAI-compatible providers
129+
130+
### Example: Adding an Ollama Model
131+
132+
~~~json
133+
{
134+
"name": "Mistral 7B Local",
135+
"model_id": "mistral-7b",
136+
"base_url": "http://your-hosted-model",
137+
"provider": "Ollama",
138+
"api_key": "",
139+
"context_window": 8192,
140+
"max_output_tokens": 4096,
141+
"fallback_model": "gpt-4o-mini"
142+
}
143+
~~~
144+
145+
### Example: Adding a Grok Model (OpenAI-Compatible)
146+
147+
~~~json
113148
{
114149
"name": "Model for Grok",
115150
"model_id": "grok-1",
116151
"base_url": "https://api.grok.ai/v1",
117152
"provider": "OpenAI",
118153
"api_key": "grk-1234abcd...",
154+
"context_window": 32000,
155+
"max_output_tokens": 8000,
119156
"fallback_model": "gpt-4o-mini"
120157
}
121-
```
158+
~~~
159+
122160
After saving, reload the Builder. Your custom model will appear at the top of the model list. You can now select it for testing or deployment just like built-in models.
123161

124162
<InfoCallout title="Compatibility">
125-
Any model API compatible with the OpenAI schema (`/v1/chat/completions`, `/v1/completions`, etc.) will work with SmythOS custom models. This includes integrations built with `ollama.cpp`, `Grok`, `Together AI`, or even custom endpoints hosted on services like Google Colab (if properly exposed).
163+
SmythOS supports two SDK protocols for custom models:
164+
165+
1. **Ollama SDK** — For models running on Ollama instances
166+
2. **OpenAI SDK** — For any model API compatible with the OpenAI schema (`/v1/chat/completions`, `/v1/completions`, etc.)
167+
168+
This includes integrations built with Ollama, Grok, Together AI, Replicate, or even custom endpoints hosted on services like Google Colab (if properly exposed).
126169
</InfoCallout>
127170

171+
### Using Custom Models
172+
173+
Custom models appear in all LLM selection dropdowns across SmythOS:
174+
- **Agent Settings** → Default LLM
175+
- **Skill components** with LLM options
176+
- **Any component** that accepts LLM selection
177+
178+
### Managing Custom Models
179+
180+
- **Edit**: Go to **Vault → Custom Models**, click the Edit icon
181+
- **Delete**: Click the Delete icon
182+
128183
### Managing Keys
129184

130185
Custom model API keys are stored securely in Vault.

0 commit comments

Comments
 (0)