You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/agent-studio/key-concepts/vault.mdx
+66-11Lines changed: 66 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -85,46 +85,101 @@ To add a provider, open **Vault → Add Provider** and paste your credentials.
85
85
## Custom Models
86
86
87
87
You can now add **custom models** to the SmythOS platform directly through the Vault.
88
-
This feature lets you connect your own model servers or third-party hosted APIs, as long as they expose endpoints compatible with the **OpenAI SDK**.
88
+
This feature lets you connect your own model servers or third-party hosted APIs, supporting both **Ollama SDK** and **OpenAI-compatible SDK** endpoints.
89
89
90
90
### How it works
91
91
92
92
1. Go to **Vault → Custom Models**
93
93
2. Click **Add Custom Model**
94
94
3. Provide the required details:
95
95
-**Name**: The display name shown in the model list
96
-
-**Model ID**: Copy from your supported model (e.g., Grok, Together AI, etc.)
97
-
-**Base URL**: The API base endpoint (must be public, not localhost or private IP)
98
-
-**Provider**: Select **OpenAI-compatible**
99
-
-**API Key**: Add the provider key (e.g., from Grok)
100
-
-**Context Window**: Optional limit on context tokens
96
+
-**Model ID**: Copy from your supported model (e.g., `grok-1`, `ollama-mistral-7b`, etc.)
97
+
-**Base URL**: The API base endpoint (for SaaS: must be public, not localhost or private IP)
98
+
-**Provider**: Select **Ollama** or **OpenAI**
99
+
-**API Key**: Add the provider key if required
100
+
-**Context Window**: Maximum tokens for input + output
101
+
-**Max Output Tokens**: Maximum tokens for model response
101
102
-**Fallback Model**: Used when the custom model is unreachable
102
103
103
-
<WarningCallouttitle="Important Security Note">
104
+
### Feature Toggles
105
+
106
+
-**Text Completion***(enabled by default)* — Makes model available in all LLM components
107
+
-**Function Calling / Tool Use***(disabled by default)* — Enables model for Agents with skill calling
Local or private IP addresses (like `localhost`, `127.0.0.1`, `10.x.x.x`, or `172.x.x.x`) are **not allowed** for base URLs in the SaaS environment for security reasons.
105
111
Always use a public or hosted endpoint instead (e.g., Grok, Together AI, or a remote Colab server).
112
+
113
+
**Note:** If you're running SmythOS on your own infrastructure, local endpoints are supported.
106
114
</WarningCallout>
107
115
108
116
Once saved, reload your **Builder** page to see the newly added custom model in your model list.
109
117
110
-
### Example: Adding a Grok model
118
+
### Provider Specifications
111
119
112
-
```json
120
+
#### Ollama
121
+
-**Base URL**: `http://your-hosted-model` your hosted Ollama endpoint
122
+
-**Supported Models**: Mistral, Llama 2, Neural Chat, Wizard Coder, and others available in Ollama's model library
123
+
-**Authentication**: Usually not required for local instances; include API key if your hosted instance requires it
124
+
125
+
#### OpenAI-Compatible
126
+
-**Base URL**: `https://api.openai.com/v1` (OpenAI) or provider-specific endpoint (Grok, Together AI, etc.)
-**Authentication**: API key if required for OpenAI-compatible providers
129
+
130
+
### Example: Adding an Ollama Model
131
+
132
+
~~~json
133
+
{
134
+
"name": "Mistral 7B Local",
135
+
"model_id": "mistral-7b",
136
+
"base_url": "http://your-hosted-model",
137
+
"provider": "Ollama",
138
+
"api_key": "",
139
+
"context_window": 8192,
140
+
"max_output_tokens": 4096,
141
+
"fallback_model": "gpt-4o-mini"
142
+
}
143
+
~~~
144
+
145
+
### Example: Adding a Grok Model (OpenAI-Compatible)
146
+
147
+
~~~json
113
148
{
114
149
"name": "Model for Grok",
115
150
"model_id": "grok-1",
116
151
"base_url": "https://api.grok.ai/v1",
117
152
"provider": "OpenAI",
118
153
"api_key": "grk-1234abcd...",
154
+
"context_window": 32000,
155
+
"max_output_tokens": 8000,
119
156
"fallback_model": "gpt-4o-mini"
120
157
}
121
-
```
158
+
~~~
159
+
122
160
After saving, reload the Builder. Your custom model will appear at the top of the model list. You can now select it for testing or deployment just like built-in models.
123
161
124
162
<InfoCallouttitle="Compatibility">
125
-
Any model API compatible with the OpenAI schema (`/v1/chat/completions`, `/v1/completions`, etc.) will work with SmythOS custom models. This includes integrations built with `ollama.cpp`, `Grok`, `Together AI`, or even custom endpoints hosted on services like Google Colab (if properly exposed).
163
+
SmythOS supports two SDK protocols for custom models:
164
+
165
+
1.**Ollama SDK** — For models running on Ollama instances
166
+
2.**OpenAI SDK** — For any model API compatible with the OpenAI schema (`/v1/chat/completions`, `/v1/completions`, etc.)
167
+
168
+
This includes integrations built with Ollama, Grok, Together AI, Replicate, or even custom endpoints hosted on services like Google Colab (if properly exposed).
126
169
</InfoCallout>
127
170
171
+
### Using Custom Models
172
+
173
+
Custom models appear in all LLM selection dropdowns across SmythOS:
174
+
-**Agent Settings** → Default LLM
175
+
-**Skill components** with LLM options
176
+
-**Any component** that accepts LLM selection
177
+
178
+
### Managing Custom Models
179
+
180
+
-**Edit**: Go to **Vault → Custom Models**, click the Edit icon
181
+
-**Delete**: Click the Delete icon
182
+
128
183
### Managing Keys
129
184
130
185
Custom model API keys are stored securely in Vault.
0 commit comments