Merge pull request #40 from zhengxs2018/feat-legacy-and-delay

phodal · web-flow · commit 82ec5400009b · 2024-05-24T08:00:56.000+08:00
diff --git a/docs/configuration.md b/docs/configuration.md
@@ -0,0 +1,184 @@
+# Configuration
+
+## Basic
+
+This is the default configuration and can be overridden in different functional areas.
+
+### Anthropic
+
+See [Developer doc](https://docs.anthropic.com/en/docs/intro-to-claude)
+
+- **Base URL** Anthropic API URL, See [API Reference](https://docs.anthropic.com/en/api/getting-started)
+- **API Key** Anthropic API key.
+- **Model** Chat model used
+
+### OpenAI
+
+See [open.com](https://platform.openai.com/docs/introduction)
+
+- **API Type** OpenAI Official or Microsoft azure servers.
+- **Base URL** OpenAI API URL.
+- **API Key** Our legacy keys. Provides access to all organizations and all projects that user has been added to; access [API Keys](https://platform.openai.com/account/api-keys) to view your available keys. We highly advise transitioning to project keys for best security practices, although access via this method is currently still supported.
+- **Model** Chat model used, See [Model endpoint compatibility](https://platform.openai.com/docs/models/model-endpoint-compatibility)
+- **Project** Provides access to a single project (preferred option); Access [Project API Keys](https://platform.openai.com/settings/organization/general) by selecting the specific project you wish to generate keys against.
+- **Organization** For users who belong to multiple organizations or are accessing their projects through their legacy user API key, you can pass a header to specify which organization and project is used for an API request. Usage from these API requests will count as usage for the specified organization and project.
+
+### Baidu Cloud WenXin
+
+Baidu Cloud API Key Or Secret Key, See [Create an application](https://console.bce.baidu.com/qianfan/ais/console/applicationConsole/application).
+
+- **API Key** Baidu Cloud API Key
+- **Secret Key** Baidu Cloud Secret Key
+- **Model** Chat model used
+
+### Ali Cloud TongYi
+
+See [开通 DashScope 并创建 API-KEY](https://help.aliyun.com/zh/dashscope/developer-reference/activate-dashscope-and-create-an-api-key).
+
+- **API Key** Baidu Cloud API Key
+- **Model** Chat model used
+- **EnableSearch** 启用互联网搜索，模型会将搜索结果作为文本生成过程中的参考信息，但模型会基于其内部逻辑判断是否使用互联网搜索结果。默认：关闭。
+
+## Chat
+
+### Models
+
+Shows in chat panel model selection list.
+
+![Sidepanel](./images/sidepanel.png)
+
+- **title** Display selected text
+  - **type** `string`
+- **provider** Use that LLM Provider.
+  - **type** `"anthropic" | "openai" | "qianfan" | "tongyi"`
+- **baseURL** LLM API baseURL, Default use provider config.
+  - **type** `string | undefined`
+- **apiKey** LLM API key, Default use provider config.
+  - **type** `string | undefined`
+- **secret Key** Only Baidu QianFan provider
+  - **type** `string | undefined`
+- **model** Model name
+  - **type** `string`
+- **temperature** Amount of randomness injected into the response. Ranges from 0 to 1.
+  - **type** `number | undefined`
+- **maxTokens** A maximum number of tokens to generate before stopping.
+  - **type** `number | undefined`
+- **stop** A list of strings to use as stop words.
+  - **type** `string[] | undefined`
+- **clientOptions** Optional parameters
+  - **type** `object | undefined`
+
+examples:
+
+```jsonc
+[
+  {
+    "title": "GPT-4",
+    "provider": "openai",
+    "model": "gpt-4"
+  },
+  {
+    "title": "GPT-3.5 turbo",
+    "provider": "openai",
+    "model": "gpt-3.5-turbo",
+    "temperature": 0.75
+  },
+  {
+    "title": "QWen turbo",
+    "provider": "tongyi",
+    "model": "qwen-turbo"
+  },
+  {
+    "title": "ERNIE-Bot turbo",
+    "provider": "qianfan",
+    "model": "ERNIE-Bot-turbo"
+  }
+]
+```
+
+If there is no configuration of apikey, default get from basic config.
+
+## Code completion
+
+### Model
+
+Model for overwrite provider in the base configuration. see [Autodev: OpenAI](#openai)
+
+### Template
+
+Customize your modeling cue template. 
+
+> [!IMPORTANT]
+> Variables use string replacement, please fill in strictly according to instructions.
+
+The recommended format is FIM ( filling in the middle ), for example:
+
+```sh
+<fim_prefix>{prefix}<fim_suffix>{suffix}<fim_middle>
+
+# or 
+
+<PRE>{prefix}<SUF>{suffix}<MID>
+```
+
+Available Variables: 
+
+- `prefix` Code before the cursor.
+- `suffix` Code after the cursor.
+- `language` Current editing file language, for example: "javascript".
+
+### Stops
+
+Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.
+
+### Enable Legacy Mode
+
+Use legacy `/v1/completion` instead of `/v1/chat/completion`
+
+### Request Delay
+
+Code auto-completion delay request time. Avoid excessive consumption of API tokens. `requestDelay` only works if [Autodev: Enable Code Completion](#enable-code-completion) is enabled.
+
+### Enable Rename
+
+Enable rename suggestion
+
+### Enable Code Completion
+
+Enable code-completion. Then, when the editor is triggered (e.g., a carriage return or a line feed or a content change, etc.), it automatically completes your code.
+
+## Embedding
+
+### Models
+
+### Builtin: Sentence Transformerjs in Local
+
+### OpenAI
+
+### Ollama
+
+## Legacy Config
+
+## OpenAI Compatible Config
+
+> Please use [Autodev: OpenAI](#openai) instead
+
+Model for overwrite provider in the base configuration. see `autodev.openai.model`
+
+> [!NOTE]  
+> Currently only supports OpenAI chat models.
+
+Config OpenAI example:
+
+1. open `settings.json` in vscode
+2. add the following configuration
+
+```
+  "autodev.openaiCompatibleConfig": {
+    "apiType": "openai",
+    "model": "moonshot-v1-8k",
+    "apiBase": "https://api.moonshot.cn/v1",
+    "apiKey": "xxx"
+  },
+  "autodev.completion.model": "openai",
+```
diff --git a/docs/features/code-completion.md b/docs/features/code-completion.md
@@ -0,0 +1,58 @@
+---
+layout: default
+title: Code Completions
+parent: Basic Features
+nav_order: 5
+permalink: /features/code-completion
+---
+
+Automatically completes your code based on the position of your cursor.
+
+## Enable Feature
+
+Not enabled by default, see [AutoDev: Code Completion](../configuration.md#code-completion)。
+
+```jsonc
+{
+  "autodev.openai.apiKey": "sk-xxxxx", // Your openai api key
+  "autodev.suggestion.enableCodeCompletion": true
+}
+```
+
+Now let's try writing some code.
+
+## Select code Model
+
+You can hope that you use specific code models instead of dialog models
+
+```jsonc
+{
+  "autodev.completion.model": "gpt-4o" // Overriding the default chat model
+}
+```
+
+Recommended to use a specially trained code model, or a base model that supports fim.
+
+## Connect to local model
+
+Here is an example of ollama, see [OpenAI compatibility](https://github.com/ollama/ollama/blob/main/docs/openai.md) for details.
+
+```jsonc
+{
+  "autodev.openai.baseURL": "http://127.0.0.1:11434/v1/", // Your local service url
+  "autodev.openai.apiKey": "sk-xxxxx", // Your local service api key
+  "autodev.completion.model": "codeqwen:7b-code-v1.5-q5_1" // Overriding the default chat model
+}
+```
+
+If your self-built service is deployed in a mode that does not support chat, you may need to enable [legacy mode](#enable-legacy-mode).
+
+## Enable Legacy Mode
+
+The default is the traditional `/v1/completion` instead of `/v1/chat/completion`, but you can fall back to the old mode.
+
+```jsonc
+{
+  "autodev.completion.enableLegacyMode": true
+}
+```
diff --git a/docs/quick-start.md b/docs/quick-start.md
@@ -14,110 +14,10 @@ In the vscode configuration, search for `autodev`, or click the ⚙️ button in
 在当前的设计里，由于精力不足，在 UI 设计上我们基于 Continue 的设计，因此会出现一些不符合直觉的地方，我们会在后续的版本中逐渐优化。诸如于：
 **Chat UI 和代码中的 LLM 模型需要分开配置。
 
-## Basic
+> [!IMPORTANT]
+> You must configure at least one big model for the plugin to work, see [Configuration](./configuration.md) for details.
 
-This is the default configuration and can be overridden in different functional areas.
-
-### Anthropic
-
-See [Developer doc](https://docs.anthropic.com/en/docs/intro-to-claude)
-
-- **Base URL** Anthropic API URL, See [API Reference](https://docs.anthropic.com/en/api/getting-started)
-- **API Key** Anthropic API key.
-- **Model** Chat model used
-
-### OpenAI
-
-See [open.com](https://platform.openai.com/docs/introduction)
-
-- **API Type** OpenAI Official or Microsoft azure servers.
-- **Base URL** OpenAI API URL.
-- **API Key** Our legacy keys. Provides access to all organizations and all projects that user has been added to; access [API Keys](https://platform.openai.com/account/api-keys) to view your available keys. We highly advise transitioning to project keys for best security practices, although access via this method is currently still supported.
-- **Model** Chat model used, See [Model endpoint compatibility](https://platform.openai.com/docs/models/model-endpoint-compatibility)
-- **Project** Provides access to a single project (preferred option); Access [Project API Keys](https://platform.openai.com/settings/organization/general) by selecting the specific project you wish to generate keys against.
-- **Organization** For users who belong to multiple organizations or are accessing their projects through their legacy user API key, you can pass a header to specify which organization and project is used for an API request. Usage from these API requests will count as usage for the specified organization and project.
-
-### Baidu Cloud WenXin
-
-Baidu Cloud API Key Or Secret Key, See [Create an application](https://console.bce.baidu.com/qianfan/ais/console/applicationConsole/application).
-
-- **API Key** Baidu Cloud API Key
-- **Secret Key** Baidu Cloud Secret Key
-- **Model** Chat model used
-
-### Ali Cloud TongYi
-
-See [开通 DashScope 并创建 API-KEY](https://help.aliyun.com/zh/dashscope/developer-reference/activate-dashscope-and-create-an-api-key).
-
-- **API Key** Baidu Cloud API Key
-- **Model** Chat model used
-- **EnableSearch** 启用互联网搜索，模型会将搜索结果作为文本生成过程中的参考信息，但模型会基于其内部逻辑判断是否使用互联网搜索结果。默认：关闭。
-
-## Chat
-
-### Models
-
-Shows in chat panel model selection list.
-
-![Sidepanel](./images/sidepanel.png)
-
-- **Title** Display selected text
-  - **type** `string`
-- **Provider** Use that LLM Provider.
-  - **type** `"anthropic" | "openai" | "qianfan" | "tongyi"`
-- **Base URL** LLM API baseURL, Default use provider config.
-  - **type** `string | undefined`
-- **API Key** LLM API key, Default use provider config.
-  - **type** `string | undefined`
-- **Secret Key** Only Baidu QianFan provider
-  - **type** `string | undefined`
-- **Model** Model name
-  - **type** `string`
-- **Temperature** Amount of randomness injected into the response. Ranges from 0 to 1.
-  - **type** `number | undefined`
-- **MaxTokens** A maximum number of tokens to generate before stopping.
-  - **type** `number | undefined`
-- **Stop** A list of strings to use as stop words.
-  - **type** `string[] | undefined`
-
-examples:
-
-```jsonc
-[
-  {
-    "title": "GPT-4",
-    "provider": "openai",
-    "model": "gpt-4"
-  },
-  {
-    "title": "GPT-3.5 turbo",
-    "provider": "openai",
-    "model": "gpt-3.5-turbo",
-    "temperature": 0.75
-  },
-  {
-    "title": "QWen turbo",
-    "provider": "tongyi",
-    "model": "qwen-turbo"
-  },
-  {
-    "title": "ERNIE-Bot turbo",
-    "provider": "qianfan",
-    "model": "ERNIE-Bot-turbo"
-  }
-]
-```
-
-If there is no configuration of apikey, default get from basic config.
-
-## Code completion
-
-### Model
-
-Model for overwrite provider in the base configuration. see `autodev.openai.model`
-
-> [!NOTE]  
-> Currently only supports OpenAI chat models.
+## Usage
 
 Config OpenAI example:
 
@@ -134,18 +34,6 @@ Config OpenAI example:
   "autodev.completion.model": "openai",
 ```
 
-### enableRename
-
-Enable rename suggestion
-\
-
-## Embedding
-
-### Models
-
-### Builtin: Sentence Transformerjs in Local
-
-### OpenAI
-
-### Ollama
+## Next
 
+- [Enable Code-completion](./features/code-completion.md)
diff --git a/package.json b/package.json
@@ -293,6 +293,17 @@
             ],
             "description": "Stop words of code model"
           },
+          "autodev.completion.enableLegacyMode": {
+            "type": "boolean",
+            "default": false,
+            "description": "Use legacy \"/v1/completion\" instead of \"/v1/chat/completion\"",
+            "markdownDescription": "Use legacy `/v1/completion` instead of `/v1/chat/completion`"
+          },
+          "autodev.completion.requestDelay": {
+            "type": "integer",
+            "default": 500,
+            "markdownDescription": "Code auto-completion delay request time. Avoid excessive consumption of API tokens. `requestDelay` only works if `#autodev.suggestion.enableCodeCompletion#` is enabled."
+          },
           "autodev.suggestion.enableRename": {
             "type": "boolean",
             "default": false,
diff --git a/src/action/providers/AutoDevCodeSuggestionProvider.ts b/src/action/providers/AutoDevCodeSuggestionProvider.ts
diff --git a/src/llm-provider/OpenAICompletion.ts b/src/llm-provider/OpenAICompletion.ts
