fix(llama.cpp): disable mirostat as default (#2911)

Even if increasing the quality of the output, it has shown to have performance drawbacks to be so noticeable that the confuses users about speed of LocalAI ( see also #2780 ). This changeset disables Mirostat by default (which can be still enabled manually). Signed-off-by: Ettore Di Giacinto <[email protected]> Co-authored-by: Dave <[email protected]>
mudler · Feb 6, 2025 · cc1f6f9 · cc1f6f9
1 parent 7f90ff7
commit cc1f6f9
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/core/config/backend_config.go b/core/config/backend_config.go
@@ -287,7 +287,8 @@ func (cfg *BackendConfig) SetDefaults(opts ...ConfigLoaderOption) {
 	defaultTopP := 0.95
 	defaultTopK := 40
 	defaultTemp := 0.9
-	defaultMirostat := 2
+	// https://github.com/mudler/LocalAI/issues/2780
+	defaultMirostat := 0
 	defaultMirostatTAU := 5.0
 	defaultMirostatETA := 0.1
 	defaultTypicalP := 1.0