Add results for new run with grok-2-latest

devlooped · Feb 10, 2025 · e5fec7d · e5fec7d
1 parent cf8a1bf
commit e5fec7d
Show file tree

Hide file tree

Showing 2 changed files with 9 additions and 12 deletions.
diff --git a/readme.md b/readme.md
@@ -7,7 +7,7 @@ Currently, comparison includes:
 
 * OpenAI: gpt-4o, gpt-4o-mini
 * AzureAI: gpt-40, gpt-4o-mini
-* xAI: grok-beta
+* xAI: grok-beta, grok-2-latest
 
 Prompt:
 

diff --git a/...nchmarkDotNet.Artifacts/results/AI.Benchmarks.ModelPerformance-report-github.md b/...nchmarkDotNet.Artifacts/results/AI.Benchmarks.ModelPerformance-report-github.md
@@ -8,14 +8,11 @@ Intel Core i9-10900T CPU 1.90GHz, 1 CPU, 20 logical and 10 physical cores
 
 
 ```
-| Method | Client            | Provider | Model         | Mean    | Error    | StdDev   |
-|------- |------------------ |--------- |-------------- |--------:|---------:|---------:|
-| **Chat**   | **aai-gpt-4o**        | **Azure AI** | **gpt-4o**        | **1.475 s** | **0.0814 s** | **0.2215 s** |
-| **Chat**   | **aai-gpt-4o-mini**   | **Azure AI** | **gpt-4o-mini**   | **1.485 s** | **0.0910 s** | **0.2506 s** |
-| **Chat**   | **oai-gpt-4o**        | **OpenAI**   | **gpt-4o**        | **2.472 s** | **0.2308 s** | **0.6585 s** |
-| **Chat**   | **oai-gpt-4o-mini**   | **OpenAI**   | **gpt-4o-mini**   | **2.636 s** | **0.1912 s** | **0.5456 s** |
-| **Chat**   | **xai-grok-2-latest** | **xAI**      | **grok-2-latest** |      **NA** |       **NA** |       **NA** |
-| **Chat**   | **xai-grok-beta**     | **xAI**      | **grok-beta**     | **1.596 s** | **0.1175 s** | **0.3408 s** |
-
-Benchmarks with issues:
-  ModelPerformance.Chat: DefaultJob [Client=xai-grok-2-latest]
+| Method | Client            | Provider | Model         | Mean    | Error    | StdDev   | Median  |
+|------- |------------------ |--------- |-------------- |--------:|---------:|---------:|--------:|
+| **Chat**   | **aai-gpt-4o**        | **Azure AI** | **gpt-4o**        | **1.536 s** | **0.1220 s** | **0.3298 s** | **1.445 s** |
+| **Chat**   | **aai-gpt-4o-mini**   | **Azure AI** | **gpt-4o-mini**   | **1.691 s** | **0.1988 s** | **0.5608 s** | **1.467 s** |
+| **Chat**   | **oai-gpt-4o**        | **OpenAI**   | **gpt-4o**        | **2.299 s** | **0.1650 s** | **0.4544 s** | **2.287 s** |
+| **Chat**   | **oai-gpt-4o-mini**   | **OpenAI**   | **gpt-4o-mini**   | **2.738 s** | **0.2487 s** | **0.7135 s** | **2.653 s** |
+| **Chat**   | **xai-grok-2-latest** | **xAI**      | **grok-2-latest** | **1.614 s** | **0.1312 s** | **0.3849 s** | **1.565 s** |
+| **Chat**   | **xai-grok-beta**     | **xAI**      | **grok-beta**     | **1.656 s** | **0.1114 s** | **0.3231 s** | **1.676 s** |