feat: vertexai integated with tool calling

Author: diogoncalves

docs/quickstart.mdx

@@ -45,7 +45,7 @@ description: "Let's have you setup LLMstudio"
 Using it in a Python notebook is also fairly simple! Just run the following cell:
 
 ```python
-from llmstudio import LLM
+from llmstudio.providers import LLM
 model = LLM("anthropic/claude-2.1")
 model.chat("What are Large Language Models?")
 ```

examples/01_intro_to_llmstudio.ipynb

@@ -13,7 +13,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llmstudio import LLM\n",
+    "from llmstudio.providers import LLM\n",
     "from pprint import pprint"
    ]
   },
@@ -43,7 +43,7 @@
     {
      "data": {
       "text/plain": [
-       "ChatCompletion(id='47c45235-3cba-498a-9bfa-255c3f1b829a', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='I’m called Assistant. How can I help you today?', refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729090941, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output='I’m called Assistant. How can I help you today?', chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729090942.350497, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 12, 'total_tokens': 16, 'cost_usd': 0.0002, 'latency_s': 1.0681071281433105, 'time_to_first_token_s': 0.911978006362915, 'inter_token_latency_s': 0.011577862959641676, 'tokens_per_second': 13.107299475040659})"
+       "ChatCompletion(id='ec40a65d-77f0-43f3-b78e-510c362a8b65', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\"I'm called Assistant. How can I assist you today?\", refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729179436, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output=\"I'm called Assistant. How can I assist you today?\", chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729179437.0009851, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 12, 'total_tokens': 16, 'cost_usd': 0.0002, 'latency_s': 0.8632550239562988, 'time_to_first_token_s': 0.7578170299530029, 'inter_token_latency_s': 0.007991174856821695, 'tokens_per_second': 15.059281022682017})"
       ]
      },
      "execution_count": 3,
@@ -66,18 +66,7 @@
    "cell_type": "code",
    "execution_count": 4,
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "ChatCompletion(id='65b6f534-3aea-4361-995a-822628ff7950', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\"I'm called Assistant or ChatGPT. How can I help you today?\", refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729090942, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output=\"I'm called Assistant or ChatGPT. How can I help you today?\", chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729090943.4086301, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 16, 'total_tokens': 20, 'cost_usd': 0.00026000000000000003, 'latency_s': 1.0519328117370605, 'time_to_first_token_s': 0.9430949687957764, 'inter_token_latency_s': 0.007242250442504883, 'tokens_per_second': 15.21009690113111})"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "await openai.achat(\"What's your name\", model=\"gpt-4o\")"
    ]
@@ -100,29 +89,31 @@
      "text": [
       "\n",
       "\n",
-      "Space, an immense and enigmatic expanse beyond Earth's atmosphere, captivates the imagination with its bound\n",
+      "Space, the vast expanse beyond Earth's atmosphere, captivates the human imagination with its infinite possibilities\n",
       "\n",
-      "less mysteries and wonders. It holds stars, planets, galaxies, and countless celestial phenomena that both intrigue\n",
+      " and mysteries. It is a realm where celestial bodies such as planets, stars, comets, and\n",
       "\n",
-      " and inspire humanity. The vastness of space is punctuated by brilliant constellations, swirling neb\n",
+      " galaxies reside, each offering a unique insight into the origins and evolution of the universe. Space exploration has\n",
       "\n",
-      "ulae, and the remnants of ancient supernovae, offering clues to the origins of the universe\n",
+      " propelled technological advancements and fostered international collaboration as scientists and engineers strive to unravel the secrets of cosmic phenomena\n",
       "\n",
-      ". Exploration through telescopes and spacecraft expands our understanding, revealing the intricate dance of cosmic bodies and the\n",
+      " like black holes, dark matter, and the potential for extraterrestrial life. This boundless frontier continues\n",
       "\n",
-      " potential for life beyond our planet. Space challenges us to push technological boundaries and ponder profound questions about our\n",
+      " to inspire wonder, pushing the boundaries of human knowledge and ambition as we seek to understand our place in\n",
       "\n",
-      " place in the cosmos, making it an endless frontier of discovery and curiosity.\n",
+      " the cosmos and the potential for future habitation beyond our home planet. With each mission and discovery, space\n",
+      "\n",
+      " challenges and expands our perceptions, fueling the quest for discovery and innovation.\n",
       "\n",
       "## Metrics:\n",
-      "{'cost_usd': 0.002095,\n",
+      "{'cost_usd': 0.0024100000000000002,\n",
       " 'input_tokens': 8,\n",
-      " 'inter_token_latency_s': 0.018547398955733686,\n",
-      " 'latency_s': 3.2096359729766846,\n",
-      " 'output_tokens': 137,\n",
-      " 'time_to_first_token_s': 0.7052650451660156,\n",
-      " 'tokens_per_second': 42.37240644890664,\n",
-      " 'total_tokens': 145}\n"
+      " 'inter_token_latency_s': 0.011506473863279664,\n",
+      " 'latency_s': 2.5703747272491455,\n",
+      " 'output_tokens': 158,\n",
+      " 'time_to_first_token_s': 0.7979397773742676,\n",
+      " 'tokens_per_second': 60.302491444849906,\n",
+      " 'total_tokens': 166}\n"
      ]
     }
    ],
@@ -156,40 +147,33 @@
      "text": [
       "\n",
       "\n",
-      "Space, the vast and seemingly infinite expanse beyond our planet, captivates the imagination and fuels\n",
+      "Space, the vast and seemingly infinite expanse that extends beyond our planet, captivates the imagination\n",
+      "\n",
+      " and fuels our curiosity about the universe. It is a realm where celestial bodies like stars, planets,\n",
       "\n",
-      " human curiosity. It is a realm of celestial wonders, from the swirling galaxies and shimmering stars to the\n",
+      " and galaxies interact through complex gravitational forces, giving rise to phenomena such as black holes, supernovae\n",
       "\n",
-      " enigmatic black holes and majestic nebulae. In this cosmic backdrop, the laws of physics reign supreme\n",
+      ", and cosmic radiation. The sheer scale of space is mind-boggling—vast distances measured in\n",
       "\n",
-      ", governing the motion of planets and the life cycles of stars. Space not only challenges our understanding of\n",
+      " light-years and the potential for countless worlds yet to be discovered. As humanity ventures further into this un\n",
       "\n",
-      " the universe but also invites exploration, as humanity pushes the boundaries of technology to uncover its secrets. As\n",
+      "charted territory with missions to distant planets and explorations of exoplanets, we grapple with fundamental questions\n",
+      "\n",
+      " about our place in the cosmos and the possibility of life beyond Earth. Each new discovery adds to our\n",
+      "\n",
+      " understanding of the origin and evolution of the universe, inspiring dreams of what lies beyond the final frontier.\n",
       "\n",
-      " we gaze at the night sky, we are reminded of our place in the cosmos and the endless possibilities\n",
       "\n",
-      " that lie beyond, beckoning us to explore and discover the mysteries of the universe.\n",
       "\n",
       "## Metrics:\n",
-      "{'cost_usd': 8.46e-05,\n",
+      "{'cost_usd': 9.839999999999999e-05,\n",
       " 'input_tokens': 8,\n",
-      " 'inter_token_latency_s': 0.0063348164523605015,\n",
-      " 'latency_s': 2.652230739593506,\n",
-      " 'output_tokens': 139,\n",
-      " 'time_to_first_token_s': 1.7840969562530518,\n",
-      " 'tokens_per_second': 52.031672033614456,\n",
-      " 'total_tokens': 147}\n"
-     ]
-    },
-    {
-     "ename": "",
-     "evalue": "",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[1;31mThe Kernel crashed while executing code in the current cell or a previous cell. \n",
-      "\u001b[1;31mPlease review the code in the cell(s) to identify a possible cause of the failure. \n",
-      "\u001b[1;31mClick <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info. \n",
-      "\u001b[1;31mView Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details."
+      " 'inter_token_latency_s': 0.014622557163238525,\n",
+      " 'latency_s': 2.8971738815307617,\n",
+      " 'output_tokens': 162,\n",
+      " 'time_to_first_token_s': 0.5567238330841064,\n",
+      " 'tokens_per_second': 55.57139701774939,\n",
+      " 'total_tokens': 170}\n"
      ]
     }
    ],

examples/01_intro_to_llmstudio_core.ipynb

@@ -20,7 +20,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llmstudio_core import LLMCore as LLM\n",
+    "from llmstudio_core.providers import LLMCore as LLM\n",
     "from pprint import pprint"
    ]
   },
@@ -51,7 +51,7 @@
     {
      "data": {
       "text/plain": [
-       "ChatCompletion(id='e67fd10c-e2d4-442d-9140-1d73d5345e78', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='I’m OpenAI’s language model, and I don’t have a personal name, but you can call me Assistant! How can I help you today?', refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729091027, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output='I’m OpenAI’s language model, and I don’t have a personal name, but you can call me Assistant! How can I help you today?', chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729091027.668681, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 31, 'total_tokens': 35, 'cost_usd': 0.000485, 'latency_s': 0.9814362525939941, 'time_to_first_token_s': 0.5442612171173096, 'inter_token_latency_s': 0.013376839458942413, 'tokens_per_second': 33.624190988236926})"
+       "ChatCompletion(id='7047f538-e331-4b0e-82d7-7dee0e628efc', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='I’m called Assistant, but you can name me whatever you like! How can I assist you today?', refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729182272, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output='I’m called Assistant, but you can name me whatever you like! How can I assist you today?', chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729182273.070267, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 21, 'total_tokens': 25, 'cost_usd': 0.000335, 'latency_s': 0.9371509552001953, 'time_to_first_token_s': 0.7164719104766846, 'inter_token_latency_s': 0.009395230900157581, 'tokens_per_second': 24.542470849946167})"
       ]
      },
      "execution_count": 3,
@@ -78,7 +78,7 @@
     {
      "data": {
       "text/plain": [
-       "ChatCompletion(id='2abb7143-6fec-453f-a088-8e68bd024450', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\"I don't have a personal name, but you can call me Assistant. How can I help you today?\", refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729091028, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output=\"I don't have a personal name, but you can call me Assistant. How can I help you today?\", chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729091028.540685, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 22, 'total_tokens': 26, 'cost_usd': 0.00035, 'latency_s': 0.8616046905517578, 'time_to_first_token_s': 0.48931884765625, 'inter_token_latency_s': 0.01686550270427357, 'tokens_per_second': 26.6943764956423})"
+       "ChatCompletion(id='ac79b4be-d765-4b96-949c-8d5663bab6ae', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\"I don't have a personal name, but you can call me Assistant. How can I help you today?\", refusal=None, role='assistant', function_call=None, tool_calls=None))], created=1729182278, model='gpt-4o', object='chat.completion', service_tier=None, system_fingerprint=None, usage=None, chat_input=\"What's your name\", chat_output=\"I don't have a personal name, but you can call me Assistant. How can I help you today?\", chat_output_stream='', context=[{'role': 'user', 'content': \"What's your name\"}], provider='openai', deployment='gpt-4o-2024-08-06', timestamp=1729182283.168303, parameters={}, metrics={'input_tokens': 4, 'output_tokens': 22, 'total_tokens': 26, 'cost_usd': 0.00035, 'latency_s': 5.415894031524658, 'time_to_first_token_s': 5.212275981903076, 'inter_token_latency_s': 0.009226820685646751, 'tokens_per_second': 4.246759605361987})"
       ]
      },
      "execution_count": 4,
@@ -108,33 +108,33 @@
      "text": [
       "\n",
       "\n",
-      "Space, the final frontier, is an awe-inspiring expanse that has captivated human imagination for\n",
+      "Space, the final frontier, is a vast and mysterious expanse that has captivated human curiosity for\n",
       "\n",
-      " centuries. It is a vast, seemingly infinite realm filled with wonders beyond our full comprehension, from radiant\n",
+      " centuries. It stretches beyond the grasp of our everyday experiences, an infinite void punctuated by celestial wonders\n",
       "\n",
-      " stars and swirling galaxies to mysterious black holes and distant, perhaps habitable, planets. Space is not\n",
+      " like stars, galaxies, and planets. The study of space, known as astronomy, allows us to\n",
       "\n",
-      " merely the absence of earthly bounds but a vast, dynamic tapestry woven with the delicate interplay of gravity,\n",
+      " explore the origins of the universe, the intricacies of cosmic formations, and the potential for life beyond\n",
       "\n",
-      " time, and matter. The exploration of space presents unparalleled opportunities for scientific discovery, technological advancement, and\n",
+      " Earth. With advancements in technology, such as powerful telescopes and space probes, we continue to unlock\n",
       "\n",
-      " a deeper understanding of our place in the universe. As we venture further into the cosmos, peering\n",
+      " the secrets of the cosmos, gaining deeper insights into phenomena like black holes, quasars, and super\n",
       "\n",
-      " through sophisticated telescopes and embarking on ambitious missions, space continues to challenge our intellect, inspiring a profound\n",
+      "novae. Space exploration not only fuels scientific discovery but also inspires a sense of wonder and possibility,\n",
       "\n",
-      " sense of curiosity and a relentless pursuit of knowledge — inviting us to explore its boundless possibilities and seek\n",
+      " reminding us of our place in the universe and the boundless potential for discovery that lies beyond our home\n",
       "\n",
-      " answers to some of humanity's most fundamental questions.\n",
+      " planet.\n",
       "\n",
       "## Metrics:\n",
-      "{'cost_usd': 0.002635,\n",
+      "{'cost_usd': 0.0024850000000000002,\n",
       " 'input_tokens': 8,\n",
-      " 'inter_token_latency_s': 0.009682952656465418,\n",
-      " 'latency_s': 2.4021809101104736,\n",
-      " 'output_tokens': 173,\n",
-      " 'time_to_first_token_s': 0.7550098896026611,\n",
-      " 'tokens_per_second': 71.18531301297199,\n",
-      " 'total_tokens': 181}\n"
+      " 'inter_token_latency_s': 0.01804932694376251,\n",
+      " 'latency_s': 3.679605007171631,\n",
+      " 'output_tokens': 163,\n",
+      " 'time_to_first_token_s': 0.754986047744751,\n",
+      " 'tokens_per_second': 44.29823301205141,\n",
+      " 'total_tokens': 171}\n"
      ]
     }
    ],
@@ -174,24 +174,13 @@
       "## Metrics:\n",
       "{'cost_usd': 6.45e-06,\n",
       " 'input_tokens': 3,\n",
-      " 'inter_token_latency_s': 0.02283191680908203,\n",
-      " 'latency_s': 1.0239500999450684,\n",
+      " 'inter_token_latency_s': 0.017167633230035954,\n",
+      " 'latency_s': 1.0735399723052979,\n",
       " 'output_tokens': 10,\n",
-      " 'time_to_first_token_s': 0.7725250720977783,\n",
-      " 'tokens_per_second': 11.719321088638754,\n",
+      " 'time_to_first_token_s': 0.8838629722595215,\n",
+      " 'tokens_per_second': 11.177972231654723,\n",
       " 'total_tokens': 13}\n"
      ]
-    },
-    {
-     "ename": "",
-     "evalue": "",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[1;31mThe Kernel crashed while executing code in the current cell or a previous cell. \n",
-      "\u001b[1;31mPlease review the code in the cell(s) to identify a possible cause of the failure. \n",
-      "\u001b[1;31mClick <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info. \n",
-      "\u001b[1;31mView Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details."
-     ]
     }
    ],
    "source": [
@@ -206,6 +195,13 @@
     "        pprint(chunk.metrics)\n",
     "    i+=1"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {