labs/openai-agents/openai-agents.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# APIM ❤️ AI Agents\n",
    "\n",
    "## OpenAI Agents Lab\n",
    "\n",
    "![flow](../../images/openai-agents.gif)\n",
    "\n",
    "Playground to try the [OpenAI Agents](https://openai.github.io/openai-agents-python/) with Azure OpenAI models and API based tools through Azure API Management. This enables limitless opportunities for AI agents while maintaining control through Azure API Management!\n",
    "\n",
    "### Prerequisites\n",
    "\n",
    "- [Python 3.12 or later version](https://www.python.org/) installed\n",
    "- [VS Code](https://code.visualstudio.com/) installed with the [Jupyter notebook extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) enabled\n",
    "- [Python environment](https://code.visualstudio.com/docs/python/environments#_creating-environments) with the [requirements.txt](../../requirements.txt) or run `pip install -r requirements.txt` in your terminal\n",
    "- [An Azure Subscription](https://azure.microsoft.com/free/) with [Contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#contributor) + [RBAC Administrator](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#role-based-access-control-administrator) or [Owner](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#owner) roles\n",
    "- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed and [Signed into your Azure subscription](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)\n",
    "\n",
    "▶️ Click `Run All` to execute all steps sequentially, or execute them `Step by Step`...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='0'></a>\n",
    "### 0️⃣ Initialize notebook variables\n",
    "\n",
    "- Resources will be suffixed by a unique string based on your subscription id.\n",
    "- Adjust the location parameters according your preferences and on the [product availability by Azure region.](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?cdn=disable&products=cognitive-services,api-management) \n",
    "- Adjust the OpenAI model and version according the [availability by region.](https://learn.microsoft.com/azure/ai-services/openai/concepts/models) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "metadata": {}
   },
   "outputs": [],
   "source": [
    "import os, sys, json\n",
    "sys.path.insert(1, '../../shared')  # add the shared directory to the Python path\n",
    "import utils\n",
    "\n",
    "deployment_name = os.path.basename(os.path.dirname(globals()['__vsc_ipynb_file__']))\n",
    "resource_group_name = f\"lab-{deployment_name}\" # change the name to match your naming style\n",
    "resource_group_location = \"uksouth\"\n",
    "\n",
    "apim_sku = 'Basicv2'\n",
    "\n",
    "openai_resources = [ {\"name\": \"openai1\", \"location\": \"uksouth\"}]\n",
    "\n",
    "\n",
    "openai_model_name = \"gpt-4o-mini\"\n",
    "openai_model_version = \"2024-07-18\"\n",
    "openai_model_sku = \"GlobalStandard\"\n",
    "openai_deployment_name = \"gpt-4o-mini\"\n",
    "openai_api_version = \"2024-10-21\"\n",
    "\n",
    "utils.print_ok('Notebook initialized')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='1'></a>\n",
    "### 1️⃣ Verify the Azure CLI and the connected Azure subscription\n",
    "\n",
    "The following commands ensure that you have the latest version of the Azure CLI and that the Azure CLI is connected to your Azure subscription."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = utils.run(\"az account show\", \"Retrieved az account\", \"Failed to get the current az account\")\n",
    "\n",
    "if output.success and output.json_data:\n",
    "    current_user = output.json_data['user']['name']\n",
    "    tenant_id = output.json_data['tenantId']\n",
    "    subscription_id = output.json_data['id']\n",
    "\n",
    "    utils.print_info(f\"Current user: {current_user}\")\n",
    "    utils.print_info(f\"Tenant ID: {tenant_id}\")\n",
    "    utils.print_info(f\"Subscription ID: {subscription_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='2'></a>\n",
    "### 2️⃣ Create deployment using 🦾 Bicep\n",
    "\n",
    "This lab uses [Bicep](https://learn.microsoft.com/azure/azure-resource-manager/bicep/overview?tabs=bicep) to declarative define all the resources that will be deployed in the specified resource group. Change the parameters or the [main.bicep](main.bicep) directly to try different configurations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the resource group if doesn't exist\n",
    "utils.create_resource_group(resource_group_name, resource_group_location)\n",
    "\n",
    "# Define the Bicep parameters\n",
    "bicep_parameters = {\n",
    "    \"$schema\": \"https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#\",\n",
    "    \"contentVersion\": \"1.0.0.0\",\n",
    "    \"parameters\": {\n",
    "        \"apimSku\": { \"value\": apim_sku },\n",
    "        \"openAIConfig\": { \"value\": openai_resources },\n",
    "        \"openAIDeploymentName\": { \"value\": openai_deployment_name },\n",
    "        \"openAIModelName\": { \"value\": openai_model_name },\n",
    "        \"openAIModelVersion\": { \"value\": openai_model_version },\n",
    "        \"openAIModelSKU\": { \"value\": openai_model_sku },\n",
    "        \"openAIAPIVersion\": { \"value\": openai_api_version }\n",
    "    }\n",
    "}\n",
    "\n",
    "# Write the parameters to the params.json file\n",
    "with open('params.json', 'w') as bicep_parameters_file:\n",
    "    bicep_parameters_file.write(json.dumps(bicep_parameters))\n",
    "\n",
    "# Run the deployment\n",
    "output = utils.run(f\"az deployment group create --name {deployment_name} --resource-group {resource_group_name} --template-file main.bicep --parameters params.json\",\n",
    "    f\"Deployment '{deployment_name}' succeeded\", f\"Deployment '{deployment_name}' failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='3'></a>\n",
    "### 3️⃣ Get the deployment outputs\n",
    "\n",
    "Retrieve the required outputs from the Bicep deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain all of the outputs from the deployment\n",
    "output = utils.run(f\"az deployment group show --name {deployment_name} -g {resource_group_name}\", f\"Retrieved deployment: {deployment_name}\", f\"Failed to retrieve deployment: {deployment_name}\")\n",
    "\n",
    "if output.success and output.json_data:\n",
    "    app_insights_name = utils.get_deployment_output(output, 'applicationInsightsName', 'Application Insights Name')    \n",
    "    apim_resource_gateway_url = utils.get_deployment_output(output, 'apimResourceGatewayURL', 'APIM API Gateway URL')\n",
    "    openai_subscription_key = utils.get_deployment_output(output, 'openAISubscriptionKey', 'OpenAI Subscription Key (masked)', True).replace(\"Bearer \", \"\")\n",
    "    tools_subscription_key = utils.get_deployment_output(output, 'toolsSubscriptionKey', 'Tools Subscription Key (masked)', True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⚙️ Install OpenAI Agents SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install openai-agents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sdk'></a>\n",
    "### 🧪 Test the API using the OpenAI SDK\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import AzureOpenAI\n",
    "\n",
    "client = AzureOpenAI(\n",
    "    azure_endpoint=apim_resource_gateway_url,\n",
    "    api_key=openai_subscription_key,\n",
    "    api_version=openai_api_version\n",
    ")\n",
    "response = client.chat.completions.create(model=openai_model_name, messages=[\n",
    "                {\"role\": \"system\", \"content\": \"You are a sarcastic, unhelpful assistant.\"},\n",
    "                {\"role\": \"user\", \"content\": \"Can you tell me the time, please?\"}\n",
    "])\n",
    "print(\"💬 \",response.choices[0].message.content)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='basic'></a>\n",
    "### 🧪 Basic test with the Agents SDK\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import AsyncAzureOpenAI\n",
    "from agents import Agent, Runner, set_default_openai_client, set_default_openai_api, set_tracing_disabled\n",
    "import nest_asyncio\n",
    "nest_asyncio.apply()\n",
    "\n",
    "client = AsyncAzureOpenAI(azure_endpoint=apim_resource_gateway_url,\n",
    "                            api_key=openai_subscription_key,\n",
    "                            api_version=openai_api_version)\n",
    "set_default_openai_client(client)\n",
    "set_default_openai_api(\"chat_completions\")\n",
    "agent = Agent(name=\"Sarcastic Assistant\", instructions=\"You are a sarcastic, unhelpful assistant.\", model=openai_deployment_name)\n",
    "\n",
    "result = Runner.run_sync(agent, \"Can you tell me the time, please?\")\n",
    "print(\"💬\", result.final_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='handoffs'></a>\n",
    "### 🧪 Handoffs example\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agents import Agent, Runner\n",
    "import asyncio\n",
    "\n",
    "spanish_agent = Agent(\n",
    "    name=\"Spanish agent\",\n",
    "    instructions=\"You only speak Spanish.\",\n",
    "    model=openai_deployment_name\n",
    ")\n",
    "\n",
    "english_agent = Agent(\n",
    "    name=\"English agent\",\n",
    "    instructions=\"You only speak English\",\n",
    "    model=openai_deployment_name\n",
    ")\n",
    "\n",
    "triage_agent = Agent(\n",
    "    name=\"Triage agent\",\n",
    "    instructions=\"Handoff to the appropriate agent based on the language of the request.\",\n",
    "    handoffs=[spanish_agent, english_agent],\n",
    "    model=openai_deployment_name\n",
    ")\n",
    "\n",
    "async def main():\n",
    "    client = AsyncAzureOpenAI(azure_endpoint=apim_resource_gateway_url,\n",
    "                              api_key=openai_subscription_key,\n",
    "                                api_version=openai_api_version)\n",
    "    set_default_openai_client(client)\n",
    "    set_default_openai_api(\"chat_completions\")\n",
    "    set_tracing_disabled(True)\n",
    "    \n",
    "    result = await Runner.run(triage_agent, input=\"Hola, ¿cómo estás?\")\n",
    "    print(\"💬\", result.final_output)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    asyncio.run(main())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='weatherapi'></a>\n",
    "### 🧪 Run agent with Weather API from Azure API Management"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio, requests\n",
    "from agents import Agent, Runner, function_tool\n",
    "\n",
    "@function_tool\n",
    "def get_weather(city: str) -> str:\n",
    "    response = requests.get(f\"{apim_resource_gateway_url}/weatherservice/weather?city={city}\", headers = {'api-key':tools_subscription_key})\n",
    "    return response.text\n",
    "\n",
    "agent = Agent(\n",
    "    name=\"weather agent\",\n",
    "    instructions=\"You are a helpful assistant that provides wheather information. Always provide the temperature in Celsius.\",\n",
    "    tools=[get_weather],\n",
    "    model=openai_deployment_name\n",
    ")\n",
    "\n",
    "async def main():\n",
    "    client = AsyncAzureOpenAI(azure_endpoint=apim_resource_gateway_url,\n",
    "                              api_key=openai_subscription_key,\n",
    "                              api_version=openai_api_version)\n",
    "    set_default_openai_client(client)\n",
    "    set_default_openai_api(\"chat_completions\")\n",
    "    set_tracing_disabled(True)\n",
    "\n",
    "    result = await Runner.run(agent, input=\"Return a summary of the temperature in Seattle and 3 other sister cities in Europe?\")\n",
    "    print(\"💬\", result.final_output)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    asyncio.run(main())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='logicapp'></a>\n",
    "### 🧪 Run agent with OpenAPI Backend and Logic Apps workflow\n",
    "\n",
    "⚙️ **Tools**:\n",
    "- Get Product Catalog - OpenAPI Backend mocked with an APIM policy.\n",
    "- Place Order - A Logic Apps workflow that processes orders with a maximum of five items.\n",
    "\n",
    "✨ **Expected Behavior**:\n",
    "- The agent receives a user request to order 11 smartphones.\n",
    "- The agent calls the product catalog API to retrieve the product SKU and available stock quantity.\n",
    "- If the order quantity exceeds available stock, the agent will respond that the order cannot be processed due to insufficient stock.\n",
    "- If stock is available, the agent will initiate the order workflow, which will fail because the quantity exceeds the maximum limit of five items.\n",
    "- As the agent was instructed to recover from errors, it will place multiple orders, each with a quantity below the maximum limit, ensuring the total equals the desired order quantity.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio, requests\n",
    "from agents import Agent, Runner, function_tool\n",
    "\n",
    "@function_tool\n",
    "def get_product_catalog(category: str) -> str:\n",
    "    response = requests.get(f\"{apim_resource_gateway_url}/catalogservice/product?category={category}\", headers = {'api-key':tools_subscription_key})\n",
    "    return response.text\n",
    "\n",
    "@function_tool\n",
    "def place_order(sku: str, quantity: int) -> str:\n",
    "    response = requests.post(f\"{apim_resource_gateway_url}/orderservice/PlaceOrder/paths/invoke\", headers = {'api-key':tools_subscription_key}, json={\"sku\": sku, \"quantity\": quantity})\n",
    "    return response.text\n",
    "\n",
    "agent = Agent(\n",
    "    name=\"sales agent\",\n",
    "    instructions=\"You are a helpful sales assistant that helps users order products. Recover from errors if any and proceed with multiple orders if needed without user confirmation to fulfill the total order.\",\n",
    "    tools=[get_product_catalog, place_order],\n",
    "    model=openai_deployment_name\n",
    ")\n",
    "\n",
    "async def main():\n",
    "    client = AsyncAzureOpenAI(azure_endpoint=apim_resource_gateway_url,\n",
    "                              api_key=openai_subscription_key,\n",
    "                              api_version=openai_api_version)\n",
    "    set_default_openai_client(client)\n",
    "    set_default_openai_api(\"chat_completions\")\n",
    "    set_tracing_disabled(True)\n",
    "\n",
    "    result = await Runner.run(agent, input=\"Please order one smartphone for me and one for each of my ten friends.\")\n",
    "    print(\"💬\", result.final_output)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    asyncio.run(main())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='kql'></a>\n",
    "### 🔍 Analyze Application Insights custom metrics with a KQL query\n",
    "\n",
    "With this query you can get the custom metrics that were emitted by Azure APIM. Note that it may take a few minutes for data to become available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "query = \"\\\"\" + \"customMetrics \\\n",
    "| where name == 'Total Tokens' \\\n",
    "| where timestamp >= ago(1h) \\\n",
    "| extend parsedCustomDimensions = parse_json(customDimensions) \\\n",
    "| extend apimSubscription = tostring(parsedCustomDimensions.['Subscription ID']) \\\n",
    "| summarize TotalValue = sum(value) by apimSubscription, bin(timestamp, 1m) \\\n",
    "| order by timestamp asc\" + \"\\\"\"\n",
    "\n",
    "output = utils.run(f\"az monitor app-insights query --app {app_insights_name} -g {resource_group_name} --analytics-query {query}\",\n",
    "    f\"App Insights query succeeded\", f\"App Insights query  failed\")\n",
    "\n",
    "table = output.json_data['tables'][0]\n",
    "df = pd.DataFrame(table.get(\"rows\"), columns = [col.get(\"name\") for col in table.get('columns')])\n",
    "df['timestamp'] = pd.to_datetime(df['timestamp']).dt.strftime('%H:%M')\n",
    "\n",
    "df\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='plot'></a>\n",
    "### 🔍 Plot the custom metrics results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib as mpl\n",
    "mpl.rcParams['figure.figsize'] = [15, 7]\n",
    "if df.empty:\n",
    "    print(\"No data to plot\")\n",
    "else:\n",
    "    df_pivot = df.pivot(index='timestamp', columns='apimSubscription', values='TotalValue')\n",
    "    ax = df_pivot.plot(kind='bar', stacked=True)\n",
    "    plt.title('Total token usage over time by APIM Subscription')\n",
    "    plt.xlabel('Time')\n",
    "    plt.ylabel('Tokens')\n",
    "    plt.legend(title='APIM Subscription')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='clean'></a>\n",
    "### 🗑️ Clean up resources\n",
    "\n",
    "When you're finished with the lab, you should remove all your deployed resources from Azure to avoid extra charges and keep your Azure subscription uncluttered.\n",
    "Use the [clean-up-resources notebook](clean-up-resources.ipynb) for that."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}