cookbook/in_prompt_completion.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Du8uj5YhJow"
      },
      "source": [
        "# In-prompt chat completion\n",
        "\n",
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/masci/banks/blob/main/cookbook/in_prompt_completion.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>\n",
        "\n",
        "Being able to build a prompt dynamically is instrumental for many use cases,\n",
        "from retrieval to few-shots prompting, and very often an LLM can help you\n",
        "generating the text used to render a template.\n",
        "\n",
        "Banks provides the tools to use an LLM directly during the template rendering,\n",
        "reducing the roundtrips needed to create the final prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LTucJvi7Xor_"
      },
      "outputs": [],
      "source": [
        "!pip install banks"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m_fMVIlPrqwl"
      },
      "source": [
        "Let's gather some document that will be used later in the prompt, in this case a plain text version of the \"Prompt Engineering\" page on Wikipedia"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "olXn4w53rqIA"
      },
      "outputs": [],
      "source": [
        "import requests\n",
        "\n",
        "url = 'https://gist.githubusercontent.com/masci/3de92dbe61ba1e78b3f7ccde964dd339/raw/b8339f463cd3dc5397cb23080d89318dca7753ec/prompt_engineering.txt'\n",
        "r = requests.get(url)\n",
        "document = r.text"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rmn1t6gqhJoy"
      },
      "source": [
        "Let's implement prompt chaining with the help of an LLM:\n",
        "- The first part of the prompt will create a list of quotes calling GPT-4o\n",
        "- We use the list of quotes from the previous step to build the second part of the prompt\n",
        "- We use GPT-4o to run the second prompt and get the final answer\n",
        "\n",
        "To do this we'll use the `{% completion %}` tag and will assign its output to local template variables,\n",
        "let's see how."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w4G0sLMsj4Eh"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from banks import Prompt\n",
        "\n",
        "## Banks will call OpenAI already when creating the Prompt object, so we need to set this upfront\n",
        "os.environ[\"OPENAI_API_KEY\"] = \"\"\n",
        "\n",
        "p = Prompt(\"\"\"\n",
        "{# First prompt generate the quotes #}\n",
        "{% set quotes %}\n",
        "{% completion model=\"gpt-4o\" %}\n",
        "  {% chat role=\"system\" %}\n",
        "    You are a helpful assistant. Your task is to help answer a question given in a document.\n",
        "    The first step is to extract quotes relevant to the question from the document,\n",
        "    delimited by ####. Please output the list of quotes using <quotes></quotes>.\n",
        "    Respond with \"No relevant quotes found!\" if no relevant quotes were found.\n",
        "  {% endchat %}\n",
        "  {% chat role=\"user\" %}\n",
        "    ####\n",
        "    {{ document }}\n",
        "    ####\n",
        "  {% endchat %}\n",
        "{% endcompletion %}\n",
        "{% endset %}\n",
        "\n",
        "{# Second and final prompt will use the quotes and do the actual query #}\n",
        "{% set final %}\n",
        "{% completion model=\"gpt-4o\" %}\n",
        "{% chat role=\"system\" %}\n",
        "Given a set of relevant quotes (delimited by <quotes></quotes>) extracted from\n",
        "a document and the original document (delimited by ####), please compose an\n",
        "answer to the question. Ensure that the answer is accurate, has a friendly tone,\n",
        "and sounds helpful.\n",
        "{% endchat %}\n",
        "\n",
        "{% chat role=\"user\" %}\n",
        "####\n",
        "{{ document }}\n",
        "####\n",
        "{{ quotes }}\n",
        "\n",
        "{{ question }}\n",
        "{% endchat %}\n",
        "{% endcompletion %}\n",
        "{% endset %}\n",
        "\n",
        "{# We output the final result #}\n",
        "{{ final }}\n",
        "\"\"\")\n",
        "\n",
        "print(p.text({\"question\": \"What is In-context learning? Limit the answer to one sentence.\", \"document\": document}))"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "banks",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}