python-recipes/RAG/05_nvidia_ai_rag_redis.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aiHV6ip2f5id"
      },
      "source": [
        "# Accelerating RAG with NVIDIA and Redis\n",
        "\n",
        "[NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B.\n",
        "\n",
        "These models, hosted on the NVIDIA API catalog, are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack.\n",
        "\n",
        "This demo showcases a RAG stack using LangChain, NVIDIA for model serving. Mistral *(Mixtral 8x7B model)*, and Redis.\n",
        "\n",
        "**Get Started Here:**\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/nvidia_ai_rag_redis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "37rbBPKdL09o"
      },
      "source": [
        "## 1. Setup\n",
        "Before we begin, we must install some required libraries, authenticate with NVIDIA, create a Redis database, and initialize other required components.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7MMyrImcPPBo"
      },
      "source": [
        "### Install required libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "collapsed": true,
        "id": "jUI0oAbuTjui",
        "outputId": "ad28b925-5e50-4644-a38b-6de80450153a"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade -q langchain-core langchain-community langchain-nvidia-ai-endpoints\n",
        "%pip install -q \"unstructured[pdf]\" sentence-transformers\n",
        "%pip install -q \"redisvl>=0.4.1\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "pc-IxYu3wnQm"
      },
      "outputs": [],
      "source": [
        "import IPython\n",
        "\n",
        "app = IPython.Application.instance()\n",
        "app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "THynXaAhPRNb"
      },
      "source": [
        "### Connect to NVIDIA hosted LLM and embedding models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8HnbKLcdU_1r",
        "outputId": "6df45f69-eb9c-4e83-bbe7-2069471d31e6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Enter your NVIDIA API key: ··········\n"
          ]
        }
      ],
      "source": [
        "import getpass\n",
        "import os\n",
        "\n",
        "if not os.environ.get(\"NVIDIA_API_KEY\", \"\").startswith(\"nvapi-\"):\n",
        "    nvapi_key = getpass.getpass(\"Enter your NVIDIA API key: \")\n",
        "    assert nvapi_key.startswith(\"nvapi-\"), f\"{nvapi_key[:5]}... is not a valid key\"\n",
        "    os.environ[\"NVIDIA_API_KEY\"] = nvapi_key"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4DvCWkLrUIbG"
      },
      "outputs": [],
      "source": [
        "from langchain_core.messages import ChatMessage\n",
        "from langchain_core.output_parsers import StrOutputParser\n",
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n",
        "\n",
        "# Create LLM instance with the Mistral model\n",
        "llm = ChatNVIDIA(\n",
        "    model=\"mistralai/mixtral-8x22b-instruct-v0.1\",\n",
        "    temperature=0.1,\n",
        "    top_p=1.0,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "id": "8OT2wUiHU3rR",
        "outputId": "e60cf9f7-fc6b-4927-bb31-88b7884cc9c0"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "' \"Remember, every step you take, no matter how small, is a step towards your goal; keep going, you\\'re doing great!\"'"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "llm.invoke(\"Write me a one sentence encouragement.\").content"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Df3349oQWRAB"
      },
      "outputs": [],
      "source": [
        "from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings\n",
        "\n",
        "emb = NVIDIAEmbeddings()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tAzka0TNWVrz",
        "outputId": "4ecd5744-cfd4-4601-cd9c-93a2874fe1a7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "1024"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "len(emb.embed_documents([\"test\"])[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PucGDUf4fHn2"
      },
      "source": [
        "### Fetch sample dataset\n",
        "\n",
        "For the demo we will focus on a chevy colorado truck brochure.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IeNQp1k3fJk1",
        "outputId": "fa11df08-4094-407c-8509-1fb2d41466fc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "--2024-05-23 16:48:12--  https://raw.githubusercontent.com/redis-developer/LLM-Document-Chat/main/docs/2022-chevrolet-colorado-ebrochure.pdf\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 3566101 (3.4M) [application/octet-stream]\n",
            "Saving to: ‘data/2022-chevrolet-colorado-ebrochure.pdf’\n",
            "\n",
            "data/2022-chevrolet 100%[===================>]   3.40M  --.-KB/s    in 0.1s    \n",
            "\n",
            "2024-05-23 16:48:12 (29.8 MB/s) - ‘data/2022-chevrolet-colorado-ebrochure.pdf’ saved [3566101/3566101]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "!mkdir -p 'data/'\n",
        "!wget 'https://raw.githubusercontent.com/redis-developer/LLM-Document-Chat/main/docs/2022-chevrolet-colorado-ebrochure.pdf' -O 'data/2022-chevrolet-colorado-ebrochure.pdf'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p5kx9ePDwwp6"
      },
      "source": [
        "### Install Redis locally\n",
        "If you have a Redis db running elsewhere with [Redis Stack](https://redis.io/docs/about/about-stack/) installed, you don't need to run it on this machine."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vs4KZURX4XpT",
        "outputId": "16754e79-10f1-4122-8ca8-8b14fd3bd205"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb jammy main\n",
            "Starting redis-stack-server, database path /var/lib/redis-stack\n"
          ]
        }
      ],
      "source": [
        "%%sh\n",
        "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
        "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
        "sudo apt-get update  > /dev/null 2>&1\n",
        "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
        "redis-stack-server --daemonize yes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VsNJn75A0VJm"
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "## RAG with NVIDIA hosted LLM and Redis\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QkNqSckkbYVm"
      },
      "source": [
        "### Prepare document for RAG"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HG_w_jPSmXVb",
        "outputId": "b1a0e4ce-eea4-49e5-ffb0-8086655fe4a3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Listing available documents ... ['data/2022-chevrolet-colorado-ebrochure.pdf']\n"
          ]
        }
      ],
      "source": [
        "from langchain.text_splitter import SentenceTransformersTokenTextSplitter\n",
        "from langchain.document_loaders import UnstructuredFileLoader\n",
        "\n",
        "# Load list of pdfs from a folder\n",
        "data_path = f\"data/\"\n",
        "docs = [os.path.join(data_path, file) for file in os.listdir(data_path)]\n",
        "\n",
        "print(\"Listing available documents ...\", docs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BtTSE4ZmmDt5",
        "outputId": "c77a1d49-5faf-4fda-b4ff-9f23a1fe86da"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Done preprocessing. Created 35 chunks\n"
          ]
        }
      ],
      "source": [
        "text_splitter = SentenceTransformersTokenTextSplitter()\n",
        "\n",
        "loader = UnstructuredFileLoader(\n",
        "    docs[0], mode=\"single\", strategy=\"fast\"\n",
        ")\n",
        "\n",
        "# extract, load, and make chunks\n",
        "chunks = loader.load_and_split(text_splitter)\n",
        "\n",
        "print(\"Done preprocessing. Created\", len(chunks), \"chunks\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XMEf_sdvcIT7"
      },
      "source": [
        "### Load documents to Redis vector database"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZJuXwbodbwD5"
      },
      "outputs": [],
      "source": [
        "from langchain_community.vectorstores.redis import Redis\n",
        "\n",
        "vectordb = Redis.from_documents(\n",
        "    documents=chunks,\n",
        "    embedding=emb,\n",
        "    redis_url=\"redis://localhost:6379\"\n",
        "  )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5RXQF_fRcD8i"
      },
      "outputs": [],
      "source": [
        "retriever = vectordb.as_retriever(\n",
        "    search_type=\"similarity_distance_threshold\",\n",
        "    search_kwargs={\"distance_threshold\":0.4}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLxoFY9Y1iIv"
      },
      "source": [
        "### Design RAG prompt\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JfJ_DdvL1hdQ"
      },
      "outputs": [],
      "source": [
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "\n",
        "prompt = ChatPromptTemplate.from_template(\n",
        "    \"\"\"You are an intelligent assistant specializing in the Chevy Colorado\n",
        "    2022. You have access to the car manual and production information and should help\n",
        "    answer users questions based on provided context below. Be truthful and honest --\n",
        "    do not make anything up if it's not clearly provided in the context below.\n",
        "\n",
        "    User Question: {question}\n",
        "\n",
        "    Context:\\n{context}\n",
        "\n",
        "    Answer:\"\"\"\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-7jDDXTjdFqy"
      },
      "source": [
        "### Build RAG chain"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5IiIEF1G1wLM"
      },
      "outputs": [],
      "source": [
        "# Build the RAG chain using LCEL\n",
        "from langchain_core.runnables import RunnableParallel, RunnablePassthrough\n",
        "from langchain_core.pydantic_v1 import BaseModel\n",
        "\n",
        "\n",
        "def format_docs(docs):\n",
        "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
        "\n",
        "\n",
        "chain = (\n",
        "    RunnableParallel({\"context\": retriever | format_docs, \"question\": RunnablePassthrough()})\n",
        "    | prompt\n",
        "    | llm\n",
        "    | StrOutputParser()\n",
        ")\n",
        "\n",
        "\n",
        "# Add typing for input\n",
        "class Question(BaseModel):\n",
        "    __root__: str\n",
        "\n",
        "\n",
        "chain = chain.with_types(input_type=Question)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u5lsMVKc2TsF"
      },
      "source": [
        "Let's test this out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 104
        },
        "id": "VAhZYyf22bW9",
        "outputId": "71c51e6c-bf22-45b3-dcaa-4c21bc9e6cc0"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "' The 2022 Chevy Colorado is available in four models: WT, LT, Z71, and ZR2. These models can come in either an extended cab or crew cab configuration. The engines available are a 2.5L 4-cylinder, a 3.6L V6, and a Duramax 2.8L turbo-diesel engine. The Duramax engine provides up to 7,700 lbs. of towing capacity and can deliver up to 30 mpg on the highway. The ZR2 model is an off-road beast with the capability to conquer tough trails. Additionally, there is an available ZR2 Bison Edition which includes 17-inch AEV.'"
            ]
          },
          "execution_count": 49,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chain.invoke(\"What models are available for the chevy colorado?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oSRYapAa4Oue"
      },
      "source": [
        "Now we can apply this same logic to nodes from our pdf document."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WepiLB5Bd7e2"
      },
      "source": [
        "### Add sources to the response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qQ5FrIo5d-pM"
      },
      "outputs": [],
      "source": [
        "rag_chain_from_docs = (\n",
        "    RunnablePassthrough.assign(context=(lambda x: format_docs(x[\"context\"])))\n",
        "    | prompt\n",
        "    | llm\n",
        "    | StrOutputParser()\n",
        ")\n",
        "\n",
        "rag_chain_with_source = RunnableParallel(\n",
        "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
        ").assign(answer=rag_chain_from_docs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8HqAYMIdd-rd",
        "outputId": "5fe227e5-21ce-4065-b3ae-cd1a9b62862b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'context': [Document(page_content='2022 colorado choose your adventure. the 2022 colorado delivers everything you could ask for in a midsize pickup. engine choices that are powerful and efficient, including an available gm - exclusive duramax® 2. 8l turbo - diesel engine that provides up to 7, 700 lbs. of towing1, 2 muscle. a zr2 off - road beast with the capability to conquer tough trails. and a comfortable interior filled with convenience and technology features. so go ahead. choose your best life in colorado. colorado crew cab zr2 in sand dune metallic with available zr2 dusk special edition. vehicle shown can tow up to 5, 000 lbs. 2, 3 1 requires colorado crew cab short box lt 2wd with available trailering package, lt convenience package and safety package. 2 maximum trailering ratings are intended for comparison purposes only. before you buy a vehicle or use it for trailering, carefully review the trailering section of the owner ’ s manual. the trailering capacity of your specific vehicle may vary. the weight of passengers, cargo and options or accessories may reduce the amount you can trailer. 3 requires available trailering package and automatic locking rear differential on lt ; requires available trailering package on z71. due to current supply chain shortages, certain features shown have limited or late availability, or are no longer available. see the window label or a dealer regarding the features on an individual vehicle. introducing colorado colorado at a glance. four models : wt, lt, z71 and zr2 extended cab or crew cab three engines 2. 5l 4 - cylinder 3. 6l v6 duramax 2. 8l turbo - diesel 30 mpg highway 1 with available diesel engine 7, 700 lbs. maximum trailering weight2 with available diesel engine apple carplay®3 and android auto™4 compatibility available zr2 bison edition includes 17 - inch aev', metadata={'id': 'doc:5313bad541f24eb58d856c7ca8f04fc4:17319c587eaa4dd69eacd57b50ff1a5e', 'source': 'data/2022-chevrolet-colorado-ebrochure.pdf'})],\n",
              " 'question': 'What models are available for the chevy colorado?',\n",
              " 'answer': ' The 2022 Chevy Colorado is available in four models: WT, LT, Z71, and ZR2. These models can come in either an extended cab or crew cab configuration. The engines available are a 2.5L 4-cylinder, a 3.6L V6, and a Duramax 2.8L turbo-diesel. The Colorado can achieve up to 30 mpg on the highway with the available diesel engine. The maximum trailering weight is 7,700 lbs with the available diesel engine. The Colorado also offers Apple CarPlay® and Android Auto™ compatibility. Additionally, there is an available ZR2 Bison Edition which includes 17-inch AEV.'}"
            ]
          },
          "execution_count": 48,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "rag_chain_with_source.invoke(\"What models are available for the chevy colorado?\")"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "gpuType": "T4",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.11.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}