all_rag_techniques/explainable_retrieval.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explainable Retrieval in Document Search\n",
    "\n",
    "## Overview\n",
    "\n",
    "This code implements an Explainable Retriever, a system that not only retrieves relevant documents based on a query but also provides explanations for why each retrieved document is relevant. It combines vector-based similarity search with natural language explanations, enhancing the transparency and interpretability of the retrieval process.\n",
    "\n",
    "## Motivation\n",
    "\n",
    "Traditional document retrieval systems often work as black boxes, providing results without explaining why they were chosen. This lack of transparency can be problematic in scenarios where understanding the reasoning behind the results is crucial. The Explainable Retriever addresses this by offering insights into the relevance of each retrieved document.\n",
    "\n",
    "## Key Components\n",
    "\n",
    "1. Vector store creation from input texts\n",
    "2. Base retriever using FAISS for efficient similarity search\n",
    "3. Language model (LLM) for generating explanations\n",
    "4. Custom ExplainableRetriever class that combines retrieval and explanation generation\n",
    "\n",
    "## Method Details\n",
    "\n",
    "### Document Preprocessing and Vector Store Creation\n",
    "\n",
    "1. Input texts are converted into embeddings using OpenAI's embedding model.\n",
    "2. A FAISS vector store is created from these embeddings for efficient similarity search.\n",
    "\n",
    "### Retriever Setup\n",
    "\n",
    "1. A base retriever is created from the vector store, configured to return the top 5 most similar documents.\n",
    "\n",
    "### Explanation Generation\n",
    "\n",
    "1. An LLM (GPT-4) is used to generate explanations.\n",
    "2. A custom prompt template is defined to guide the LLM in explaining the relevance of retrieved documents.\n",
    "\n",
    "### ExplainableRetriever Class\n",
    "\n",
    "1. Combines the base retriever and explanation generation into a single interface.\n",
    "2. The `retrieve_and_explain` method:\n",
    "   - Retrieves relevant documents using the base retriever.\n",
    "   - For each retrieved document, generates an explanation of its relevance to the query.\n",
    "   - Returns a list of dictionaries containing both the document content and its explanation.\n",
    "\n",
    "## Benefits of this Approach\n",
    "\n",
    "1. Transparency: Users can understand why specific documents were retrieved.\n",
    "2. Trust: Explanations build user confidence in the system's results.\n",
    "3. Learning: Users can gain insights into the relationships between queries and documents.\n",
    "4. Debugging: Easier to identify and correct issues in the retrieval process.\n",
    "5. Customization: The explanation prompt can be tailored for different use cases or domains.\n",
    "\n",
    "## Conclusion\n",
    "\n",
    "The Explainable Retriever represents a significant step towards more interpretable and trustworthy information retrieval systems. By providing natural language explanations alongside retrieved documents, it bridges the gap between powerful vector-based search techniques and human understanding. This approach has potential applications in various fields where the reasoning behind information retrieval is as important as the retrieved information itself, such as legal research, medical information systems, and educational tools."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "\n",
    "sys.path.append(os.path.abspath(os.path.join(os.getcwd(), '..'))) # Add the parent directory to the path sicnce we work with notebooks\n",
    "from helper_functions import *\n",
    "from evaluation.evalute_rag import *\n",
    "\n",
    "# Load environment variables from a .env file\n",
    "load_dotenv()\n",
    "\n",
    "# Set the OpenAI API key environment variable\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.getenv('OPENAI_API_KEY')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the explainable retriever class "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ExplainableRetriever:\n",
    "    def __init__(self, texts):\n",
    "        self.embeddings = OpenAIEmbeddings()\n",
    "\n",
    "        self.vectorstore = FAISS.from_texts(texts, self.embeddings)\n",
    "        self.llm = ChatOpenAI(temperature=0, model_name=\"gpt-4o-mini\", max_tokens=4000)\n",
    "\n",
    "        \n",
    "        # Create a base retriever\n",
    "        self.retriever = self.vectorstore.as_retriever(search_kwargs={\"k\": 5})\n",
    "        \n",
    "        # Create an explanation chain\n",
    "        explain_prompt = PromptTemplate(\n",
    "            input_variables=[\"query\", \"context\"],\n",
    "            template=\"\"\"\n",
    "            Analyze the relationship between the following query and the retrieved context.\n",
    "            Explain why this context is relevant to the query and how it might help answer the query.\n",
    "            \n",
    "            Query: {query}\n",
    "            \n",
    "            Context: {context}\n",
    "            \n",
    "            Explanation:\n",
    "            \"\"\"\n",
    "        )\n",
    "        self.explain_chain = explain_prompt | self.llm\n",
    "\n",
    "    def retrieve_and_explain(self, query):\n",
    "        # Retrieve relevant documents\n",
    "        docs = self.retriever.get_relevant_documents(query)\n",
    "        \n",
    "        explained_results = []\n",
    "        \n",
    "        for doc in docs:\n",
    "            # Generate explanation\n",
    "            input_data = {\"query\": query, \"context\": doc.page_content}\n",
    "            explanation = self.explain_chain.invoke(input_data).content\n",
    "            \n",
    "            explained_results.append({\n",
    "                \"content\": doc.page_content,\n",
    "                \"explanation\": explanation\n",
    "            })\n",
    "        \n",
    "        return explained_results\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a mock example and explainable retriever instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Usage\n",
    "texts = [\n",
    "    \"The sky is blue because of the way sunlight interacts with the atmosphere.\",\n",
    "    \"Photosynthesis is the process by which plants use sunlight to produce energy.\",\n",
    "    \"Global warming is caused by the increase of greenhouse gases in Earth's atmosphere.\"\n",
    "]\n",
    "\n",
    "explainable_retriever = ExplainableRetriever(texts)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"Why is the sky blue?\"\n",
    "results = explainable_retriever.retrieve_and_explain(query)\n",
    "\n",
    "for i, result in enumerate(results, 1):\n",
    "    print(f\"Result {i}:\")\n",
    "    print(f\"Content: {result['content']}\")\n",
    "    print(f\"Explanation: {result['explanation']}\")\n",
    "    print()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}