generated from mozilla-ai/Blueprint-template
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Update API * Update index * Fix repo_url * Update getting-started * Updates with model loader * Update step-by-step guide * Update defaults * Update contributions * reorder * Lint * Fix import * fix test_workflow * Update README.md * Add demo/notebook * Add heading * Remove outdated text * Update Python version * Update name
- Loading branch information
Showing
8 changed files
with
262 additions
and
27 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,239 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Structured Q&A" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Source code: https://github.com/mozilla-ai/structured-qa" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Docs: https://mozilla-ai.github.io/structured-qa" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## GPU Check" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "First, you'll need to enable GPUs for the notebook:\n", | ||
| "\n", | ||
| "- Navigate to `Edit`→`Notebook Settings`\n", | ||
| "- Select T4 GPU from the Hardware Accelerator section\n", | ||
| "- Click `Save` and accept.\n", | ||
| "\n", | ||
| "Next, we'll confirm that we can connect to the GPU:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import torch\n", | ||
| "\n", | ||
| "if not torch.cuda.is_available():\n", | ||
| " raise RuntimeError(\"GPU not available\")\n", | ||
| "else:\n", | ||
| " print(\"GPU is available!\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Installing dependencies" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%pip install --quiet https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.4-cu122/llama_cpp_python-0.3.4-cp311-cp311-linux_x86_64.whl\n", | ||
| "%pip install --quiet structured-qa" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Uploading data" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from google.colab import files\n", | ||
| "\n", | ||
| "uploaded = files.upload()" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Converting document to a directory of sections" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from pathlib import Path\n", | ||
| "from structured_qa.preprocessing import document_to_sections_dir\n", | ||
| "\n", | ||
| "input_file = list(uploaded.keys())[0]\n", | ||
| "sections_dir = f\"output/{Path(input_file).stem}\"\n", | ||
| "section_names = document_to_sections_dir(input_file, sections_dir)\n", | ||
| "section_names" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Loading model" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from structured_qa.model_loaders import load_llama_cpp_model\n", | ||
| "\n", | ||
| "model = load_llama_cpp_model(\n", | ||
| " \"bartowski/Qwen2.5-3B-Instruct-GGUF/Qwen2.5-3B-Instruct-f16.gguf\"\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Find, Retrieve, and Answer" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "FIND_PROMPT = \"\"\"\n", | ||
| "You are given two pieces of information:\n", | ||
| "1. A user question.\n", | ||
| "2. A list of valid section names.\n", | ||
| "\n", | ||
| "Your task is to:\n", | ||
| "- Identify exactly one `section_name` from the provided list that seems related to the user question.\n", | ||
| "- Return the `section_name` exactly as it appears in the list.\n", | ||
| "- Do NOT return any additional text, explanation, or formatting.\n", | ||
| "- Do NOT combine multiple section names into a single response.\n", | ||
| "\n", | ||
| "Here is the list of valid `section_names`:\n", | ||
| "\n", | ||
| "```\n", | ||
| "{SECTIONS}\n", | ||
| "```\n", | ||
| "\n", | ||
| "Now, based on the input question, return the single most relevant `section_name` from the list.\n", | ||
| "\"\"\"" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "ANSWER_PROMPT = \"\"\"\n", | ||
| "You are a rigorous assistant answering questions.\n", | ||
| "You only answer based on the current information available.\n", | ||
| "\n", | ||
| "The current information available is:\n", | ||
| "\n", | ||
| "```\n", | ||
| "{CURRENT_INFO}\n", | ||
| "```\n", | ||
| "\n", | ||
| "If the current information available is not enough to answer the question,\n", | ||
| "you must return the following message and nothing else:\n", | ||
| "\n", | ||
| "```\n", | ||
| "I need more info.\n", | ||
| "```\n", | ||
| "\"\"\"" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "QUESTION = \"What optimizer was used to train the model?\"" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from structured_qa.workflow import find_retrieve_answer\n", | ||
| "\n", | ||
| "find_retrieve_answer(\n", | ||
| " question=QUESTION,\n", | ||
| " model=model,\n", | ||
| " sections_dir=sections_dir,\n", | ||
| " find_prompt=FIND_PROMPT,\n", | ||
| " answer_prompt=ANSWER_PROMPT,\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": ".venv", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "name": "python", | ||
| "version": "3.10.12" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters