diff --git a/.project/spec.yaml b/.project/spec.yaml index acbfcbf..bc5962f 100644 --- a/.project/spec.yaml +++ b/.project/spec.yaml @@ -143,6 +143,23 @@ execution: proxy: trim_prefix: true url: http://localhost:3030 + - name: Code Walkthrough + type: custom + class: webapp + start_command: cd /project/code/walkthrough && python -m http.server --directory + /project/code/walkthrough 8000 + health_check_command: curl -f "http://localhost:8000/" + stop_command: pkill -f "^python -m http.server --directory /project/code/walkthrough" + user_msg: "" + logfile_path: "" + timeout_seconds: 60 + icon_url: "" + webapp_options: + autolaunch: true + port: "8000" + proxy: + trim_prefix: true + url: http://localhost:8000/ resources: gpu: requested: 0 diff --git a/README.md b/README.md index 338a59b..1329d4b 100644 --- a/README.md +++ b/README.md @@ -21,17 +21,18 @@ micro-service](https://docs.nvidia.com/nim/large-language-models/latest/introduc to help our partners and customers build effective RAG pipeline with ease. -NIM Anywhere contains all the tooling required to start integrating NIMs -for RAG. It natively scales out to full-sized labs and up to production -environments. This is great news for building a RAG architecture and -easily adding NIMs as needed. If you're unfamiliar with RAG, it -dynamically retrieves relevant external information during inference -without modifying the model itself. Imagine you're the tech lead of a -company with a local database containing confidential, up-to-date -information. You don’t want OpenAI to access your data, but you need the -model to understand it to answer questions accurately. The solution is -to connect your language model to the database and feed them with the -information. +NIM Anywhere contains all the tooling required to start integrating NIM +microservices for RAG. It natively scales out to full-sized labs and up +to production environments. This is great news for building a RAG +architecture and easily adding NIM microservices as needed. If you're +unfamiliar with RAG, it dynamically retrieves relevant + +external information during inference without modifying the model +itself. Imagine you're the tech lead of a company with a local database +containing confidential, up-to-date information. You don’t want OpenAI +to access your data, but you need the model to understand it to answer +questions accurately. The solution is to connect your language model to +the database and feed them with the information. To learn more about why RAG is an excellent solution for boosting the accuracy and reliability of your generative AI models, [read this diff --git a/code/walkthrough/code_walkthrough.html b/code/walkthrough/code_walkthrough.html new file mode 100644 index 0000000..358d5d2 --- /dev/null +++ b/code/walkthrough/code_walkthrough.html @@ -0,0 +1,162 @@ + + + + + + + + + + How to build a RAG Chain + + + + + +
+

How to build a RAG Chain

+

+EPG TME +

+

2024

+
+
+

Imports

+ +
+
+

Langchain NVIDIA Integration

+ +

LangChain is a framework for developing applications powered by large language models (LLMs). In the process of developing a LLM application, there are many things to consider: which language model to choose, how to load documents, how to embed and retrieve loaded documents, etc. Here LangChain acts as a orchestrator of all the components in the development process and unify them in a single framework to make them compatible with one another.

+

Here the langchain_nvidia_ai_endpoints is one of the integration provided by LangChian. NVIDIA wrapped popular language models in NIMs to make them optimized to deliver the best performance on NVIDIA accelerated infrastructure. One easy way of using NIM is to do it through LangChian endpoint.

+

NIMs provide easy, consistent, and familiar APIs for running optimized inference on an AI models you want.

+
+
+

from langchain_nvidia_ai_endpoints import ChatNVIDIA

+ +
+
+

Other LangChain Libraries

+ +

Here we import other langchain packages. This is a good opportunity to explain other components in our development of a LLM chain.

+
+
+

retriever

+

Packages that are related to retrievers are: - from langchain_core.documents import Document - from langchain.retrievers import ContextualCompressionRetriever - from langchain_milvus.vectorstores.milvus import Milvus

+ +
+
+

Chat history

+ +
+ +
+

Actual Code for the chain

+ +
+
+

Embedding Model

+ +
+
+

Vector store

+ +
+
+

Vector store

+ +
+
+

Document Formatting

+ +
+
+

Language Model Initialization:

+ +
+
+

Document Retrieval Function

+ +
+
+

Question Parsing and Condensing

+ +
+
+

Combining the Chain

+ +
+
+

Configuring Message History

+ +
+ + diff --git a/code/walkthrough/code_walkthrough.txt b/code/walkthrough/code_walkthrough.txt new file mode 100644 index 0000000..edb0dc2 --- /dev/null +++ b/code/walkthrough/code_walkthrough.txt @@ -0,0 +1,189 @@ +% How to build a RAG Chain +% EPG TME +% 2024 + +# Imports + +## Langchain NVIDIA Integration + +- from langchain_nvidia_ai_endpoints import ChatNVIDIA, NVIDIAEmbeddings, NVIDIARerank +- explanation: + +LangChain is a framework for developing applications powered by large language models (LLMs). +In the process of developing a LLM application, there are many things to consider: +which language model to choose, how to load documents, how to embed and retrieve loaded documents, +etc. Here LangChain acts as a orchestrator of all the components in the development process +and unify them in a single framework to make them compatible with one another. + +Here the langchain_nvidia_ai_endpoints is one of the integration provided by LangChian. NVIDIA wrapped +popular language models in NIMs to make them optimized to deliver the best performance on +NVIDIA accelerated infrastructure. One easy way of using NIM is to do it through LangChian endpoint. + +NIMs provide easy, consistent, and familiar APIs for running optimized inference on an AI models you want. + +## from langchain_nvidia_ai_endpoints import ChatNVIDIA +- This class provides access to a NVIDIA NIM for chat. +By default, it connects to a hosted NIM, but can be configured to connect +to a local NIM using the base_url parameter. + +## Other LangChain Libraries + +- from langchain_core.documents import Document +- from langchain.retrievers import ContextualCompressionRetriever +- from langchain_milvus.vectorstores.milvus import Milvus + +- from langchain_community.chat_message_histories import RedisChatMessageHistory +- from langchain_core.runnables.history import RunnableWithMessageHistory + +- from langchain_core.output_parsers import StrOutputParser +- from langchain_core.runnables import RunnablePassthrough, chain + + +Here we import other langchain packages. This is a good opportunity to explain other +components in our development of a LLM chain. + +## retriever +Packages that are related to retrievers are: +- from langchain_core.documents import Document +- from langchain.retrievers import ContextualCompressionRetriever +- from langchain_milvus.vectorstores.milvus import Milvus + +- A retriever is an interface that returns documents given an unstructured query. +In most cases, a retriever relies on a vectorestore. Retriever retrieves documents +from a database that stores the digital representation(vector) of documents. + +- Document in LangChain is the class for storing a piece of text and associated information. + +- The Contextual Compression Retriever +passes queries to the base retriever, takes the initial documents and passes them +through the Document Compressor. The Document Compressor takes a list of documents +and shortens it by reducing the contents of documents or dropping documents altogether. +Here we can use a NVIDIA infrastructure optimized NIM - reranking NIM as a compressor. + +- Milvus is a database that store, index, and manage embedding vectors generated by machine learning models. +We choose Milvus here because it is GPU accelarated. + + +## Chat history +- from langchain_community.chat_message_histories import RedisChatMessageHistory +- from langchain_core.runnables.history import RunnableWithMessageHistory + +- The chat history is sequence of messages. Each message has a role (e.g., "user", "assistant", +ie, from human or from the model), content (e.g., text, multimodal data), and additional metadata. + +- use the RedisChatMessageHistory class from the langchain-redis package to store +and manage chat message history using Redis. Together with RunnableWithMessageHistory, they keep track of the message history. + + +# Actual Code for the chain +## Embedding Model + +- embedding_model = NVIDIAEmbeddings( + model=app_config.embedding_model.name, + base_url=str(app_config.embedding_model.url), + api_key=app_config.nvidia_api_key, +) + +- This creates an embedding model using NVIDIA’s embeddings, which convert text or other data types into numerica + +## Vector store +- vector_store = Milvus( + embedding_function=embedding_model, + connection_args={"uri": app_config.milvus.url}, + collection_name=app_config.milvus.collection_name, + auto_id=True, + timeout=10, +) + +- This block initializes a vector store using Milvus, a database optimized for vector-based retrieval. + +## Vector store + +- retriever = vector_store.as_retriever() +- This converts the vector_store into a retriever object, enabling it to perform similarity searches. Given a query, the retriever can find vectors in Milvus that are close (in semantic space) to the query’s embedding. + +- reranker = NVIDIARerank( + model=app_config.reranking_model.name, + base_url=str(app_config.reranking_model.url), + api_key=app_config.nvidia_api_key, +) +- reranking_retriever = ContextualCompressionRetriever(base_compressor=reranker, base_retriever=retriever) +- This defines a final retrieval pipeline combining the retriever with the reranker. ContextualCompressionRetriever uses the reranker (base_compressor) to compress and order the results from the initial retrieval step (base_retriever). This pipeline returns the most relevant results based on the reranking model’s assessment. + +## Document Formatting + +- def format_docs(docs: list[Document]) -> str: + """Take in a list of docs and concatenate the content, separating by newlines.""" + return "\n\n".join(doc.page_content for doc in docs) +- format_docs is a helper function that takes a list of documents (docs) and concatenates their content with two newline characters in between. +This prepares documents for output in a readable format for the user. + +## Language Model Initialization: +- llm = ChatNVIDIA( + model=app_config.llm_model.name, + curr_mode="nim", + base_url=str(app_config.llm_model.url), + api_key=app_config.nvidia_api_key, +) +- Initializes an NVIDIA-powered language model (ChatNVIDIA) + +## Document Retrieval Function +- @chain +async def retrieve_context(msg, config) -> str: + """The Retrieval part of the RAG chain.""" + use_kb = msg["use_kb"] + use_reranker = msg["use_reranker"] + question = msg["question"] + + if not use_kb: + return "" + + if use_reranker: + return (reranking_retriever | format_docs).invoke(question, config) + + return (retriever | format_docs).invoke(question, config) + +- This asynchronous function (retrieve_context) retrieves relevant documents based on the user’s question. +- If use_kb is False, it returns an empty string (no retrieval). +If use_reranker is True, it uses the reranking_retriever, which rerank the retrieved documents for accuracy, then formats the results. +Otherwise, it uses the basic retriever without reranking + +## Question Parsing and Condensing +- @chain +async def question_parsing(msg, config) -> str: + """Condense the question with chat history""" + + condense_question_prompt = prompts.CONDENSE_QUESTION_TEMPLATE.with_config(run_name="condense_question_prompt") + condensed_chain = condense_question_prompt | llm | StrOutputParser().with_config(run_name="condense_question_chain") + if msg["history"]: + return condensed_chain.invoke(msg, config) + return msg["question"] +- This function condenses the user’s question along with any existing chat history to maintain context. + +## Combining the Chain +- my_chain = ( + { + "context": retrieve_context, + "question": question_parsing, + "history": itemgetter("history"), + } + | RunnablePassthrough().with_config(run_name="LLM Prompt Input") + | prompts.CHAT_PROMPT + | llm +) +- This combines the components into a chain for question-answering: +retrieve_context fetches relevant documents. +question_parsing formats the question with history. +RunnablePassthrough and CHAT_PROMPT set up the prompt format for the language model (llm) to process and respond. + +## Configuring Message History + +- my_chain = RunnableWithMessageHistory( + my_chain, + lambda session_id: RedisChatMessageHistory(session_id, url=str(app_config.redis_dsn)), + input_messages_key="question", + output_messages_key="output", + history_messages_key="history", +).with_types(input_type=ChainInputs) + +- Wraps my_chain with message history using Redis, storing the chat history under a specific session ID for continuity. diff --git a/docs/_SUMMARY.md b/docs/_SUMMARY.md index 0c83b1b..ac812a2 100644 --- a/docs/_SUMMARY.md +++ b/docs/_SUMMARY.md @@ -2,7 +2,8 @@ Please join #cdd-nim-anywhere slack channel if you are a internal user, open an One of the primary benefit of using AI for Enterprises is their ability to work with and learn from their internal data. Retrieval-Augmented Generation ([RAG](https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/)) is one of the best ways to do so. NVIDIA has developed a set of micro-services called [NIM micro-service](https://docs.nvidia.com/nim/large-language-models/latest/introduction.html) to help our partners and customers build effective RAG pipeline with ease. -NIM Anywhere contains all the tooling required to start integrating NIMs for RAG. It natively scales out to full-sized labs and up to production environments. This is great news for building a RAG architecture and easily adding NIMs as needed. If you're unfamiliar with RAG, it dynamically retrieves relevant +NIM Anywhere contains all the tooling required to start integrating NIM microservices for RAG. It natively scales out to full-sized labs and up to production environments. This is great news for building a RAG architecture and easily adding NIM microservices as needed. If you're unfamiliar with RAG, it dynamically retrieves relevant + external information during inference without modifying the model itself. Imagine you're the tech lead of a company with a local database containing confidential, up-to-date information. You don’t want OpenAI to access your data, but you need the model to understand it to answer questions accurately. The solution is to connect your language model to the database and feed them with the information.