api.mdx

title
Methods

RAGChat exposes several useful methods:

Method	Description
ragChat.chat	automatically retrieves relevant knowledge to include in LLM response
ragChat.context	access or modify your Vector DB
ragChat.history	access or modify your chat history

chat

The chat method enables conversations with a language model (LLM) using your vector database as a knowledge base and chat history.

Key Features:

• Get LLM answers based on your knowledge base
• Custom message history length
• Supports streaming for a better UX
• Customizable hooks for fine-grained control over the chat process
• Optional pure LLM chat mode without RAG

Important Considerations:

• Adjust similarityThreshold to fine-tune accuracy of results
• Balance information: Too much context can lead to inaccuracies
• Use historyLength and topK options to manage the amount of data fed to the LLM

Request

Question to ask your RAG chat Number of previous conversation messages included in the request to LLM

{" "}

Configuration to retain chat history. After the specified time, the history is automatically deleted

{" "}

Value from 0-1 to determine when a piece of information from your knowledge base is considered relevant to the current query (default: 0.5)

{" "}

Number of data points included in request to LLM

{" "}

Details of the rate limit applied (if any)

{" "}

Hook to modify or get data and details of each chunk. Can be used to alter streamed content.

{" "}

Hook to access (and modify) the retrieved context

{" "}

Hook to access (and modify) the conversation history

{" "}

Allows you to disable the inclusion of the knowledge base with query and just chat to the LLM directly

{" "}

Stream AI-responses in real time as they are generated

{" "}

Chat session ID of the user interacting with the application

{" "}

Limit included context elements to a specific namespace (i.e. results for a specific user)

{" "}

Metadata about your chat message. This could be used to store anything in the chat history. By default, the RAG Chat SDK uses this to persist the used model name in the history.

{" "}

Rate limit session ID of the user interacting with the application Provide a custom LLM prompt with access to the user question, chat history and knowledge base items

Response

LLM output as a readable stream or a string Indicates if the output is a stream (mostly for internal use) Metadata attached to item in knowledge base (if any)

Usage Tips:

• Adjust historyLength based on the complexity of your conversations
• Use topK to control the number of relevant context items retrieved
• Enable streaming for a better UX
• Use the onContextFetched and onChatHistoryFetched hooks to inspect and modify data before it's sent to the LLM
• Set disableRAG to true when building custom pipelines or for simple LLM interactions

Example:

const chatResponse = await ragChat.chat({
  input: "What are the key features of our latest product?",
  options: {
    historyLength: 5,
    topK: 3,
    streaming: true,
    onContextFetched: (context) => {
      console.log("Retrieved context:", context)
      return context
    },
    metadata: { source: "product_inquiry" },
  },
})

if (chatResponse.isStream) {
  for await (const chunk of chatResponse.output) {
    console.log(chunk)
  }
} else {
  console.log(chatResponse.output)
}

context

Context describes the items retrieved from your vector database considered relevant for the current query. Add, delete or update data from your knowledge base:

Method	Description
add	add various data types into a vector database.
addMany	add multiple pieces of data at once.
delete	remove data from your knowledge base
deleteEntireContext	clears your entire knowledge base

add

The add method adds various types of context to your application. It supports multiple data formats, automatically handles text splitting, and simplifies adding and managing context.

Supported Data Types:

• Plain text
• Embeddings
• PDF files
• CSV files
• Text files
• HTML content (from files or URLs)

Key Features:

• Automatic text splitting for PDF, CSV, and HTML input
• Flexible configuration options for each data type
• Support for both file and URL sources for HTML content

Request

Available data types: **"text"**, **"embedding"**, **"pdf"**, **"csv"**, **"text-file"**, **"html"**. The data to add. This is required for types **"text"** and **"embedding"**. The source of the file. This is required for types **"pdf"**, **"csv"**, **"text-file"**, and "html" when the source is a file. The URL source. This is required for type **"html"** when the source is a URL. Additional options for adding context. Configuration for text splitting. This is optional for types **"pdf"**, **"text-file"**, and **"html"**. Configuration specific to PDF parsing. Configuration specific to CSV parsing. Configuration specific to HTML parsing. Optional ID for the context.

Response

Indicates if the operation was successful. The IDs of the saved items. The error message if the operation failed.

Usage Tips:

• For large documents, consider using the PDF or text file options to use automatic text splitting.
• When adding CSV data, use the csvConfig to specify which column to use and adjust the delimiter if necessary.
• For web content, you can either specify an HTML file or a URL using the appropriate configuration.
• Use the options parameter to fine-tune how the context is added and processed.

Example:

const result = await context.add({
  type: "pdf",
  fileSource: "./document.pdf",
  pdfConfig: { splitPages: true },
  options: { metadata: { source: "company_report" } },
})

if (result.success) {
  console.log(`Added ${result.ids.length} context items`)
} else {
  console.error(`Failed to add context: ${result.error}`)
}

addMany

The addMany method is an efficient way to add multiple contexts to your application in a single operation. It extends the capabilities of the add method to handle batch processing, making it ideal for large-scale data ingestion.

delete

The delete method allows you to remove one or more context items from your vector database. It provides an easy way to manage your knowledge base by deleting specific items or groups of items.

Key Features:

• Supports deletion of a single item or multiple items in a single operation
• Optional namespace specification for targeted deletion

Request

The ID(s) of the context items to be deleted. Optional namespace to specify which index to delete from. If not specified, the default namespace will be deleted.

Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to finish.

Usage Tips:

• Use a single ID string to delete a single item, or an array of ID strings for batch deletions
• Specify a namespace when working with multiple indices to ensure you're deleting from the correct one
• Always wait for the delete operation to complete before proceeding with other operations.

Example:

// Deleting a single item
await context.delete({
  id: "item_123",
  namespace: "product_catalog",
})

// Deleting multiple items
await context.delete({
  id: ["item_456", "item_789", "item_101"],
  namespace: "customer_data",
})

deleteEntireContext

The deleteEntireContext method removes all knowledge base items, either for a single namespace or for all namespaces.

Key Features:

• Clears your entire knowledge base
• Optional: Specify a namespace for targeted deletion

Request

Optional namespace to specify which index to reset. If not specified, the entire vector database will be reset.

Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to complete.

Usage Tips:

• Use this method with caution, as it will permanently delete all context items.
• Specify a namespace if you only want to reset a specific part of your database.

Example:

// Resetting the entire vector database
await context.deleteEntireContext()
console.log("Entire context deleted successfully")

// Resetting a specific namespace
await context.deleteEntireContext({
  namespace: "temporary_data",
})

This operation is irreversible. Make sure you really want to delete all context items before you before calling this method. It's recommended that you only use this method when you're absolutely sure you want to delete all data.

history

The history feature allows you to manage your chat history. It provides methods to add, retrieve, and delete messages, giving you full control over the conversation data stored in your application.

In typical use, you won't need to manipulate your chat history manually. However, having the ability to directly access and modify messages can be useful in certain scenarios.

Key Features:

• Add individual messages to the chat history
• Retrieve messages from the chat history
• Delete specific messages or entire conversations
• Support for multiple chat sessions
• Optional time-to-live (TTL) for automatic session expiry

addMessage

The addMessage method allows you to add a new message to the chat history for a specific session.

Request

The message object to be added to the chat history. It includes the following fields: Specifies whether the message is from the assistant or the user. The message content. Optional metadata associated with the message. Optional metadata about token usage: Number of input tokens used. Number of output tokens used. Total number of tokens used. Note: The 'id' field is omitted as it's handled internally. Optional identifier for the chat session. If not provided, a default session ID will be used. Optional time-to-live (in seconds) for the chat session. If provided, the session will automatically expire after this duration.

Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to complete.

Usage Tips:

• Ensure the role is set to either "assistant" or "user"
• Use the usage_metadata to track token usage if needed for your use case
• Use unique session IDs to manage multiple concurrent conversations
• Set a TTL for sessions that should automatically expire

Example:

await history.addMessage({
  message: {
    role: "user",
    content: "What's the weather like today?",
    metadata: {
      userLocation: "New York",
      timestamp: Date.now(),
    },
    usage_metadata: {
      input_tokens: 6,
      output_tokens: 0,
      total_tokens: 6,
    },
  },
  sessionId: "user_456_weather_chat",
  sessionTTL: 1800, // Session will expire after 30 minutes
})

deleteMessages

The deleteMessages method allows you to remove all messages associated with a specific chat session.

Request

The identifier of the chat session whose messages you want to delete.

Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to complete.

Usage Tips:

• Use this method to clear the history of a specific chat session.
• Be careful when using this method, as it will permanently delete all messages in the session.

Example:

const sessionIdToDelete = "user_789_support_chat"

await history.deleteMessages({ sessionId: sessionIdToDelete })

console.log(`All messages in session ${sessionIdToDelete} have been deleted`)

This operation is irreversible. Once messages are deleted, they cannot be recovered. Use this method carefully and implement safeguards in your application to prevent accidental data loss.

getMessages

The getMessages method retrieves several messages from a chat session's history. It provides flexibility in selecting the range of messages to fetch and returns them in reverse chronological order (newest first).

Request

Chat session ID from which to retrieve messages. If not provided, a default session ID will be used. The amount of messages to retrieve. The starting index from which to retrieve messages (i.e. for pagination). Defaults to 0 (most recent message).

Response

An array of UpstashMessage objects, where each object contains: Indicates whether the message is from the assistant or the user. The actual content of the message. Additional metadata associated with the message. Metadata about token usage: Number of input tokens used. Number of output tokens used. Total number of tokens used. A unique identifier for the message, generated based on its position in the history.

Usage Tips:

• Use the sessionId parameter to retrieve messages from specific conversations
• Adjust the amount parameter to control how many messages you want to fetch
• Use the startIndex parameter to paginate through the chat history or fetch older messages
• The returned messages are in reverse chronological order (newest first)
• Each message includes an id field, which is generated based on its position in the chat history

Example:

const messages = await history.getMessages({
  sessionId: "user_123_support_chat",
  amount: 10,
  startIndex: 0,
})

console.log(`Retrieved ${messages.length} messages:`)
messages.forEach((msg) => {
  console.log(`${msg.role}: ${msg.content} (ID: ${msg.id})`)
})