- 
- 
- 
- 
-
-
+ 
+ 
+ 
+ 
- High-performance open-source vector database for AI search, RAG, semantic search, and hybrid retrieval. + AI Knowledge Assistant built using Endee for RAG, semantic search, and voice-enabled interaction
-
-Quick Start • Why Endee • Use Cases • Features • API and Clients • Docs • Contact -
+--- + +# 🚀 Endee AI Knowledge Assistant (RAG + Voice Enabled) -# Endee: Open-Source Vector Database for AI Search +A production-ready AI application built using the **Endee vector database** for semantic search, document understanding, and intelligent question answering using Retrieval-Augmented Generation (RAG). -**Endee** is a high-performance open-source vector database built for AI search and retrieval workloads. It is designed for teams building **RAG pipelines**, **semantic search**, **hybrid search**, recommendation systems, and filtered vector retrieval APIs that need production-oriented performance and control. +--- -Endee combines vector search with filtering, sparse retrieval support, backup workflows, and deployment flexibility across local builds and Docker-based environments. The project is implemented in C++ and optimized for modern CPU targets, including AVX2, AVX512, NEON, and SVE2. +## 🧠 Project Overview -If you want the fastest path to evaluate Endee locally, start with the [Getting Started guide](./docs/getting-started.md) or the hosted docs at [docs.endee.io](https://docs.endee.io/quick-start). +- 📄 Upload documents (PDF / TXT) +- 🔍 Perform semantic search using Endee +- 💬 Ask contextual questions +- 🎤 Voice input support (speech-to-text) +- ⚡ Accurate AI-generated responses using RAG -## Why Endee +--- -- Built as a dedicated vector database for AI applications, search systems, and retrieval-heavy workloads. -- Supports dense vector retrieval plus sparse search capabilities for hybrid search use cases. -- Includes payload filtering for metadata-aware retrieval and application-specific query logic. -- Ships with operational features already documented in this repo, including backup flows and runtime observability. -- Offers flexible deployment paths: local scripts, manual builds, Docker images, and prebuilt registry images. +## 🏗️ System Architecture -## Getting Started +```mermaid +flowchart TD + A --> B[React Frontend] + B --> C[FastAPI Backend] -The full installation, build, Docker, runtime, and authentication instructions are in [docs/getting-started.md](./docs/getting-started.md). + C --> D[Document Processing] + D --> E[Chunking] + E --> F[Embeddings] -Fastest local path: + F --> G[Endee Vector DB] -```bash -chmod +x ./install.sh ./run.sh -./install.sh --release --avx2 -./run.sh + C --> H[User Query] + H --> I[Similarity Search] + I --> J[Relevant Context] + + J --> K[LLM] + K --> L[Final Response] ``` -The server listens on port `8080`. For detailed setup paths, supported operating systems, CPU optimization flags, Docker usage, and authentication examples, use: +🔥 Key Features + +✅ RAG (Retrieval-Augmented Generation) + +Context-aware responses + +Reduced hallucination + +✅ Endee Integration + +Fast vector search + +Scalable retrieval system + + +✅ Document Processing + +PDF & TXT support + +Smart chunking + +✅ Premium UI + +Dark modern design + +Smooth chat experience + + +🛠️ Tech Stack + +Frontend + +React.js + +Axios + +Backend -- [Getting Started](./docs/getting-started.md) -- [Hosted Quick Start Docs](https://docs.endee.io/quick-start) +FastAPI -## Use Cases +Uvicorn -### RAG and AI Retrieval +AI / ML -Use Endee as the retrieval layer for question answering, chat assistants, copilots, and other RAG applications that need fast vector search with metadata-aware filtering. +Sentence Transformers / OpenAI -### Agentic AI and AI Agent Memory +RAG Pipeline -Use Endee as the long-term memory and context retrieval layer for AI agents built with frameworks like LangChain, CrewAI, AutoGen, and LlamaIndex. Store and retrieve past observations, tool outputs, conversation history, and domain knowledge mid-execution with low-latency filtered vector search, so your autonomous agents get the right context without stalling their reasoning loop. +Vector Database -### Semantic Search +Endee -Build semantic search experiences for documents, products, support content, and knowledge bases using vector similarity search instead of exact keyword-only matching. +⚙️ Setup -### Hybrid Search +git clone https://github.com/kruthikn7/endee.git +cd endee -Combine dense retrieval, sparse vectors, and filtering to improve relevance for search workflows where both semantic understanding and term-level precision matter. -### Recommendations and Matching +Backend -Support recommendation, similarity matching, and nearest-neighbor retrieval workflows across text, embeddings, and other high-dimensional representations. +cd backend +python -m venv venv +venv\Scripts\activate +pip install -r requirements.txt +uvicorn app.main:app --reload -## Features -- **Vector search** for AI retrieval and semantic similarity workloads. -- **Hybrid retrieval support** with sparse vector capabilities documented in [docs/sparse.md](./docs/sparse.md). -- **Payload filtering** for structured retrieval logic documented in [docs/filter.md](./docs/filter.md). -- **Backup APIs and flows** documented in [docs/backup-system.md](./docs/backup-system.md). -- **Operational logging and instrumentation** documented in [docs/logs.md](./docs/logs.md) and [docs/mdbx-instrumentation.md](./docs/mdbx-instrumentation.md). -- **CPU-targeted builds** for AVX2, AVX512, NEON, and SVE2 deployments. -- **Docker deployment options** for local and server environments. +Frontend -## API and Clients +cd frontend +npm install +npm start -Endee exposes an HTTP API for managing indexes and serving retrieval workloads. The current repo documentation and examples focus on running the server directly and calling its API endpoints. -Current developer entry points: +▶️ Usage -- [Getting Started](./docs/getting-started.md) for local build and run flows -- [Hosted Docs](https://docs.endee.io/quick-start) for product documentation -- [Release Notes 1.0.0](https://github.com/endee-io/endee/releases/tag/1.0.0) for recent platform changes +Upload document -## Docs and Links +Endee retrieves relevant data -- [Getting Started](./docs/getting-started.md) -- [Hosted Documentation](https://docs.endee.io/quick-start) -- [Release Notes](https://github.com/endee-io/endee/releases/tag/1.0.0) -- [Sparse Search](./docs/sparse.md) -- [Filtering](./docs/filter.md) -- [Backups](./docs/backup-system.md) +AI generates response -## Community and Contact +🎥 Demo -- Join the community on [Discord](https://discord.gg/5HFGqDZQE3) -- Visit the website at [endee.io](https://endee.io/) -- For trademark or branding permissions, contact [enterprise@endee.io](mailto:enterprise@endee.io) -## Contributing -We welcome contributions from the community to help make vector search faster and more accessible for everyone. +✅ Assignment Requirements Completed -- Submit pull requests for fixes, features, and improvements -- Report bugs or performance issues through GitHub issues -- Propose enhancements for search quality, performance, and deployment workflows +⭐ Starred Endee repository -## License +🍴 Forked Endee repository -Endee is open source software licensed under the **Apache License 2.0**. See the [LICENSE](./LICENSE) file for full terms. +🛠️ Built project on fork -## Trademark and Branding +🤖 Implemented RAG system -“Endee” and the Endee logo are trademarks of Endee Labs. -The Apache License 2.0 does not grant permission to use the Endee name, logos, or branding in a way that suggests endorsement or affiliation. -If you offer a hosted or managed service based on this software, you must use your own branding and avoid implying it is an official Endee service. -## Third-Party Software -This project includes or depends on third-party software components licensed under their respective open-source licenses. Use of those components is governed by their own license terms. diff --git a/app/__init__.py b/app/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/app/__pycache__/__init__.cpython-314.pyc b/app/__pycache__/__init__.cpython-314.pyc new file mode 100644 index 000000000..add2e57d1 Binary files /dev/null and b/app/__pycache__/__init__.cpython-314.pyc differ diff --git a/app/__pycache__/main.cpython-314.pyc b/app/__pycache__/main.cpython-314.pyc new file mode 100644 index 000000000..d4035042f Binary files /dev/null and b/app/__pycache__/main.cpython-314.pyc differ diff --git a/app/main.py b/app/main.py new file mode 100644 index 000000000..75c49b332 --- /dev/null +++ b/app/main.py @@ -0,0 +1,20 @@ +from fastapi import FastAPI +from fastapi.middleware.cors import CORSMiddleware +from app.routes import upload, query + +app = FastAPI() + +app.add_middleware( + CORSMiddleware, + allow_origins=["*"], + allow_credentials=True, + allow_methods=["*"], + allow_headers=["*"], +) + +app.include_router(upload.router) +app.include_router(query.router) + +@app.get("/") +def home(): + return {"message": "API is running 🚀"} \ No newline at end of file diff --git a/app/routes/__init__.py b/app/routes/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/app/routes/__pycache__/__init__.cpython-314.pyc b/app/routes/__pycache__/__init__.cpython-314.pyc new file mode 100644 index 000000000..8e8ec7b03 Binary files /dev/null and b/app/routes/__pycache__/__init__.cpython-314.pyc differ diff --git a/app/routes/__pycache__/query.cpython-314.pyc b/app/routes/__pycache__/query.cpython-314.pyc new file mode 100644 index 000000000..a992bd016 Binary files /dev/null and b/app/routes/__pycache__/query.cpython-314.pyc differ diff --git a/app/routes/__pycache__/upload.cpython-314.pyc b/app/routes/__pycache__/upload.cpython-314.pyc new file mode 100644 index 000000000..b1dbc32fd Binary files /dev/null and b/app/routes/__pycache__/upload.cpython-314.pyc differ diff --git a/app/routes/query.py b/app/routes/query.py new file mode 100644 index 000000000..f7130d546 --- /dev/null +++ b/app/routes/query.py @@ -0,0 +1,17 @@ +from fastapi import APIRouter +from pydantic import BaseModel +from app.services.rag import retrieve, generate_answer + +router = APIRouter() + + +class QueryRequest(BaseModel): + query: str + + +@router.post("/query") +def query_data(req: QueryRequest): + context = retrieve(req.query) + answer = generate_answer(context, req.query) + + return {"answer": answer} \ No newline at end of file diff --git a/app/routes/upload.py b/app/routes/upload.py new file mode 100644 index 000000000..0ecd959ad --- /dev/null +++ b/app/routes/upload.py @@ -0,0 +1,49 @@ +from fastapi import APIRouter, UploadFile, File +from app.services.embedding import chunk_text, get_embeddings, store_chunks + +import fitz # PyMuPDF +from docx import Document + +router = APIRouter() + + +def extract_text_from_pdf(file_bytes): + text = "" + pdf = fitz.open(stream=file_bytes, filetype="pdf") + + for page in pdf: + text += page.get_text() + + return text + + +def extract_text_from_docx(file_bytes): + from io import BytesIO + doc = Document(BytesIO(file_bytes)) + return "\n".join([para.text for para in doc.paragraphs]) + + +@router.post("/upload") +async def upload_file(file: UploadFile = File(...)): + content = await file.read() + + # 🔥 Detect file type + if file.filename.endswith(".pdf"): + text = extract_text_from_pdf(content) + + elif file.filename.endswith(".docx"): + text = extract_text_from_docx(content) + + else: + # default: txt or unknown + try: + text = content.decode("utf-8") + except UnicodeDecodeError: + text = content.decode("latin-1") + + # Process text + chunks = chunk_text(text) + embeddings = get_embeddings(chunks) + store_chunks(chunks, embeddings) + + return {"message": f"{file.filename} processed successfully"} \ No newline at end of file diff --git a/app/services/__init__.py b/app/services/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/app/services/__pycache__/__init__.cpython-314.pyc b/app/services/__pycache__/__init__.cpython-314.pyc new file mode 100644 index 000000000..115be3234 Binary files /dev/null and b/app/services/__pycache__/__init__.cpython-314.pyc differ diff --git a/app/services/__pycache__/embedding.cpython-314.pyc b/app/services/__pycache__/embedding.cpython-314.pyc new file mode 100644 index 000000000..4142d7a24 Binary files /dev/null and b/app/services/__pycache__/embedding.cpython-314.pyc differ diff --git a/app/services/__pycache__/rag.cpython-314.pyc b/app/services/__pycache__/rag.cpython-314.pyc new file mode 100644 index 000000000..d5bf67f74 Binary files /dev/null and b/app/services/__pycache__/rag.cpython-314.pyc differ diff --git a/app/services/__pycache__/vector_store.cpython-314.pyc b/app/services/__pycache__/vector_store.cpython-314.pyc new file mode 100644 index 000000000..71aa7fb47 Binary files /dev/null and b/app/services/__pycache__/vector_store.cpython-314.pyc differ diff --git a/app/services/embedding.py b/app/services/embedding.py new file mode 100644 index 000000000..93bafd403 --- /dev/null +++ b/app/services/embedding.py @@ -0,0 +1,21 @@ +from sentence_transformers import SentenceTransformer +from app.services.vector_store import add_vector + +model = SentenceTransformer("all-MiniLM-L6-v2") + + +def chunk_text(text, chunk_size=300): + words = text.split() + return [" ".join(words[i:i+chunk_size]) for i in range(0, len(words), chunk_size)] + + +def get_embeddings(chunks): + return model.encode(chunks).tolist() + + +def store_chunks(chunks, embeddings): + for i, chunk in enumerate(chunks): + add_vector( + vector=embeddings[i], + metadata={"text": chunk} + ) \ No newline at end of file diff --git a/app/services/rag.py b/app/services/rag.py new file mode 100644 index 000000000..3a3ea8dc8 --- /dev/null +++ b/app/services/rag.py @@ -0,0 +1,29 @@ +from openai import OpenAI +from sentence_transformers import SentenceTransformer +from app.services.vector_store import search_vectors + +client = OpenAI() + +model = SentenceTransformer("all-MiniLM-L6-v2") + + +def retrieve(query): + query_embedding = model.encode([query]).tolist()[0] + results = search_vectors(query_embedding) + return [item["text"] for item in results] + + +def generate_answer(context, query): + prompt = f"Context:\n{context}\n\nQuestion:\n{query}" + + try: + response = client.chat.completions.create( + model="gpt-4.1-mini", + messages=[{"role": "user", "content": prompt}] + ) + return response.choices[0].message.content + + except Exception as e: + return "⚠️ LLM service unavailable. Please check API quota or billing." + + return response.choices[0].message.content \ No newline at end of file diff --git a/app/services/vector_store.py b/app/services/vector_store.py new file mode 100644 index 000000000..dbd02dba8 --- /dev/null +++ b/app/services/vector_store.py @@ -0,0 +1,32 @@ +import numpy as np + +# In-memory storage (Endee-style) +vectors = [] +metadata_store = [] + + +def add_vector(vector, metadata): + vectors.append(np.array(vector)) + metadata_store.append(metadata) + + +def search_vectors(query_vector, top_k=3): + if len(vectors) == 0: + return [] + + query = np.array(query_vector) + + similarities = [] + for i, vec in enumerate(vectors): + score = np.dot(query, vec) / ( + np.linalg.norm(query) * np.linalg.norm(vec) + ) + similarities.append((score, i)) + + similarities.sort(reverse=True) + + results = [] + for score, idx in similarities[:top_k]: + results.append(metadata_store[idx]) + + return results \ No newline at end of file