Build better LLM apps — faster, smarter, production-ready.
A curated, list of 100+ libraries and frameworks for AI engineers building with Large Language Models. This toolkit includes battle-tested tools, frameworks, templates, and reference implementations for developing, deploying, and optimizing LLM-powered systems.
| Tool | Description | Language | License |
|---|---|---|---|
| Pinecone | Managed vector database for production AI applications | API/SDK | Commercial |
| Weaviate | Open-source vector database with GraphQL API | Go | BSD-3 |
| Qdrant | Vector similarity search engine with extended filtering | Rust | Apache-2.0 |
| Chroma | Open-source embedding database for LLM apps | Python | Apache-2.0 |
| Milvus | Cloud-native vector database for scalable similarity search | Go/C++ | Apache-2.0 |
| FAISS | Library for efficient similarity search and clustering | C++/Python | MIT |
| Deep Lake | AI-native data lake with versioned datasets, optimized for embeddings and multimodal storage | Python | Apache-2.0 |
| Vectara | Managed RAG platform with APIs for retrieval and generation | Python/Go | Commercial |
| Tool | Description | Language | License |
|---|---|---|---|
| LangChain | Framework for developing LLM applications | Python/JS | MIT |
| LlamaIndex | Data framework for LLM applications | Python | MIT |
| Haystack | End-to-end NLP framework for production | Python | Apache-2.0 |
| DSPy | Framework for algorithmically optimizing LM prompts | Python | MIT |
| Semantic Kernel | SDK for integrating AI into conventional programming languages | C#/Python/Java | MIT |
| Langflow | Visual no-code platform for building and deploying LLM workflows | Python/TypeScript | MIT |
| Flowise | Drag-and-drop UI for creating LLM chains and agents | TypeScript | MIT |
| Promptflow | Workflow orchestration for LLM pipelines, evaluation, and deployment | Python | MIT |
| Tool | Description | Language | License |
|---|---|---|---|
| Docling | AI-powered toolkit converting PDF, DOCX, PPTX, HTML, images into structured JSON/Markdown with layout, OCR, table, and code recognition | Python | MIT |
| pdfplumber | Drill through PDFs at a character level, extract text & tables, and visually debug extraction | Python | MIT |
| PyMuPDF (fitz) | Lightweight, high-performance PDF parser for text/image extraction and manipulation | Python / C | AGPL-3.0 |
| PDF.js | Browser-based PDF renderer with text extraction capabilities | JavaScript | Apache-2.0 |
| Camelot | Extracts structured tabular data from PDFs into DataFrames and CSVs | Python | MIT |
| Unstructured | Parse PDFs, DOCX, HTML into structured JSON for LLM workflows | Python | Apache-2.0 |
| pdfminer.six | Detailed PDF text extraction and layout analysis | Python | MIT |
| Llama Parse | Structured parsing of PDFs and documents optimized for LLMs | Python | Apache-2.0 |
| MegaParse | Universal parser for PDFs, HTML, and semi-structured documents | Python | Apache-2.0 |
| ExtractThinker | Intelligent document extraction framework with schema mapping | Python | MIT |
| PyMuPDF4LLM | Wrapper around PyMuPDF for LLM-ready text, tables, and image extraction | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| RAGFlow | Open-source RAG engine based on deep document understanding | Python | Apache-2.0 |
| Verba | Retrieval Augmented Generation (RAG) chatbot | Python | BSD-3 |
| PrivateGPT | Interact with documents using local LLMs | Python | Apache-2.0 |
| AnythingLLM | All-in-one AI application for any LLM | JavaScript | MIT |
| Quivr | Your GenAI second brain | Python/TypeScript | Apache-2.0 |
| Jina | Cloud-native neural search framework for multimodal RAG | Python | Apache-2.0 |
| txtai | All-in-one embeddings database for semantic search and workflows | Python | Apache-2.0 |
| FastGraph RAG | Graph-based RAG framework for structured retrieval | Python | MIT |
| Chonkie | Chunking utility for efficient document processing in RAG | Python | - |
| FlashRAG | Low-latency RAG research toolkit with modular design and benchmarks | Python | - |
| Llmware | Lightweight framework for building RAG-based apps | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Evals | OpenAI's framework for creating and running LLM evaluations | Python | MIT |
| Ragas | Evaluation framework for RAG pipelines | Python | Apache-2.0 |
| Opik | DevOps platform for evaluation, monitoring, and observability | Python | Apache-2.0 |
| Phoenix | ML observability for LLM, vision, language, and tabular models | Python | Apache-2.0 |
| DeepEval | LLM evaluation framework for unit testing LLM outputs | Python | Apache-2.0 |
| TruLens | Evaluation and tracking for LLM experiments | Python | MIT |
| UpTrain | Open-source tool to evaluate and improve LLM applications | Python | Apache-2.0 |
| Giskard | Testing framework for ML/LLMs with bias and robustness checks | Python | Apache-2.0 |
| Weave | Experiment tracking, debugging, and logging for LLM workflows | Python | Apache-2.0 |
| Lighteval | Lightweight and fast evaluation framework from Hugging Face | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Hugging Face Hub | Client library for Hugging Face Hub | Python | Apache-2.0 |
| MLflow | Platform for ML lifecycle management | Python | Apache-2.0 |
| Weights & Biases | Developer tools for ML | Python | MIT |
| DVC | Data version control for ML projects | Python | Apache-2.0 |
| ClearML | End-to-end MLOps platform with LLM support | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Firecrawl | AI-powered web crawler that extracts and structures content for LLM pipelines | TypeScript | MIT |
| Scrapy | Fast, high-level web crawling & scraping framework | Python | BSD-3 |
| Playwright | Web automation & scraping with headless browsers | TypeScript/Python/Java/.NET | Apache-2.0 |
| BeautifulSoup | Easy HTML/XML parsing for quick scraping tasks | Python | MIT |
| Selenium | Browser automation framework (supports scraping) | Multiple | Apache-2.0 |
| Newspaper3k | News & article extraction library | Python | MIT |
| Crawl4AI | Fast, lightweight, and modern web crawling & scraping library for AI data pipelines | Python | Apache-2.0 |
| Colly | High-performance scraping framework for Go | Go | BSD-2 |
| Trafilatura | Extract clean text from web pages for LLM training corpora | Python | MIT |
| ScrapeGraphAI | Use LLMs to extract structured data from websites and documents | Python | MIT |
| Crawlee | Web scraping and crawling framework for large-scale data collection | TypeScript | Apache-2.0 |
| Framework | Description | Language | License |
|---|---|---|---|
| Google's ADK | Flexible and modular framework for developing and deploying AI agents | Python / Java | Apache-2.0 |
| AutoGen | Multi-agent conversation framework | Python | CC-BY-4.0 |
| CrewAI | Framework for orchestrating role-playing autonomous AI agents | Python | MIT |
| LangGraph | Build resilient language agents as graphs | Python | MIT |
| AgentOps | Python SDK for AI agent monitoring, LLM cost tracking, benchmarking | Python | MIT |
| Swarm | Educational framework for exploring ergonomic, lightweight multi-agent orchestration | Python | MIT |
| Agency Swarm | An open-source agent framework designed to automate your workflows | Python | MIT |
| Multi-Agent Systems | Research into multi-agent systems and applications | Python | MIT |
| Auto-GPT | Autonomous AI agent for task execution using GPT models | Python | MIT |
| BabyAGI | Task-driven autonomous agent inspired by AGI | Python | MIT |
| SuperAGI | Infrastructure for building and managing autonomous agents | Python | MIT |
| Griptape | Framework for building AI agents with structured pipelines and memory | Python | Apache-2.0 |
| Letta (MemGPT) | Long-term memory management for LLM agents | Python | MIT |
| Agno | Framework for building AI agents with RAG, workflows, and memory | Python | Apache-2.0 |
| Agents SDK | SDK from Vercel for building agentic workflows and applications | TypeScript | Apache-2.0 |
| Smolagents | Lightweight agent framework from Hugging Face | Python | Apache-2.0 |
| Pydantic AI | Agent framework built on Pydantic for structured reasoning | Python | MIT |
| CAMEL | Multi-agent framework enabling role-play and collaboration | Python | Apache-2.0 |
| Swarms | Enterprise agent orchestration framework (“Agency Swarm”) | Python | MIT |
| Langroid | Framework for building multi-agent conversational systems | Python | Apache-2.0 |
| Upsonic | Agent framework focused on context management and tool use | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| PyTorch Lightning | High-level PyTorch interface for LLMs | Python | Apache-2.0 |
| unsloth | Fine-tune LLMs faster with less memory | Python | Apache-2.0 |
| Axolotl | Post-training pipeline for AI models | Python | Apache-2.0 |
| LLaMA-Factory | Easy & efficient LLM fine-tuning | Python | Apache-2.0 |
| PEFT | Parameter-Efficient Fine-Tuning library | Python | Apache-2.0 |
| DeepSpeed | Distributed training & inference optimization | Python | MIT |
| TRL | Train transformer LMs with reinforcement learning | Python | Apache-2.0 |
| Transformers | Pretrained models for text, vision, and audio tasks | Python | Apache-2.0 |
| LitGPT | Train and fine-tune LLMs lightning fast | Python | Apache-2.0 |
| Ludwig | Low-code framework for custom LLMs | Python | Apache-2.0 |
| xTuring | Fast fine-tuning of open-source LLMs | Python | Apache-2.0 |
| RL4LMs | RL library to fine-tune LMs to human preferences | Python | Apache-2.0 |
| torchtune | PyTorch-native library for fine-tuning LLMs | Python | BSD-3 |
| Accelerate | Library to easily train on multiple GPUs/TPUs with mixed precision | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| LLM Compressor | Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment | Python | Apache-2.0 |
| LightLLM | Lightweight Python-based LLM inference and serving framework with easy scalability and high performance | Python | Apache-2.0 |
| vLLM | High-throughput and memory-efficient inference and serving engine for LLMs | Python | Apache-2.0 |
| TensorRT-LLM | NVIDIA library for optimizing LLM inference with TensorRT | C++/Python | Apache-2.0 |
| WebLLM | High-performance in-browser LLM inference engine | TypeScript/Python | Apache-2.0 |
| SkyPilot | Unified framework to run ML workloads and LLMs on any cloud (AWS, GCP, Azure, Lambda, etc.) with auto-spot, data syncing, and cost optimization. | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Guardrails | Add guardrails to large language models | Python | MIT |
| NeMo Guardrails | Toolkit for adding programmable guardrails to LLM conversational systems | Python | Apache-2.0 |
| Garak | LLM vulnerability scanner | Python | MIT |
| DeepTeam | LLM red teaming framework | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Reflex | Build full-stack web apps powered by LLMs with Python-only workflows and reactive UIs. | Python | Apache-2.0 |
| Gradio | Create quick, interactive UIs for LLM demos and prototypes. | Python | Apache-2.0 |
| Streamlit | Build and share AI/ML apps fast with Python scripts and interactive widgets. | Python | Apache-2.0 |
| Taipy | End-to-end Python framework for building production-ready AI apps with dashboards and pipelines. | Python | Apache-2.0 |
| AI SDK UI | Vercel’s AI SDK for building chat & generative UIs | TypeScript | Apache-2.0 |
| Simpleaichat | Minimal Python interface for prototyping conversational LLMs | Python | MIT |
| Chainlit | Framework for building and debugging LLM apps with a rich UI | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Ollama | Get up and running with large language models locally | Go | MIT |
| LM Studio | Desktop app for running local LLMs | - | Commercial |
| GPT4All | Open-source chatbot ecosystem | C++ | MIT |
| LocalAI | Self-hosted OpenAI-compatible API | Go | MIT |
| llama.cpp | Lightweight, high-performance inference engine for running LLMs locally across CPU, GPU, and mobile backends | C++ | MIT |
| LiteLLM | Lightweight OpenAI-compatible gateway for multiple LLM providers | Python | MIT |
| AI Gateway | Gateway for managing LLM requests, caching, and routing | Python | Apache-2.0 |
| Langcorn | Serve LangChain applications via FastAPI with production-ready endpoints | Python | MIT |
| LitServe | High-speed GPU inference server with autoscaling and batch support | Python | Apache-2.0 |
| Platform | Description | Pricing | Features |
|---|---|---|---|
| Clarifai | Lightning-fast compute for AI models & agents | Free tier + Pay-as-you-go | Pre-trained models, Deploy your own models on Dedicated compute, Model training, Workflow automation |
| Modal | Serverless platform for AI/ML workloads | Pay-per-use | Serverless GPU, Auto-scaling |
| Replicate | Run open-source models with a cloud API | Pay-per-use | Pre-built models, Custom training |
| Together AI | Cloud platform for open-source models | Various | Open models, Fine-tuning |
| Anyscale | Ray-based platform for AI applications | Enterprise | Distributed training, Serving |
| Groq | Ultra-fast LPU inference engine for running open AI models | Free tier + Pay-per-use | Blazing fast speed, High throughput, Low latency, Open model support |
| OpenRouter | Universal API to find and route to the best LLMs from various providers | Free tier + Pay-per-use | Multi-provider access, Unified API, Model comparison, Caching |
| RouteLLM | Dynamic router for selecting best LLMs based on cost & performance | Open-source | Cost optimization, Multi-LLM routing |
We welcome contributions! This toolkit grows stronger with community input.
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-tool) - Add your contribution (new tool, template, or tutorial)
- Submit a pull request
- Quality over quantity - Focus on tools and resources that provide real value
- Production-ready - Include tools that work in real-world scenarios
- Well-documented - Provide clear descriptions and usage examples
- Up-to-date - Ensure tools are actively maintained
Get weekly AI engineering insights, tool reviews, and exclusive demos and AI Projects delivered to your inbox:
📧 Subscribe to AI Engineering Newsletter →
Join 100,000+ engineers building better LLM applications
Built with ❤️ for the AI Engineering community
Star ⭐ this repo if you find it helpful!