AnythingLLM

AnythingLLM is an all-in-one AI desktop application developed by Mintplex Labs Inc. that brings together document chat, AI agents, and LLM interactions in a single, user-friendly interface. With over 63,200 GitHub stars, it is one of the most popular local AI applications.

The application is designed for users who want to leverage AI without requiring technical setup. It supports connecting to any LLM provider (OpenAI, Anthropic, local models via Ollama, and more), uploading and chatting with documents using RAG, running AI agents with custom tools, and doing it all fully offline and locally. It is available on macOS, Windows, and Linux, as well as a cloud-hosted version and Docker deployment.

AnythingLLM is hosted on GitHub at github.com/Mintplex-Labs/anything-llm and is available under the MIT license.

• Document Chat (RAG) - Upload PDFs, Word docs, text files, code files, and more. Ask questions and get answers grounded in your documents.
• Multi-Model Support - Connect to OpenAI, Anthropic, Google, Ollama, LM Studio, LocalAI, vLLM, and any OpenAI-compatible API.
• AI Agents - Built-in agent system with customizable tools, skills, and system prompts for automated task execution.
• Full Privacy - Run completely offline with local models and local vector storage. No data leaves your machine.
• Custom Models - Supports any model provider, including custom endpoints and self-hosted models.
• Agent Skills - Pre-built skills for web search, code execution, image generation, and more.
• Slash Commands - Quick access to common actions and built-in tools.
• Cross-Platform - Desktop apps for macOS, Windows, Linux, plus Docker and cloud deployment.

AnythingLLM can be installed in multiple ways depending on your preferred setup:

Method	Platform	Best For
Desktop App	macOS, Windows, Linux	Individual users, zero setup
Docker	Any with Docker	Self-hosted teams, servers
Cloud	Web browser	Teams, no infrastructure mgmt

Download the desktop app from anythingllm.com or run with Docker: docker pull mintplexlabs/anythingllm.

The core feature of AnythingLLM is the ability to upload documents and chat with them using Retrieval-Augmented Generation (RAG). You can upload PDFs, Word documents, plain text files, code files, and more. The application processes each document, embeds it into a vector store, and retrieves relevant sections when you ask questions.

Supported document types include PDF, DOCX, TXT, CSV, JSON, MD, code files, and many more. The vector storage can use local options (LanceDB, Chroma) or cloud options (Pinecone, Weaviate, Qdrant). Embedding models can be local or cloud-based. All processing happens on your machine when using local embeddings and models.

AnythingLLM includes a built-in AI agent system that can perform multi-step tasks using tools. The agent system supports custom system prompts that define the agent's behavior, skills that add capabilities like web search or code execution, and slash commands for quick operations.

You can create multiple agents with different configurations and switch between them. The agent marketplace allows browsing and installing community-created skills and prompts. Agents can access your documents, perform web searches, execute code in sandboxed environments, generate images, and interact with external APIs through custom tools.

AnythingLLM is model-agnostic and supports virtually any LLM provider. Key supported providers include:

• Cloud APIs - OpenAI, Anthropic, Google Gemini, Mistral AI, Groq, Together.ai
• Local Inference - Ollama, LM Studio, LocalAI, llama.cpp, vLLM
• Custom Endpoints - Any OpenAI-compatible API endpoint
• Embedding Models - OpenAI, local models via Ollama, sentence-transformers
• Vector Stores - LanceDB, Chroma, Pinecone, Weaviate, Qdrant, Milvus

You can switch between providers and models per workspace, making it easy to use different models for different tasks. The application auto-discovers available models from connected providers.

AnythingLLM competes with several other AI desktop applications and RAG tools. Here is how it stacks up:

Criterion	AnythingLLM	LM Studio	ChatGPT Desktop
License	MIT (open source)	Proprietary (free)	Proprietary
Document RAG	Built-in, multi-format	None	Limited
Local Models	Via Ollama, LM Studio	Built-in (GGUF)	No
AI Agents	Built-in with skills	No	Limited
Privacy	Full local/offline	Full local	Cloud-dependent
Deployment	Desktop + Docker + Cloud	Desktop only	Desktop + Web