Ollama

Ollama is a cross-platform tool that makes it easy to run open-source large language models locally. With a single command, you can download and run models like Gemma, LLaMA, Mistral, Qwen, and DeepSeek on macOS, Windows, Linux, and Docker. Ollama bundles llama.cpp as its inference backend and provides a CLI, REST API, and native client libraries for Python and JavaScript.

Ollama has grown into one of the most popular tools in the local AI ecosystem, with deep integrations into AI coding agents (Claude Code, OpenCode, Codex), chat interfaces (Open WebUI, LibreChat, Lobe Chat), and development tools. It supports GPU acceleration via NVIDIA CUDA, AMD ROCm, and Apple Metal. The project is MIT-licensed and hosted at ollama.com with source at github.com/ollama/ollama.

Install Ollama with a single command on any major platform:

macOS

curl -fsSL https://ollama.com/install.sh | sh

Windows

irm https://ollama.com/install.ps1 | iex

Linux

curl -fsSL https://ollama.com/install.sh | sh

Docker

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Ollama provides a straightforward model management workflow through its CLI. Models are pulled from the Ollama library, which hosts hundreds of popular open models and their variants.

# Run a model (downloads if not present)
ollama run gemma4

# List downloaded models
ollama list

# Pull a specific model
ollama pull llama3.2

# Remove a model
ollama rm llama3.2

# Show model details
ollama show gemma4

Models can be imported from GGUF files, Safetensors, or PyTorch checkpoints using Modelfiles, giving you full control over prompt templates, system messages, and inference parameters. The library includes Gemma 4, LLaMA 3.2/3.3, Mistral, Mixtral, Qwen 2.5, DeepSeek, Phi-4, and many more.

Ollama Launch

The ollama launch command provides one-click integration with AI coding agents and applications, automatically configuring the connection:

ollama launch claude       # Integrate with Claude Code
ollama launch opencode     # Integrate with OpenCode
ollama launch openclaw     # Personal AI assistant across messaging platforms

Ollama runs an HTTP server on port 11434 with a comprehensive REST API for running and managing models. The API is compatible with the OpenAI chat completions format, making it a drop-in replacement for OpenAI in many applications.

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{"role": "user", "content": "Why is the sky blue?"}],
  "stream": false
}'

API Endpoints

Endpoint	Purpose
/api/chat	Chat completions (OpenAI-compatible)
/api/generate	Text generation (non-chat)
/api/embeddings	Generate text embeddings
/api/pull	Download a model
/api/tags	List local models

Ollama has one of the largest ecosystems of any local LLM tool, with integrations spanning AI coding agents, chat interfaces, desktop apps, and development tools.

Category	Examples
Coding Agents	Claude Code, OpenCode, Codex, Copilot CLI, Droid
Web Interfaces	Open WebUI, LibreChat, Lobe Chat, NextChat
Desktop Apps	AnythingLLM, Cherry Studio, Msty, BoltAI
Libraries	ollama-python, ollama-js, LangChain, LlamaIndex
Search & RAG	Perplexica, Dify.AI, AnythingLLM

Ollama uses Modelfiles to define custom models. A Modelfile is a configuration file that specifies the base model, prompt template, system message, parameters (temperature, top-p, context length), and custom license. You can create a Modelfile to customize any existing model or import external GGUF files.

# Example Modelfile
FROM llama3.2
SYSTEM "You are a helpful coding assistant."
PARAMETER temperature 0.3
PARAMETER context_length 16384

ollama create my-coder --file Modelfile
ollama run my-coder

Environment variables like OLLAMA_HOST (default: 0.0.0.0), OLLAMA_MODELS (model storage path), and OLLAMA_KEEP_ALIVE (model unloading timeout) allow server-level configuration. GPU acceleration is auto-detected, or can be configured via OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS.