What it is

Ollama is an open-source tool for running large language models, built to make running an open model on your own computer as easy as installing ordinary software. It wraps up downloading, format handling, hardware setup and starting a server. Once installed, a single ollama run command fetches a model and drops you into a chat.

Beyond the CLI, Ollama runs a local HTTP server (by default at http://localhost:11434) with its own API plus compatibility with the OpenAI and Anthropic formats. Many apps written for cloud LLMs can therefore switch to a local model just by changing the endpoint.

Its scope has grown. Alongside local models it now offers cloud models that need no local GPU, and it can launch coding agents such as Claude Code, Codex and OpenCode backed by open models. Ollama stresses that anything you run locally never leaves your machine.

It supports macOS, Windows and Linux. The code is on GitHub, and there are official Python and JavaScript libraries.

Key features

  • One command to run a model. ollama run gemma4 downloads the model and starts an interactive session. The library covers open models for coding, vision, embeddings and reasoning.
  • Model management. ollama pull, ollama ls, ollama ps, ollama stop and ollama rm work much like managing Docker images.
  • Local API server. Chat, generate and embedding endpoints, plus OpenAI and Anthropic compatibility for easy migration.
  • Capabilities. The docs cover streaming, thinking, structured outputs, vision, embeddings, tool calling and web search, depending on the model.
  • Custom models with a Modelfile. Set parameters, a system prompt and a template on top of an existing model and package it as your own.
  • Cloud models. Sign in to use hosted models, or call the cloud API directly with an API key and no local install.
  • One-step app integrations. Commands like ollama launch claude start a coding agent on Ollama models. On macOS, Ollama models can also be connected to the Claude and ChatGPT desktop apps.

How it works

At its core Ollama is a long-running local inference server. The CLI, desktop app and third-party programs all talk to it over HTTP. A client sends a request, and the server loads the model into memory or VRAM as needed, runs inference and returns the result, streamed or all at once.

Models are organised as a Modelfile plus weights. The Modelfile is similar in spirit to a Dockerfile, describing a model through a set of instructions:

  • FROM (required): the base model, either an existing model or Safetensors or GGUF weights;
  • PARAMETER: runtime settings such as temperature and context window size (num_ctx);
  • TEMPLATE: the full prompt template;
  • SYSTEM: the system message;
  • plus MESSAGE, LICENSE, CAPABILITY and others.

ollama create packages these with the weights into a runnable model, and ollama show --modelfile prints the Modelfile of any model. The same model in the library usually comes in several sizes and quantisations; gemma4, for example, has tags from e2b to 31b so you can pick one that fits your hardware.

One interface for local and cloud. Ollama’s API has a local and a cloud entry point with matching paths:

  • Native API: http://localhost:11434/api locally, https://ollama.com/api in the cloud;
  • OpenAI compatibility: http://localhost:11434/v1 and https://ollama.com/v1;
  • Anthropic clients: base URL http://localhost:11434 or https://ollama.com, with the client appending /v1/messages.

Local requests need no key; cloud requests carry an API key in the header. After signing in, you can also reach cloud models through your local server. Creating or deleting models only works against a local server. The upshot is that an app can move between local debugging and cloud execution by changing the address, not the code.

How to use it

  1. Install. Download Ollama for macOS, Windows or Linux, run ollama in a terminal and follow the prompts to sign in for cloud models or pick a local one.
  2. Run a model.
ollama run gemma4
  1. Call the API. Local requests need no API key:
curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{"role": "user", "content": "Hello"}]
}'
  1. Use the OpenAI SDK. Set the base URL to http://localhost:11434/v1.
  2. Customise a model. Write a Modelfile:
FROM gemma4
PARAMETER temperature 0.7
SYSTEM You are a patient programming tutor. Keep answers short.

Then run ollama create my-tutor -f Modelfile and start it with ollama run my-tutor.

Use cases

  • Privacy-sensitive data: contracts, medical records or internal code that should not go to the cloud.
  • Offline environments: using LLMs without, or with restricted, network access.
  • Development and testing: debug LLM features against a local model to save on API costs.
  • Backend for coding agents: run Claude Code, OpenCode and similar tools on open models.
  • Learning and experimentation: try different open models and compare them on real tasks.

FAQ

How big a model can my computer run? Mostly a question of RAM and VRAM. The library lists approximate sizes per tag; small gemma4 variants are a few GB, while the largest are close to 20GB. Start small and adjust based on speed and quality.

Do local models cost anything? Ollama says local models are always free. Cloud models have a free tier and paid plans; see the website for pricing.

Is my data uploaded? Not when running locally. For cloud models, Ollama states that prompts are not used for training; see its data handling page for details.

How do I connect an existing app? For most OpenAI-compatible apps, set the base URL to http://localhost:11434/v1 and use a local model name. For Anthropic clients, set the base URL to http://localhost:11434.