Best Tools to Run LLMs Locally

Ollama is the default way to run open models on your own machine, and the tool most other apps plug into. Install it on macOS, Windows or Linux, type ollama run gemma4, and it downloads the model and starts a chat. The desktop app adds a chat window with file drag-and-drop, and a local API on port 11434 accepts OpenAI- and Anthropic-style requests, so coding tools, note apps and front ends such as Open WebUI connect in seconds. The ollama launch command sets up Claude Code, Codex or OpenCode against a local model.

On Macs, Ollama moved to Apple's MLX framework in 2026. In Ollama's own test on an M5 Max, output speed rose from 58 to 112 tokens per second. NVIDIA cards (compute capability 5.0 and up) run through CUDA, and AMD cards through ROCm or Vulkan.

Is Ollama free? Yes for local use: running models on your own hardware is unlimited and the code is MIT-licensed. The paid plans (Pro $20/month, Max $100/month) only cover its optional cloud models.

Pick it if you want the simplest, best-supported local runtime. Skip it if you prefer browsing models in a full graphical app (LM Studio) or need to serve many users at once (vLLM).

LM Studio is the easiest local AI app for people who do not want a terminal. It looks like a normal desktop program: search for a model, pick a size, click download and start chatting. It runs GGUF models through llama.cpp and, on Macs, Apple's MLX format. Developers get a local server with OpenAI- and Anthropic-compatible endpoints, a headless daemon (llmster) for servers, the lms command-line tool, SDKs for JavaScript and Python, and MCP tool support.

In July 2026 the company added LM Studio Bionic, a separate agent app for coding and file work that can use local models or optional paid cloud models. The original LM Studio app is still available and free.

The licence is the main thing to check. LM Studio is closed source. It has been free for work since July 2025, and the current terms allow personal and internal business use, but you may not modify it, reverse engineer it or resell it as a hosted service. Its privacy policy says local chats never leave your device and the app has no telemetry.

Pick it if you want a polished app with built-in model search. Skip it if you need open-source software you can audit or ship inside your own product.

llama.cpp is the engine under much of local AI. LM Studio, Jan, LocalAI and GPT4All run models with it, Ollama uses it for GGUF models, and GGUF, its file format, is the standard way local models are shared. That is why new open models usually work here first. Using it directly gives you the most control over quantization, context length, GPU offload and speculative decoding.

It has also become easier to start with. The project's new home, llama.app, offers a one-line installer and a llama serve command that starts an OpenAI-compatible server with a built-in web chat. Version 0.5.0, released on 23 September 2026, added faster CUDA and Metal paths and new server options, and fresh builds ship several times a day.

It runs on almost anything, from Apple silicon and NVIDIA, AMD and Intel graphics cards to plain CPUs. The MIT licence lets you ship it inside your own product. Its team (ggml.ai) joined Hugging Face in February 2026 and says the project stays fully open source; NVIDIA has since agreed to buy Hugging Face.

Pick it if you want full control, the newest models first, or an engine to build on. Skip it if you want a point-and-click app.

Jan is the best fully open-source alternative to LM Studio. It is a desktop app for macOS, Windows and Linux with a clean chat window, a model hub, projects, file upload and web search, and it runs models locally through llama.cpp or, on Macs, MLX. You can also add cloud models from OpenAI, Anthropic, Gemini, Groq or OpenRouter with your own API keys and switch between local and cloud in the same app.

For developers there is a local API server, a command-line tool and MCP support for connecting tools, plus integrations with Claude Code. Jan's team also ships its own small models, such as Jan-v3-4B and Jan-Code-4B, built for agent tasks.

The code is Apache 2.0, and Jan says it collects no data until you pick tracking settings at first launch. The team says the app has passed 6.7 million downloads.

The main weakness is pace. The latest stable release, v0.8.4, came out on 23 July 2026, two months before we checked, while Ollama and llama.cpp ship every week or every day.

Pick it if you want an open-source, private app that mixes local and cloud models. Skip it if you need the newest model architectures on day one.

AnythingLLM is the best choice if your main goal is chatting with your own documents. Drop PDFs, Word files or web pages into a workspace and it builds a private search index (RAG) on your machine, so answers can draw on your files. Around that sit AI agents with tools such as web search, file editing and scheduled jobs, a meeting assistant that transcribes calls on your computer, and an Android app.

The desktop app is one download for macOS, Windows and Linux. It ships with a built-in model engine, based on Ollama's open-source engine, and can recommend a model for your hardware, so you do not need anything else installed. If you already run Ollama, LM Studio or LocalAI, or want a cloud model, you can connect those instead.

It is MIT-licensed and free on desktop or self-hosted with Docker, which also adds multi-user workspaces. Mintplex Labs sells a hosted cloud version from $50 a month.

Raw speed depends on the engine underneath, and the docs say the built-in engine is not a full Ollama replacement, so power users often pair it with Ollama or LM Studio.

Pick it if you want private document chat and agents with little setup. Skip it if you only need a fast model runner.

vLLM is what you use when a local model has to serve many people at once. Instead of a desktop app, it is a Python server that loads a model onto your GPUs and exposes an OpenAI-compatible API. Its PagedAttention memory system raised throughput 2 to 4 times over earlier serving systems at the same latency in the original research paper, and it batches many requests together so one GPU can serve a whole team.

It loads a wide range of Hugging Face models and supports tool calling, structured outputs, speculative decoding and multi-GPU setups. If you would rather not run the servers yourself, the hosts on our best AI inference providers list sell the same kind of service by the token.

It is not a beginner tool. vLLM officially runs on Linux (Windows users need WSL), wants an NVIDIA GPU with compute capability 7.5 or newer (or AMD ROCm, Intel, or Apple silicon through the vLLM-Metal plugin), and you configure it from the command line. For one person on a laptop, Ollama or LM Studio is much quicker to set up.

Pick it if you are hosting a model for a team, an app or an internal API. Skip it if you just want to chat with a model on your own computer.

LocalAI is a self-hosted, drop-in replacement for cloud AI APIs. It runs as one server that answers OpenAI-, Anthropic-, Ollama- and ElevenLabs-style requests, so existing apps usually need only a new URL. Behind that API it can switch engines per model: llama.cpp, vLLM or MLX for text, Whisper and Parakeet for speech, diffusers for images, plus text-to-speech, voice cloning and vision models.

That breadth is its selling point. One install can cover chat, transcription, image generation and voice for a small team or a home lab, and its model gallery lists more than 1,200 models. The project says every feature has a CPU path, so it works without a GPU, and it can spread work across several machines.

It is MIT-licensed and actively released (v4.10.0 on 17 September 2026). The trade-off is complexity: you set models up through config files and choose backends yourself, and it is less polished than Ollama or LM Studio for one person who only wants a chat window.

Pick it if you want one private server that replaces several cloud APIs. Skip it if you only need a local chat app.

Open WebUI is not a model runner. It is the best self-hosted chat interface to put in front of one: a ChatGPT-style web app that connects to Ollama, vLLM, any OpenAI-compatible server or cloud APIs. Run one Docker command and your household or team gets a shared, private chat site with accounts, permissions, document search (RAG), web search, voice, tools and plugins.

It has grown into an agent platform too. Its Open Terminal feature gives a model a real terminal and file system inside the chat, and it connects to autonomous agents. It has about 153,000 GitHub stars, and releases arrive often (v0.11.4 on 21 September 2026). A desktop app now exists for people who do not want Docker.

Check the licence before a big rollout. Since v0.6.6 Open WebUI uses its own licence: it is free and BSD-style, but deployments with more than 50 users in any 30 days must keep the Open WebUI name and logo unless they buy an enterprise licence. Contributors also sign a contributor licence agreement.

Pick it if you want a private ChatGPT-style site for a team on top of Ollama or vLLM. Skip it if you want a single desktop app with no server to run.

MLX is Apple's own machine learning framework for Apple silicon, and MLX-LM is its Python package for running and fine-tuning language models with it. Install it with pip install mlx-lm, then generate text, chat, or start a local server (mlx_lm.server, with an API similar to OpenAI's) from the terminal. Thousands of ready-converted models sit in the mlx-community group on Hugging Face.

MLX is built around the Mac's unified memory, and Ollama, LM Studio and Jan all use the MLX framework on Apple silicon. Using MLX-LM directly gives you features the apps do not: LoRA and full fine-tuning on your Mac, quantizing and uploading your own models, and spreading one model across several Macs.

The downsides are reach and polish. It is built for Apple silicon (the core MLX framework also has Linux CUDA and CPU builds), there is no app, and its own docs say the built-in server is not meant for production. The last PyPI release was 0.31.3 in April 2026, though the code on GitHub was still being updated in September.

Pick it if you have an Apple silicon Mac and want the newest MLX features or to fine-tune locally. Skip it if you are on Windows or want a point-and-click app.

GPT4All was one of the first easy desktop apps for local AI, and it still installs and runs on Windows, macOS and Linux with a simple chat window. Its LocalDocs feature lets you chat with a folder of files on your computer, it has a local API server, and its MIT licence is as open as it gets. It also runs on old hardware: Nomic lists an Intel Core i3 2nd generation or AMD Bulldozer CPU as the minimum, and there is a build for Snapdragon Windows laptops.

The problem is that it has stopped moving. The last release, v3.10.0, came out on 25 February 2025, and the GitHub repository has had no new code since May 2025, even though users still open issues. Local AI changes every month, and a runner that has not been updated for 19 months is likely to miss newer model families and the faster engines that Ollama, LM Studio and Jan now use.

We keep it on the list because many older guides still recommend it. For now, it is not our pick for new users.

Pick it if it already works for you on an older machine. Skip it if you are starting fresh; choose Jan or LM Studio instead.