How to run AI on your own computer
No subscription, no account, nothing leaving your machine, and it works on a plane. Here is what it actually takes, with real memory numbers instead of "a modern PC", an honest look at every tool worth using including the free ones, and a straight answer on where it falls short.
If you have 16 GB of RAM, you can run a useful AI on your own computer today, for free, and it will be good at summarizing, drafting, rewriting, and answering ordinary questions. If you have a graphics card with 8 GB or more of VRAM, it will be fast enough that you stop thinking about it. What you will not get is a frontier model, and anyone telling you otherwise is selling something.
What local AI actually means
A model is a file. A large one, but a file. Running AI locally means that file sits on your disk, and the arithmetic happens on your own processor or graphics card.
When you use a cloud service, your words travel to a datacenter, a very large model computes a reply, and the text comes back. When you run locally, none of that happens. You can turn off your network and keep working. Nothing is retained by anybody, nothing is trained on, and there is no per-message cost because nobody is renting you a machine.
The catch is the size of the file. Cloud providers run models far larger than anything that fits on a consumer machine, on hardware that costs more than a car. Your hardware sets your ceiling, and the entire practical question of local AI is whether the model that fits under your ceiling is good enough for what you actually do.
Two words worth knowing
Parameters are the model's size, written as 7B or 14B for billions. Bigger is generally more capable and always needs more memory. Quantization compresses those parameters to fewer bits, usually 4, which shrinks a model to roughly a quarter of its original size for a small quality cost. Nearly every model you will download is already quantized, which is why an 8-billion-parameter model is a 5 GB file rather than a 16 GB one.
Why bother, and why not
I run local models daily and I also keep cloud keys configured. Both statements are honest, and the reasons pull in opposite directions.
| Reason to run locally | What it actually gets you |
|---|---|
| Privacy | The text never leaves the machine. This is the one that matters for client work, medical questions, financial documents, anything under an NDA, and anything about your family. |
| No subscription | No monthly fee and no per-token bill. Download once, use it forever, pay only for electricity. |
| It works offline | On a plane, in a basement, during an outage, and in a country where the service is blocked. |
| Nothing is rate limited | Run it all day. Nobody throttles you, deprecates your model, or changes the terms. |
| It cannot be taken away | The file is yours. A model you have downloaded cannot be retired out from under you. |
And the honest case against. A model that fits on your laptop is meaningfully less capable than a frontier model, and the gap is not subtle on hard work. Local models are weaker at long multi-step reasoning, large or unfamiliar codebases, and obscure factual knowledge, and they hallucinate more when pushed past what they know. First-time setup is more friction than opening a website, even with good tools. And your laptop will get warm and your battery will drain, because you are now doing the work the datacenter was doing.
Where the line actually falls
In my own use, an 8B model handles summarizing, drafting, rewriting, extracting from documents, formatting, brainstorming, and ordinary questions well enough that I stopped noticing. The moment I want hard reasoning or unfamiliar code, I feel the difference immediately. Most people's daily work is on the first list, which is why local is more practical than its reputation suggests.
The hardware you actually need
Most guides say "a modern computer with a decent graphics card", which helps nobody. Here are real numbers. The rule is simple: the model has to fit in memory, with room left over.
| Model size | File, at 4-bit | Comfortable on | Good for |
|---|---|---|---|
| 3B | around 2 GB | 8 GB RAM, any machine from the last several years | Quick rewriting and simple questions. Noticeably limited. |
| 7B to 8B | around 4 to 5 GB | 16 GB RAM, or 8 GB of VRAM to be fast | The sweet spot. Genuinely useful for daily work. |
| 13B to 14B | around 8 to 9 GB | 32 GB RAM, or 12 GB of VRAM | Better reasoning and noticeably better writing. |
| 32B | around 20 GB | 24 GB of VRAM, or a lot of patience | Strong, and getting close to useful for real code. |
| 70B | around 42 GB | Two large cards, or a Mac with plenty of unified memory | Excellent, and out of reach for most machines. |
Add headroom on top of the file size. The conversation itself lives in memory too, and a long one can add a gigabyte or more. If a model is exactly the size of your free memory, it will run badly or not at all. Leave a few gigabytes spare.
RAM works, VRAM is fast. A model can run entirely on your processor using ordinary system memory, and on 16 GB of RAM an 8B model works fine. It will produce a few words per second, which is readable but you will watch it. Moving the same model onto a graphics card is typically several times faster, to the point where it arrives quicker than you read. If you are choosing hardware for this, VRAM is the number that matters and more of it beats a faster chip with less.
Apple Silicon is unusually good at this, because the processor and graphics share one pool of memory. A Mac with 32 GB of unified memory can run models that would need an expensive dedicated card on a PC.
Do not guess at your own numbers
Open your task manager or system monitor and look at total memory, and at your graphics card's dedicated memory if you have one. Those two numbers tell you which row of the table above you are on. If you would rather not go looking, the hardware check on our download page reads them in your browser and tells you which size fits, without installing anything or sending the result anywhere.
Every tool worth using
These are the real options, including the free ones that compete with what I sell. I would rather you picked the right one than the one I make.
| Tool | What it is | Pick it if |
|---|---|---|
| llama.cpp free, open source |
The inference engine most of the others are built on. A command-line tool, and the closest thing to bare metal. | You want maximum control, you are building something, or you enjoy this sort of thing. |
| Ollama free, open source |
The most popular way to run a model locally. One command pulls a model and runs it, and it exposes a local API other apps can talk to. | You are comfortable in a terminal. This is the default answer for developers and it deserves to be. |
| LM Studio free |
A desktop app with a model browser and a chat window. Point, click, download, chat. | You want a graphical app, you are happy to choose your own models, and you do not need it to do anything else. |
| Jan, GPT4All free, open source |
Open source desktop chat apps in the same territory as LM Studio, each with a different emphasis. | You want an open source graphical app and want to compare a few before settling. |
| Valence paid, 7-day trial |
A desktop app that runs on-device models and can also use your own cloud keys, with memory you can read, a Kids Mode, and an X-Ray view of why it answered as it did. | You want it to work without research, you want local and cloud in one place, or you want the family side. |
Where I actually sit in that list. Ollama and LM Studio are good and free, and if you are technical and only want a local chat window, use them and keep your money. What I built is for the case where you do not want to research quantization formats to get started, and where you want on-device models and, when you choose, your own OpenAI, Anthropic, Google, or xAI key living in the same app, with a Kids Mode on the family machine. That is a real difference for some people and no difference at all for others, and you can tell which you are from the paragraph above.
The fastest way to start
Whichever tool you choose, the first hour looks the same. This is the order that avoids the usual frustration.
-
Find your two numbers
Total system RAM, and dedicated graphics memory if you have a card. Match them to the table above and you know your model size before you download anything.
-
Start one size below your ceiling
If 14B would just barely fit, start at 8B. A model that runs quickly gets used, and a model that makes your machine crawl gets uninstalled by Thursday. You can always move up once it is working.
-
Expect a large download once
Several gigabytes, and it happens a single time per model. After that it loads from disk in seconds and never touches the network again.
-
Allow it through the firewall on first run
Most local AI tools run a small server on your own machine so the app can talk to the model. Your operating system will ask about it. This is expected and the traffic does not leave the computer.
-
Give it real work on day one
Not "tell me a joke". Paste in a document you actually need summarized, or an email you need rewritten. The gap between local and cloud is smallest exactly on this kind of work, so it is the fairest first impression and the honest test of whether this is worth it for you.
Questions people ask
Is local AI as good as ChatGPT?
No. A model that fits on your machine is smaller than one running on a rack of datacenter hardware, and the difference shows on hard reasoning, long or unfamiliar code, and obscure knowledge.
For summarizing, drafting, rewriting, extracting from documents, and ordinary questions, a good 8B model is genuinely useful and the gap narrows to something most people stop noticing. Judge it on your own work rather than on benchmarks.
Is it really free?
The models are free to download and there is no cost per message, so after the initial download you are paying for electricity. Several of the best tools are free and open source too. Paid applications built on top are selling you an easier experience, not access to the models.
Can I run this without a graphics card?
Yes. Models run on the processor using ordinary system memory, and on 16 GB of RAM an 8B model is perfectly usable. It generates a few words per second, so you will watch it write rather than have it appear at once. A graphics card mainly buys speed, not capability.
Does the model learn from my conversations?
No. The file on your disk is fixed and does not change as you use it. If an app appears to remember things about you, it is storing notes separately and adding them back into the conversation, which is a feature of the app rather than a change to the model. Worth knowing where those notes live and whether you can read and delete them.
Is local AI private enough for confidential work?
A genuinely local model does not send your text anywhere, which is the strongest privacy position available short of not using AI at all. Two cautions. Check whether the app you are using also has cloud features that might route a request outward, and know which mode you are in. And local does not mean encrypted at rest, so the ordinary rules about disk encryption and who can use your computer still apply.
Last reviewed September 2026
Find out what your machine can actually run
The hardware check reads your memory in the browser and tells you which model size fits. Nothing is installed and nothing is sent anywhere.
7-day free trial, no account and no card. One purchase, every platform. Windows is available now. macOS and Android are in private alpha and available on request.