What "local AI" means

Your words go to one of two places

When you type a question to an AI tool, that question has to reach the thing that answers it. There are only two places that thing can be.

It can be a computer somewhere else, owned by a company. You type, your words travel over the internet, the answer travels back. This is cloud AI. It is how most AI tools you have used work.

Or it can be your own computer. You type, and your words go nowhere. The answering happens on the machine on your desk. This is local AI.

That is the whole difference. Everything else in this course follows from it.

What is actually running

The thing that answers you is called a model. You will meet models properly in the next lesson. For now, one fact is enough: a model is a file.

It is a big file. Qwen publishes a small model called Qwen3-0.6B. Its download page lists the model itself as one file, model.safetensors, at 1.5 GB.

Big ones are much larger. Ollama's own example output shows a 70 billion parameter model taking up 42 GB once it is loaded.

But it is still a file that sits on a disk, the way a photo or a song does. Local AI means that file is on your disk, and a program on your computer reads it to work out an answer. Cloud AI means that file is on a company's disk, and their program reads it.

Backed by Qwen3-0.6B repository files, Ollama FAQ.

Where the file actually sits

This is not a figure of speech. After you download a model you can go and look at it in a folder.

Ollama is one of the common programs for this. Its documentation gives the exact place. On a Mac the models live in a folder named .ollama inside your home folder. On Windows that same folder sits inside your user folder. On Linux it is kept under a shared system folder instead.

The names of the files are plain too. The llama.cpp team publishes OpenAI's gpt-oss-20b as one whole file called gpt-oss-20b-MXFP4.gguf. Its listed size is 12.1 GB.

You can copy that file to another disk. You can back it up. You can delete it and get 12.1 GB back. Nothing about it is magic, and nothing about it needs permission from a website.

Backed by Ollama FAQ, gpt-oss-20b GGUF repository files, GGUF file format specification.

Why anyone bothers

Running a model yourself is more work than opening a website. People do it for three reasons.

• Privacy. Your words never leave the machine, so there is nothing to be stored, read, or leaked by anyone else.
• It keeps working. No internet, no account, no outage. The licence that came with the model still has a say in what you may use it for, which lesson 7 goes into.
• No meter running. Once the file is on your disk, using it costs you electricity. The computer strong enough to run it is a real cost, and you paid that one up front.

Backed by LM Studio offline operation.

And what it costs you

None of that is free. You pay for it in three other ways, and this course will not pretend otherwise.

Your computer has to be strong enough, and the biggest, cleverest models will not fit on it. You have to set it up and keep it tidy. And the model you can run at home is usually not as capable as the best one a company can run in a building full of machines.

That is a real trade, not a flaw. Track 4 of this course is about making it deliberately, work by work, instead of picking a side.

The mistake almost everybody makes

Here is the trap, and it is a fair one to fall into. The program being on your computer does not mean the answering is.

Ollama runs models on your machine. You type ollama run gpt-oss:20b in a terminal window and the work happens on your hardware.

Ollama also offers cloud models. Its documentation says they are "automatically offloaded to Ollama's cloud service". You type ollama run gpt-oss:120b-cloud and the work happens on their hardware.

Same program. Same window. Almost the same line of text. The only tells are the word cloud at the end of the model name, and the fact that you had to sign in to an account first.

Ollama's own FAQ answers the privacy question twice for this reason. It says Ollama runs locally, and that it does not see your prompts or data when you run locally. For cloud models it says it processes your prompts and responses to provide the service.

Both statements are true. They are about different models in the same app. That is why "is it installed on my computer" and "does it answer on my computer" are two questions, not one.

Backed by Ollama cloud models, Ollama FAQ.

How to tell which one you are using

There is a simple test. Turn off your internet and ask it something.

A cloud tool stops. A local tool carries on, because nothing it needs is anywhere else. That single test tells you more than any label on a website.

Run it feature by feature rather than once. Ask a plain question. Then try the search button, or the file upload, or whatever else the app offers. Some apps run a model on your machine and still send some things away.

That is a reasonable design, and Track 4 is largely about building exactly that on purpose. It just means the honest answer to "is this private?" is sometimes "partly", and it is worth knowing which parts.

Where that test breaks down

The internet test has one blind spot, and you should hear it now rather than discover it later.

A model can run on a different computer inside your own home. LM Studio says it can act as a server for your local network, not only for the machine it is installed on. Ollama listens on your own machine by default, but a single setting opens it to other computers nearby.

Turn off your internet in that setup and everything still works. The test says local. Your words did leave your laptop, though. They went to the desktop in the next room.

So the sharper question is not "is this app installed?" It is "which computer works out the answer?" That is the question Track 4 keeps coming back to.

Try this today. Open the AI tool you use, and find the name of the model it is answering with. Then find the folder that model's file is kept in. If there is no file and no folder, the answer is arriving from somewhere else.

Backed by LM Studio offline operation, Ollama FAQ.