What your computer needs

Three parts do the work

Three parts of your computer decide whether a model runs, and how well it runs.

• Memory, often called RAM. Where the model sits while it is working. This is the ceiling.
• Storage, the disk. Where the file rests when it is not working. This is the shelf.
• The chip, meaning the main processor and the graphics chip if you have one. This is the speed.

Memory is the ceiling

While a model runs, its numbers have to be held in memory. All of them, at the same time.

LM Studio is one of the common apps for running models at home, and it publishes what it wants. On Windows it recommends at least 16 GB of memory, and a graphics card with at least 4 GB of its own.

On a Mac it asks for an Apple Silicon chip and macOS 14 or newer, and recommends 16 GB or more. It is honest about the low end too. An 8 GB Mac can still work, it says, if you stay with smaller models.

The size of the file is a floor, not the answer. The gpt-oss-20b model is a 12.1 GB download in the form the llama.cpp team publishes. OpenAI says that model runs within 16 GB of memory. Expect that gap every time.

Backed by LM Studio system requirements, gpt-oss-20b model card, gpt-oss-20b GGUF repository files.

What happens when it does not fit

Apps prefer to put the whole model on the graphics chip. When it will not fit there, the rest of it goes into ordinary system memory.

Ollama, another common app, lets you see this. Its ps command lists what is loaded, and a Processor column tells you where. It reads 100% GPU, or 100% CPU, or a split such as 48%/52%.

A split is not an error. It is your computer telling you the model was too big for the fast part, so some of it is sitting on the slower part.

Ollama's own example output is worth a look for scale. It shows a 70 billion parameter model taking 42 GB.

Remember that your operating system, your browser and everything else share the same memory. Whatever the total says, you do not get all of it.

Backed by Ollama FAQ.

Storage is the shelf, not the ceiling

Storage is the part people forget until it bites, so here it is with real figures.

You have already met one of them: 12.1 GB for a single model file. Three or four of those and a small disk is gone.

You can find them. Ollama keeps its models in a folder named .ollama in your home folder on a Mac, and in your user folder on Windows. On Linux it uses a shared system folder instead. A setting called OLLAMA_MODELS moves them somewhere else, if you would rather keep them on another drive.

The mistake here is quiet and almost universal. You download three models to compare them, never delete two of them, and one day the disk is full.

Keep the difference clear in your head. Storage does not decide what you can run. It decides how many you can keep. One of those is fixed by deleting a file. The other is fixed by buying a machine.

Backed by Ollama FAQ, gpt-oss-20b GGUF repository files.

Your conversation costs memory too

The model is not the only thing in memory. What you have typed is in there with it.

Ollama defines context length as the number of tokens a model has access to in memory. It then says the plain part out loud: setting a larger context length increases the memory needed to run the model.

How much you get is decided for you, based on your graphics memory. Ollama's docs give the steps. Under 24 GiB of it you get 4k of context. Between 24 and 48 you get 32k. At 48 and above you get 256k.

This is why LM Studio tells owners of 8 GB Macs to stay with smaller models and modest context sizes. It is the same limit wearing a different hat.

You can watch it. Ollama's ps output has a context column beside the size. Its example line shows a 9.6 GB model, entirely on the graphics chip, with a context of 131072.

So the room you need is the model plus the conversation, and the second part grows while you work. Lesson thirteen is about living with that.

Backed by Ollama context length, LM Studio system requirements.

The chip is the speed

Once a model fits, the chip decides how quickly the words appear.

The llama.cpp project measured one 7 billion parameter model, in one compact form, across Apple machines. It wrote about 14 tokens a second on an M1, and about 83 on the fastest M4 Max they list. Same file, same software, very different experience.

The table also lists memory bandwidth beside each machine, meaning how fast it can read from its own memory. That M1 row is 68 GB a second. The fastest M4 Max row is 546. The speeds track those figures closely, which is the clue to what is really happening: the chip spends its time reading the model.

Some chips are ruled out before you start. On Windows, LM Studio requires AVX2 support on 64-bit Intel and AMD processors. It also says Intel-based Macs are not currently supported.

The llama.cpp project lists what it is tuned for: Apple silicon through Metal, and AVX, AVX2, AVX512 and AMX on x86 chips. If your processor is older than those, this is where you will find out.

Backed by Performance of llama.cpp on Apple Silicon M-series, LM Studio system requirements, llama.cpp: LLM inference in C/C++.

Find your own two numbers

None of this needs guessing. Both numbers are a few clicks away, and the makers of your computer document where.

On a Mac, for memory: open Activity Monitor and click Memory. Apple's guide says the Physical Memory figure is the amount of RAM installed. Below it, Memory Used is how much of that is in use right now.

On a Mac, for storage: choose the Apple menu, then System Settings, then General, then Storage, then All Volumes. Apple says the light grey part of the bar is your free space.

On Windows, for memory: right-click Start and open Settings, choose System, then About. Microsoft says the amount of RAM installed is listed under Device specifications, along with your processor.

On Windows, for storage: the same Settings app, then System, then Storage. Microsoft adds that if you have several drives, Advanced storage settings shows what is free on all of them at once.

Write both numbers down. Everything left in this lesson is arithmetic on those two.

Backed by View memory usage in Activity Monitor on Mac, See used and available storage space on your Mac, Find information about your Windows device.

A check you can do before downloading

Now the arithmetic. Six steps and you are done.

• Start with the memory figure you just found. That number is your ceiling.
• Take a few gigabytes off it for your operating system and the apps you keep open.
• Pick a model file smaller than what is left, and expect it to want a little more than its size.
• Check your free storage will hold the file, and the next one you want to try.
• Load it, then look at where it landed. In Ollama, ps shows the size, the context length, and whether it went to the graphics chip, to system memory, or to both.
• If it split, or if swap starts climbing, step down one size. That is the whole skill.

Backed by Ollama FAQ, Ollama context length, View memory usage in Activity Monitor on Mac.