I Put an AI on a Thumb Drive

Plug it in, double-click, and you're chatting with an 8-billion-parameter AI model on Mac, Windows, or Linux. No internet. No account. No subscription. Nothing you type ever leaves the machine.

I call it Pocket LLM, and it took about an hour to put together, test, and mess around with.

What it actually is

Three pieces, all on one thumb drive:

The runtime: Mozilla's open-source llamafile, a single program that runs the model and serves a chat page in your browser.
The model: Dolphin 2.9.4, built on Meta's Llama 3.1 8B, compressed to about 4.9 GB so it fits on a drive and runs on an everyday laptop.
The launchers: my one-click start and stop scripts for Mac, Windows, and Linux. They wait for the model to load, open the chat, and shut everything down cleanly.

The Pocket LLM chat screen in Chrome, running at localhost:8081 — Note the address bar: localhost. Everything runs on the machine in front of you.

The specs are humble

A 6-core Intel laptop, 16 GB of RAM, no GPU. It's slow: about five tokens a second, which works out to a few words at a time. I've had faster conversations with a fax machine. But it works, even in airplane mode.

Naturally, the first thing I asked it was to explain my own job.

Prompt: explain Revenue Operations like a pirate captain. The model answers: Arr matey! Revenue Operations be the treasure map leadin' us to more loot. — 77 tokens in 15 seconds. Not wrong.

"Arr matey! Revenue Operations be the treasure map leadin' us to more loot... It's all 'bout gettin' yer crew on the same ship." I've been saying that for 15 years, just with fewer "arr"s.

It also gets things wrong

Asked about itself, it confidently told me Hugging Face built Llama. Meta did. It got the architecture wrong too, with total confidence.

The model confidently giving an incorrect answer about who built Llama — Small offline models trade some accuracy for privacy and portability.

That's the trade-off with small local models. You get privacy and portability, and you give up some accuracy. Knowing where that line sits, and which work belongs on each side of it, is most of what good AI strategy is.

Why I bothered

I've spent my career in revenue operations, network security, and finance-adjacent work, where the data is the crown jewels: pipeline, pricing, comp plans, strategy, board numbers.

Every company wants the upside of AI without handing that data to a third party. I wanted to see what fully local looks like today, so I know what's coming as these models get faster and smaller. The honest answer: it's already good enough for drafts, notes, and brainstorming on sensitive material, and not yet good enough to trust on facts without checking.

Try it yourself

It's open source, so grab it and break it yourself. The project page has more screenshots and how it works, and the code and setup guide are on GitHub.

If your company had to keep AI fully inside its walls, what's the first thing you'd use it for? Tell me on LinkedIn.