Running DeepSeek R1 distills on Your Own Hardware

Prerequisites:

You’re root. On a spare PC. Or a VM with a dedicated GPU. Running a fresh Arch Linux install (though most Linux distros will work).
Because I never trust third-party software 100%, I would not recommend running this on your main system. Unless you like unnecessary risks. In that case, go ahead, genius.

Step 1: The Setup – Because We Love Clean Slates

This is a minimal VM setup named `deepseekvm` with a 100GB NVMe SSD:

[root@deepseekvm ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2        91G   16G   75G  18% /

Step 2: Installing Ollama – The Magic Behind the Curtain

Why Ollama? Because it makes running large language models *actually* easy.
As seen on Ollama’s website, installation is a one-liner:

curl -fsSL https://ollama.com/install.sh | sh

One minute later, you should see something like:

>>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
>>> NVIDIA GPU installed.

If it doesn’t work, fix your system. That’s not my problem.

Step 3: Starting Ollama – Because It Won’t Start Itself

ollama serve

If you get errors, maybe your GPU is fake. Or you’re using an Intel HD 3000. Good luck with that.

Step 4: Running DeepSeek R1 – The Fun Part

Change to another terminal and run:

ollama run deepseek-r1:14b

Or, if your GPU is a beast (20GB VRAM or more):

ollama run deepseek-r1:32b

If everything works, you’ll see something like:

pulling manifest
pulling 6e9f90f02bb3... 100% ▕█████████████████████▏ 9.0 GB                         
pulling 369ca498f347... 100% ▕█████████████████████▏  387 B                         
pulling 6e4c38e1172f... 100% ▕█████████████████████▏ 1.1 KB                         
pulling f4d24e9138dd... 100% ▕█████████████████████▏  148 B                         
pulling 3c24b0c80794... 100% ▕█████████████████████▏  488 B                         
verifying sha256 digest
writing manifest
success

If you don’t see this, double-check your GPU drivers. Or sacrifice a chicken to the Linux gods.

Step 5: Talking to Your AI – Because Why Not?

Let’s test it:

>>> hi there

An expected response from your local NVIDIA 3090 RTX (no internet required, as this llm runs directly from YOUR HARDWARE!):

<think>
</think>
Hello! How can I assist you today? 😊

You Now Have a Local AI – No Strings Attached!