Browse Tag

local LLM

2 Articles

Running a 744-Billion Parameter LLM in a Home Lab

By Sutopo

10 Min Read

Learn how quantization, CPU offloading, and distributed inference make running a 744-Billion Parameter LLM on consumer hardware a practical engineering project.

How to Set Up and Benefit From a Local LLM at Home

By Sutopo

9 Min Read

Learn how to set up and benefit from a locally hosted large language model at home, with hardware tips, tool comparisons, and a step-by-step install guide.