Running AI models locally has moved from experiment to everyday practice. The right card depends less on brand names and more on three practical questions: how big are the models you run, how patient are you during generation, and how often do you train or fine-tune.
Start with memory, not marketing
Model size drives everything. Small language models and image generators run comfortably on 8–12 GB cards. Mid-size models want 16–24 GB. Fine-tuning or running larger models locally pushes you toward 24 GB and beyond, or a multi-card setup.
- 8–12 GB: entry inference, image generation, small LLMs
- 16–24 GB: mid-size models, comfortable experimentation
- 24 GB+: larger models, fine-tuning, multi-GPU work
Match the card to the workflow
Inference rewards memory capacity and bandwidth. Fine-tuning rewards raw compute and stability under sustained load. Be honest about which one you do most, and choose accordingly.
