
Start With VRAM, Then Look at Bandwidth
Running a language model on your own machine is first a capacity question. The weights have to fit in graphics memory, and if they do not, layers spill across the PCIe bus into system RAM and generation speed collapses from tens of tokens per second to low single digits. Everything else about a card matters only once that threshold is met.
After a model fits, memory bandwidth decides how fast it runs, because producing each token reads the entire set of weights once. That is why two cards holding identical capacity can differ by roughly a factor of two in tokens per second, and why the quickest gaming card is not automatically the right one for this job.













































