How I Doubled My Local LLM’s Speed (Without Buying New Hardware)

I remember the frustration vividly. I had just downloaded a powerful 70B parameter model, quantized it nicely to Q4_K_M, and fired up my… Continue reading on Write A Catalyst »