I study how to make large language model inference faster, more memory-efficient, and more predictable on constrained GPU systems.https://bilalrana.com/https://bilalrana.com/blog/does-compression-make-llm-inference-faster/https://bilalrana.com/blog/does-compression-make-llm-inference-faster/Saved bytes became speed only when the kernel and batch regime cooperated. We tested FP16, INT8, NF4, and 2:4 sparsity on real LLM decode shapes.Wed, 02 Sep 2026 00:00:00 GMTllm-inferencegpuquantizationcudabenchmarkshttps://bilalrana.com/blog/lower-power-higher-energy/https://bilalrana.com/blog/lower-power-higher-energy/NF4 drew less board power but used more gross energy per token. Subtracting the GPU idle floor reversed the comparison, while batching produced the largest win.Tue, 01 Sep 2026 00:00:00 GMTllm-inferencegpuquantizationbenchmarkshttps://bilalrana.com/blog/how-much-kv-cache-can-you-prune/https://bilalrana.com/blog/how-much-kv-cache-can-you-prune/A recent-window cache removed one-third of the context with almost no perplexity change, while bounded caches kept long-context memory and decode cost flat.Thu, 03 Sep 2026 00:00:00 GMTllm-inferencegpukv-cachebenchmarkshttps://bilalrana.com/blog/what-breaks-first-under-load/https://bilalrana.com/blog/what-breaks-first-under-load/First-token latency failed before decode latency, and fixed bursts showed why average request rate is not enough for capacity planning.Fri, 04 Sep 2026 00:00:00 GMTllm-inferencevllmservingbenchmarks