#LLM
Here are my latest articles filed under LLM. Showing page 1 of 1 (9 total articles).
Do you really need a large language model? When a smaller solution wins
I often get asked if an LLM is mandatory. The answer depends on the task, data, and constraints—here’s how to decide.
Why 8-bit quantization can halve LLM size and how outlier weights spoil it
8-bit quantization can shrink a language model dramatically, but a single outlier weight can break everything. Here’s how to do it right.
Why Large Language Models Forget Facts and How to Keep Them Honest
LLMs often hallucinate. I explain why and give concrete ways to keep them honest.
Building a Private Vector Database for On-Device Retrieval Augmented Generation
A step‑by‑step guide to creating a private vector store and wiring it into a local RAG pipeline that never leaves your hardware.
Why LLMs Run Out of VRAM: KV Cache Fragmentation and the PagedAttention Fix
LLMs often crash because their key‑value cache fragments. I’ll show why that happens and how PagedAttention solves the problem.
Why VRAM bandwidth determines token speed in local LLMs
VRAM bandwidth, not just size, caps how fast a local LLM can generate text. I explain the bottleneck and how to work around it.
Understanding Prompt Injection and How to Guard Your AI Applications
A clear guide to prompt injection, its risks, and simple defenses for developers building AI‑powered applications.
Understanding Chain‑of‑Thought Prompting and How to Keep LLM Reasoning Faithful
A friendly guide to chain‑of‑thought prompting and how to make LLM reasoning stay on track.
Why Better Vector Search Indexes Matter More Than More Training for LLMs
A solid vector search index can make your LLM feel smarter without any extra model training.
Need help building a website?
Send me a message and let's talk.