What Is RAG and Why Every Kenyan Startup Is Suddenly Asking for It

Imagine you are building a customer support chatbot for a Kenyan e-commerce startup. You connect it to ChatGPT or Claude, and a customer asks: "What is your return policy?" The AI confidently responds with a generic return policy that has nothing to do with your actual business. It might even invent details. This is the fundamental problem.

Large language models like GPT-4 and Claude are trained on vast amounts of internet data. They know a lot about the world in general. But they know absolutely nothing about your specific company, your products, your policies, your internal documentation, or your customer data. They cannot. That information was never in their training set.

You have three options for fixing this:

• Fine-tuning: Retrain the model on your data. Expensive, slow, requires ML expertise, and you need to retrain every time your data changes.
• Stuffing the prompt: Paste all your documentation into the system prompt. Works for tiny amounts of data, but context windows have limits and costs increase with every token.
• RAG: Search your data at query time, retrieve only the relevant parts, and inject them into the prompt. Fast, cheap, easy to update, and scales to large knowledge bases.

RAG is the right answer for almost every "I need the AI to know about my stuff" use case. It is not new (the technique was described in a 2020 research paper from Meta), but it has become the standard approach in 2025-2026 as businesses have moved from playing with AI demos to building real products.

The reason every Kenyan startup is suddenly asking for RAG is simple. They have tried building chatbots with generic AI, discovered the AI does not know anything useful about their business, and realized they need a way to give it that knowledge. RAG is that way.

RAG sounds technical, but the concept is something you already understand. Think about how you would answer a question about a topic you do not know well.

You would not try to answer from memory. You would open Google, search for the relevant information, read the top results, and then form your answer based on what you found. RAG is exactly this process, automated for an AI model.

Here is what happens when a user asks your RAG-powered chatbot a question:

Step 1: The user asks a question. "What are your delivery times to Mombasa?"

Step 2: The system searches your knowledge base. It looks through your stored documents (FAQs, product pages, policy documents, help articles) and finds the chunks that are most relevant to the question. In this case, it finds a section from your shipping policy that mentions delivery times by city.

Step 3: The relevant chunks are added to the prompt. The system builds a prompt like: "Here is the relevant information from our documentation: [shipping policy excerpt]. Based on this information, answer the customer's question: What are your delivery times to Mombasa?"

Step 4: The AI generates an answer grounded in your data. "Delivery to Mombasa typically takes 3-5 business days. For express delivery, you can expect 1-2 business days at an additional fee." This answer is accurate because it comes from your actual documentation, not from the AI making things up.

That is the entire concept. Search, retrieve, generate. The "retrieval" happens before the "generation," which is why it is called Retrieval-Augmented Generation. You are augmenting what the AI can do by giving it relevant information at the moment it needs it.

The technical magic is in step 2, specifically, how the system knows which documents are "relevant" to a given question. That is where embeddings and vector search come in, which we cover in the next section.

The hardest part of RAG is not the AI generation. It is the search. How do you find the right pieces of information when a customer asks a question that does not match your documentation word for word?

Traditional keyword search (like searching your email) looks for exact word matches. If your shipping policy says "delivery timeline" and the customer asks about "how long shipping takes," keyword search might miss the connection. RAG uses something better: semantic search powered by embeddings.

What is an embedding? An embedding is a list of numbers that represents the meaning of a piece of text. You send a sentence to an embedding model (like OpenAI's text-embedding-3-small), and it returns a vector, a list of 1,536 numbers that capture what that sentence means. The key property: sentences with similar meanings get similar vectors, even if they use completely different words.

"What are your delivery times?" and "How long does shipping take?" have very different words but nearly identical embeddings. That is what makes semantic search powerful for RAG.

How vector search works:

1. When you set up your knowledge base, you split your documents into chunks (paragraphs or sections) and generate an embedding for each chunk.
2. You store these embeddings in a vector database (Supabase with pgvector works great for this).
3. When a user asks a question, you generate an embedding for their question.
4. You search the vector database for chunks whose embeddings are closest to the question embedding.
5. The closest chunks are the most relevant to the question. You retrieve them and pass them to the AI.

"Closest" here means mathematical similarity, typically measured by cosine similarity. Two vectors pointing in roughly the same direction in high-dimensional space have high cosine similarity, which means their texts have similar meanings.

You do not need to understand the math. What you need to understand is that this system lets you find relevant information based on meaning, not just keywords. That is what makes RAG answers accurate even when users phrase questions in unexpected ways.

RAG is not just a Silicon Valley toy. It solves real problems for Kenyan businesses right now. Here are the use cases we see most often.

Customer Support Bots That Actually Know Your Products

This is the most common use case by far. You have a product catalog, FAQ pages, help documentation, and shipping policies. Instead of hiring more support agents or building a rigid decision-tree chatbot, you build a RAG system that searches your docs and answers customer questions accurately. It works 24/7, handles questions in both English and Swahili (if your docs are in both languages), and escalates to human agents only when it cannot find an answer. For e-commerce companies, SaaS products, and fintechs, this is the first AI feature that pays for itself.

Internal Knowledge Bases

Every growing company has this problem: tribal knowledge trapped in Slack messages, Google Docs, Notion pages, and people's heads. A RAG-powered internal tool lets team members ask natural-language questions and get answers pulled from across all your internal documentation. "What is our process for onboarding a new vendor?" "What were the key decisions from last quarter's board meeting?" Instead of searching through twenty documents, the team asks the AI, and it finds the answer.

Document Q&A

Law firms, consulting companies, and government agencies deal with large volumes of documents. RAG lets you build a system where users upload a contract, policy document, or report, and then ask questions about it. "What are the termination clauses in this contract?" "What does this regulation say about data protection?" The AI reads the document and answers based on its contents, with the ability to cite the specific section.

Product Recommendation and Discovery

For e-commerce platforms and marketplaces, RAG can power a conversational product discovery experience. Instead of browsing categories and applying filters, customers describe what they need: "I need a laptop under KES 80,000 for programming." The RAG system searches your product catalog and recommends specific products with explanations of why they match.

M-Pesa and Financial Services FAQ

Fintechs and M-Pesa-integrated businesses get the same questions over and over: "How do I reverse a transaction?" "Why was my payment declined?" "What are your transaction limits?" A RAG chatbot trained on your transaction troubleshooting docs handles these questions instantly, freeing up your support team for complex issues that require human judgment.

Let us get concrete. Here is what you need to build a basic RAG system, and it is less than you might think.

What you need:

• A set of documents (your knowledge base)
• An embedding model (OpenAI's text-embedding-3-small is the easiest starting point)
• A vector database (Supabase with pgvector if you are already using Supabase, or Pinecone for a standalone option)
• An AI model for generation (GPT-4o-mini, Claude Haiku, or their larger siblings)

Phase 1: Prepare your knowledge base.

Gather your documents. This might be your website content, help articles, product descriptions, policy documents, or internal wikis. Split each document into chunks of about 200-500 words. Each chunk should be self-contained, meaning someone should be able to read it and understand the point without needing the surrounding text. Splitting on paragraph or section boundaries usually works better than splitting at arbitrary character counts.

Phase 2: Embed and store.

Send each chunk to the embedding model and store the resulting vector alongside the original text in your vector database. With Supabase and pgvector, this looks like:

// Embed a chunk and store it
const embedding = await openai.embeddings.create({
  model: 'text-embedding-3-small',
  input: chunkText,
});

await supabase.from('knowledge_base').insert({
  content: chunkText,
  embedding: embedding.data[0].embedding,
  source: 'shipping-policy.md',
  section: 'Delivery Times',
});

Phase 3: Build the query pipeline.

When a user asks a question, embed their question, search for the most relevant chunks, and build the prompt:

async function answerQuestion(question: string): Promise<string> {
  // 1. Embed the question
  const questionEmbedding = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: question,
  });

  // 2. Find relevant chunks
  const { data: chunks } = await supabase.rpc('match_knowledge_base', {
    query_embedding: questionEmbedding.data[0].embedding,
    match_threshold: 0.7,
    match_count: 5,
  });

  // 3. Build the prompt with context
  const context = chunks.map((c: any) => c.content).join('\n\n');

  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      {
        role: 'system',
        content: `You are a helpful customer support assistant. Answer the question
based ONLY on the following context. If the context does not contain enough
information to answer, say so honestly. Do not make up information.

Context:
${context}`,
      },
      { role: 'user', content: question },
    ],
  });

  return response.choices[0]?.message?.content || 'I could not find an answer.';
}

That is a complete, working RAG pipeline. It is not production-ready (you would want error handling, caching, and evaluation), but it demonstrates the core pattern in about 40 lines of code. The system prompt instruction to answer "ONLY based on the context" is crucial. Without it, the AI will happily mix in information from its training data, which defeats the entire purpose of RAG.

RAG is not the only way to give AI access to custom knowledge. Here is when each approach makes sense, and why RAG is usually the right starting point.

Prompt Stuffing (Putting data directly in the prompt)

Best for: Very small amounts of data. If your entire knowledge base fits in a few thousand words, just paste it into the system prompt. No need for embeddings, vector databases, or any infrastructure. A restaurant chatbot with a menu, hours, and location can handle everything in the system prompt. The limit is practical: once your data exceeds a few pages, prompt stuffing gets expensive (you pay for all those input tokens on every request) and the AI starts struggling to find relevant information in a wall of text.

RAG (Retrieval-Augmented Generation)

Best for: Most real-world use cases. RAG shines when your knowledge base is too large for the prompt but does not require changes to the model's fundamental behavior. Customer support docs, product catalogs, internal wikis, policy documents. RAG is also the best choice when your data changes frequently, because you just update the knowledge base without touching the model. The main limitation is that RAG cannot change how the model writes or reasons. It only changes what information the model has access to.

Fine-Tuning (Retraining the model on your data)

Best for: Changing the model's behavior, tone, or output format. If you want the AI to write in your brand voice, follow a specific response format, or handle domain-specific reasoning patterns, fine-tuning is the tool. The cost is significantly higher (both money and time), you need hundreds to thousands of training examples, and you have to retrain every time the model needs to learn new behavior. For most Kenyan startups, fine-tuning is overkill until you have validated the product and need to optimize.

The practical recommendation: Start with prompt stuffing if your data is small. Move to RAG when it outgrows the prompt. Consider fine-tuning only when RAG is working but the model's output quality or style is not meeting your standards. In practice, RAG alone handles the vast majority of business use cases. You can always add fine-tuning later.

If you want a deeper technical comparison, check our detailed comparison guide.

If you want to build your first RAG system this weekend, here is a concrete plan.

Saturday morning: Set up the infrastructure.

Create a Supabase project (free tier is fine). Enable the pgvector extension. Create a table for your knowledge base with a vector column. Get an OpenAI API key if you do not have one. Total setup time: about 30 minutes.

Saturday afternoon: Index your documents.

Pick a small, focused knowledge base to start. Your company's FAQ page, a product catalog with 20-50 items, or a help documentation section. Split the content into chunks, embed them, and store them in Supabase. Write a simple script to do this. You will probably index 50-200 chunks. Total time: 2-3 hours.

Sunday morning: Build the query pipeline.

Write the search and generation code. Test it with 10-15 real questions that customers would actually ask. Tweak the system prompt based on the answers you get. The first version will not be perfect, and that is fine. Total time: 2-3 hours.

Sunday afternoon: Wire it into your app.

Build a simple chat UI (or use the one from our chatbot tutorial). Connect it to your RAG backend. Test with a few friends or teammates. Gather feedback. Total time: 2-3 hours.

By Sunday evening, you have a working RAG-powered chatbot that answers questions about your business using your actual data. It will not be production-ready (you need error handling, rate limiting, and more testing), but you will have a clear understanding of the technology and a prototype you can iterate on.

The most important takeaway: RAG is not a research project. It is an engineering task. If you can call an API, query a database, and build a basic web interface, you can build a RAG system. The barrier is not technical complexity. It is just sitting down and doing it.