Start with prompting. It costs nothing beyond your existing API usage and takes minutes to iterate. Move to RAG when you need the model to reference specific documents or data it was not trained on. Only consider fine-tuning when you need consistent style, format, or specialized behavior that prompting cannot achieve, and you have hundreds of quality training examples.
Start here. Cheapest, fastest, and good enough for most tasks.
Best for knowledge-heavy applications. Moderate setup, excellent long-term value.
Powerful but expensive and slow. Only worth it for specialized behavior at scale.
| Criterion | Prompting | RAG | Fine-tuning |
|---|---|---|---|
| Setup Cost | Minimal. Just an API key and a text editor. | Moderate. Need embeddings, a vector database, and retrieval logic. | High. Need training data, compute time, and iteration cycles. |
| Per-Query Cost | Low. You pay for prompt tokens, which can get large with long system prompts. | Low to moderate. Embedding the query plus retrieval plus generation. | Low once trained. Shorter prompts possible since behavior is baked in. |
| Time to First Result | Minutes. Write a prompt, test it, iterate. | Hours to days. Need to index documents, build the pipeline, test retrieval. | Days to weeks. Curate data, run training jobs, evaluate quality. |
| Knowledge Freshness | Limited to training data cutoff plus whatever you paste into the prompt. | Excellent. Update your knowledge base any time without retraining. | Frozen at training time. Need to retrain to add new knowledge. |
| Behavior Customization | Moderate. System prompts can shape tone and format, but models sometimes drift. | Limited. RAG adds knowledge, not personality. Combine with prompting for style. | Strong. The model learns your style, format, and domain-specific patterns. |
| Technical Difficulty | Easy. Anyone who can write clear instructions can prompt. | Moderate. Requires understanding embeddings, chunking, and retrieval. | Hard. Need ML expertise, good training data, and careful evaluation. |
| Best For | Simple tasks, prototyping, chatbots, content generation. | Q&A over documents, customer support, internal knowledge bases. | Consistent output format, domain-specific language, reducing prompt size. |
Before comparing these three approaches, you need to answer one question: what specifically do you want the LLM to do differently? The answer determines which approach to use, and getting this wrong means burning time and money on the wrong solution.
There are three fundamentally different things you might want:
Behavior problems are best solved with prompting first, then fine-tuning if prompting is not consistent enough. Knowledge problems are solved with RAG. Most real-world applications involve both, which is why the best production systems layer these approaches together.
The mistake most developers make is jumping straight to the most complex solution. Fine-tuning sounds impressive, but if a well-written system prompt would have done the job, you just spent weeks and thousands of shillings solving a problem that did not exist.
Prompt engineering is the practice of crafting instructions, examples, and context to get the behavior you want from an LLM without modifying the model itself. It is the cheapest approach by far, and it should always be your starting point.
What prompting can do well:
The cost math: Prompting costs you only the tokens you send and receive. A typical system prompt is 500-2,000 tokens. With GPT-4o, that is roughly $0.005-0.02 per conversation. With Claude Sonnet, similar range. With smaller models like GPT-4o-mini, you are looking at fractions of a cent. There is no upfront investment, no training time, and no infrastructure to maintain.
Where prompting breaks down:
For developers in Kenya building their first AI features, prompting alone will handle most use cases. A well-crafted system prompt with 3-5 examples of desired behavior is surprisingly powerful. Do not move to more complex approaches until you have genuinely exhausted what good prompting can do.
Get weekly developer tips
Join 25,000+ developers. Practical guides, job tips, and new content — straight to your inbox.
No spam. Unsubscribe anytime.
RAG (Retrieval-Augmented Generation) adds a retrieval step before the LLM generates a response. Instead of relying on the model's training data alone, you search a knowledge base for relevant documents and inject them into the prompt. The model then generates answers grounded in your actual data.
When RAG is the right choice:
The cost breakdown for RAG:
Setting up a RAG pipeline involves several cost components. First, you need to embed your documents. Using OpenAI's text-embedding-3-small, embedding 1,000 pages of text costs about $0.02. That is a one-time cost (plus re-embedding when documents change). Second, you need vector storage. Supabase with pgvector handles this on its free tier for most early-stage applications. Pinecone's free tier works too. Third, each query incurs costs: embedding the query ($0.0001), retrieving chunks (essentially free from your database), and generating the response ($0.01-0.05 depending on the model). All in, a small RAG system serving hundreds of queries per day runs under $50/month.
Where RAG falls short:
For Kenyan developers, RAG is the sweet spot for most production AI features. Building a WhatsApp bot that answers questions about your product catalog? RAG. Internal tool that searches your company wiki? RAG. Customer support assistant that references your help docs? RAG. The setup cost is modest and the ongoing costs scale linearly with usage.
Fine-tuning takes a pre-trained LLM and trains it further on your own data. The model updates its internal weights to learn the patterns in your training examples. After fine-tuning, the model produces outputs that reflect those patterns without needing them spelled out in the prompt every time.
What fine-tuning actually costs:
Let us talk real numbers. Fine-tuning GPT-4o-mini on OpenAI costs about $3 per million training tokens. If you have 500 high-quality training examples averaging 1,000 tokens each (input + output), that is 500,000 tokens, costing roughly $1.50 per training run. Sounds cheap, right? But you will run 5-15 training iterations to get the quality right, and you need someone with ML experience to curate the training data, design the evaluation criteria, and debug quality issues. The real cost is the human time, not the compute.
Fine-tuning larger models is significantly more expensive. Fine-tuning GPT-4o costs roughly 10x more per token than GPT-4o-mini. Fine-tuning open-source models (Llama 3, Mistral) can be cheaper on compute but requires your own GPU infrastructure or renting cloud GPUs [TODO: verify current pricing tiers for major providers].
When fine-tuning is genuinely worth it:
When fine-tuning is a waste of money:
For most startups in Kenya and across Africa, fine-tuning is not where you should be spending your limited budget. Get your prompting right. Build RAG if you need knowledge. Come back to fine-tuning only when you have a validated product at enough scale that the optimization makes financial sense.
Here is a straightforward way to decide which approach to use. Start at the top and work your way down.
Step 1: Try prompting.
Write a clear system prompt. Include 3-5 examples of desired behavior (few-shot prompting). Test it against 20 real-world inputs. If the results are good enough for your use case, stop here. You are done. Most developers never need to go further.
Step 2: Does the model need knowledge it does not have?
If yes, build a RAG pipeline. Embed your documents, store them in a vector database (pgvector on Supabase is the simplest path), and retrieve relevant chunks at query time. Keep your good system prompt from Step 1. RAG adds knowledge; the prompt shapes behavior.
Step 3: Is the model inconsistent despite good prompting and RAG?
If the model keeps drifting from your desired format, tone, or behavior despite clear instructions, and this inconsistency is actually hurting your product (not just mildly annoying), consider fine-tuning. But only if you have at least 200-500 high-quality training examples that demonstrate the behavior you want.
A real example from the Kenyan market:
Imagine you are building a WhatsApp bot for a M-Pesa lending service. The bot needs to answer customer questions about loan terms, check eligibility, and explain repayment schedules.
The progression is always: prompting first, RAG second, fine-tuning last. Each step adds cost and complexity. Only move to the next step when the current one genuinely is not sufficient.
The best production AI systems do not choose one approach. They layer all three. Understanding how they fit together is what separates prototypes from real products.
The layered architecture:
This architecture is how companies like Notion, Intercom, and Shopify build their AI features. But here is the thing: most of the value comes from layers 2 and 3 (RAG and prompting). Layer 1 (fine-tuning) is an optimization, not a requirement.
A practical path for African startups:
Month 1: Ship with prompting only. Get real user feedback. Understand what your users actually ask and where the model fails.
Month 2-3: Add RAG for the knowledge gaps. Index your documentation, FAQs, and product data. This usually solves 80% of the remaining quality issues.
Month 6+: If you have thousands of daily queries and consistent quality issues that prompting cannot fix, evaluate fine-tuning. By this point you will have enough real conversation logs to build a quality training dataset.
This path lets you ship fast, learn from real usage, and invest in complexity only when the data justifies it. Do not build a three-layer architecture on day one. You will spend months engineering solutions for problems your users may never have.
Let us put actual numbers on a hypothetical scenario. You are building a customer support bot for a Nairobi-based e-commerce company. The bot handles 500 conversations per day and needs to answer questions about products, shipping, and returns.
Option A: Prompting Only
Option B: Prompting + RAG
Option C: Prompting + RAG + Fine-tuning
Notice that the monthly cost difference between Option B and Option C is only about KES 1,950. The fine-tuning itself is cheap. The expensive part is the weeks of human time spent curating training data and iterating on model quality. For a startup doing 500 conversations per day, that time is almost certainly better spent improving the product, fixing bugs, or talking to customers.
Fine-tuning becomes cost-effective at much higher volumes (think 10,000+ conversations per day) where the per-query savings compound into meaningful numbers, and where the consistency improvements directly impact customer satisfaction metrics.
KES 120,000
The complete Full-Stack Software & AI Engineering course. From zero code to AI-powered products in 16 weeks. 16 modules, 208 lessons, with career support until placement.
Enrol in Full-Stack AI