Ship Your First AI Feature: Adding a Chatbot to a Real App

This tutorial walks you through adding a real AI chatbot feature to a web application. Not a demo. Not a prototype. A feature you can actually ship to users.

By the end, you will have:

• A backend API endpoint that proxies requests to an AI provider
• A chat UI component with a message list, input field, and send button
• Streaming responses that appear in real time (the typewriter effect)
• Error handling for API failures, rate limits, and network issues
• Basic cost controls to prevent your API bill from exploding

The code examples use TypeScript, React for the frontend, and a generic backend (the patterns work with Express, Next.js API routes, Supabase Edge Functions, or any other backend). If you are using a different frontend framework, the API integration patterns are the same.

This is a tutorial, not a concepts article. We are writing code, explaining decisions as we go, and building something that works. If you want the theory behind LLMs and chatbots, read our guides on context engineering and RAG first.

You have two serious options in 2026: OpenAI (GPT-4o, GPT-4o-mini) and Anthropic (Claude Sonnet, Claude Haiku). Both are excellent. Here is how to decide.

OpenAI (GPT-4o / GPT-4o-mini)

The larger ecosystem. More tutorials, more community examples, and broader framework support. GPT-4o-mini is the cost king for simple chatbot use cases, roughly $0.15 per million input tokens and $0.60 per million output tokens. If cost is your primary concern and the chatbot handles straightforward questions, GPT-4o-mini is hard to beat. The API is mature and stable.

Anthropic (Claude Sonnet / Claude Haiku)

Stronger at following nuanced instructions and maintaining consistent behavior in longer conversations. Claude Haiku is competitive on cost for simpler tasks. Claude Sonnet is the better choice if your chatbot needs to handle complex, multi-step interactions or follow detailed system prompts precisely. The API is clean and well-documented.

For Kenyan developers specifically: both APIs bill in USD, and costs add up. A chatbot handling 1,000 conversations per day with GPT-4o-mini might cost $5-15/month. The same volume with GPT-4o or Claude Sonnet could cost $50-150/month. [TODO: verify these estimates with current pricing] Start with the cheaper model and upgrade only if the quality is not good enough for your use case.

For this tutorial, we will show both OpenAI and Anthropic examples. The patterns are nearly identical. Pick whichever you prefer, and know that switching later is straightforward.

Getting your API key:

• OpenAI: Sign up at platform.openai.com, add billing, generate an API key under API Keys
• Anthropic: Sign up at console.anthropic.com, add billing, generate an API key under API Keys

Store the key in your environment variables. Never commit it to your repository.

Your AI API key is a secret. If you call the API directly from the browser, anyone can open DevTools, grab your key, and run up your bill. So the first thing you build is a backend endpoint that sits between your frontend and the AI provider.

Here is a minimal Express endpoint that proxies chat requests to OpenAI:

// server/routes/chat.ts
import { Router } from 'express';
import OpenAI from 'openai';

const router = Router();
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

router.post('/api/chat', async (req, res) => {
  try {
    const { messages } = req.body;

    if (!messages || !Array.isArray(messages) || messages.length === 0) {
      return res.status(400).json({ error: 'Messages array is required' });
    }

    // Limit conversation length to control costs
    const recentMessages = messages.slice(-20);

    const completion = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [
        {
          role: 'system',
          content: 'You are a helpful assistant for our application. Be concise and friendly.',
        },
        ...recentMessages,
      ],
      max_tokens: 500,
    });

    const reply = completion.choices[0]?.message?.content || 'Sorry, I could not generate a response.';
    res.json({ reply });
  } catch (error: any) {
    console.error('Chat API error:', error.message);

    if (error.status === 429) {
      return res.status(429).json({ error: 'Too many requests. Please wait a moment.' });
    }

    res.status(500).json({ error: 'Something went wrong. Please try again.' });
  }
});

export default router;

The equivalent using Anthropic's SDK:

// server/routes/chat.ts (Anthropic version)
import { Router } from 'express';
import Anthropic from '@anthropic-ai/sdk';

const router = Router();
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

router.post('/api/chat', async (req, res) => {
  try {
    const { messages } = req.body;

    if (!messages || !Array.isArray(messages) || messages.length === 0) {
      return res.status(400).json({ error: 'Messages array is required' });
    }

    const recentMessages = messages.slice(-20);

    const response = await anthropic.messages.create({
      model: 'claude-sonnet-4-20250514',
      max_tokens: 500,
      system: 'You are a helpful assistant for our application. Be concise and friendly.',
      messages: recentMessages,
    });

    const reply = response.content[0]?.type === 'text'
      ? response.content[0].text
      : 'Sorry, I could not generate a response.';

    res.json({ reply });
  } catch (error: any) {
    console.error('Chat API error:', error.message);
    res.status(500).json({ error: 'Something went wrong. Please try again.' });
  }
});

export default router;

Notice a few things. We are limiting conversation history to the last 20 messages (messages.slice(-20)) to keep token usage predictable. We set max_tokens: 500 to cap response length. And we handle errors gracefully instead of letting them crash the server. These are not optional. They are the baseline for responsible AI feature development.

The chat interface needs three things: a list of messages, an input field, and a send button. Let us build it.

// components/ChatWidget.tsx
import { useState, useRef, useEffect } from 'react';

interface Message {
  role: 'user' | 'assistant';
  content: string;
}

export function ChatWidget() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState('');
  const [isLoading, setIsLoading] = useState(false);
  const messagesEndRef = useRef<HTMLDivElement>(null);

  // Auto-scroll to the latest message
  useEffect(() => {
    messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
  }, [messages]);

  async function sendMessage() {
    const trimmed = input.trim();
    if (!trimmed || isLoading) return;

    const userMessage: Message = { role: 'user', content: trimmed };
    const updatedMessages = [...messages, userMessage];
    setMessages(updatedMessages);
    setInput('');
    setIsLoading(true);

    try {
      const res = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ messages: updatedMessages }),
      });

      if (!res.ok) {
        throw new Error('Failed to get response');
      }

      const data = await res.json();
      setMessages(prev => [...prev, { role: 'assistant', content: data.reply }]);
    } catch {
      setMessages(prev => [
        ...prev,
        { role: 'assistant', content: 'Sorry, something went wrong. Please try again.' },
      ]);
    } finally {
      setIsLoading(false);
    }
  }

  return (
    <div className="flex flex-col h-96 border rounded-xl bg-white">
      <div className="flex-1 overflow-y-auto p-4 space-y-3">
        {messages.map((msg, i) => (
          <div key={i} className={msg.role === 'user' ? 'text-right' : 'text-left'}>
            <span className={
              msg.role === 'user'
                ? 'inline-block bg-blue-600 text-white rounded-lg px-3 py-2'
                : 'inline-block bg-gray-100 text-gray-800 rounded-lg px-3 py-2'
            }>
              {msg.content}
            </span>
          </div>
        ))}
        {isLoading && (
          <div className="text-left text-gray-400">Thinking...</div>
        )}
        <div ref={messagesEndRef} />
      </div>
      <div className="border-t p-3 flex gap-2">
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          onKeyDown={e => e.key === 'Enter' && sendMessage()}
          placeholder="Type a message..."
          className="flex-1 border rounded-lg px-3 py-2"
        />
        <button
          onClick={sendMessage}
          disabled={isLoading || !input.trim()}
          className="bg-blue-600 text-white px-4 py-2 rounded-lg disabled:opacity-50"
        >
          Send
        </button>
      </div>
    </div>
  );
}

This is a complete, working chat component. It handles the loading state, auto-scrolls to new messages, prevents double-sending, and shows a graceful error message if the API fails. You can drop it into any page in your app.

A few UX details that matter. The "Thinking..." indicator tells users the AI is processing. Without it, the gap between sending a message and receiving a response feels broken. The Enter key sends the message, which matches every chat app users have ever used. And the send button disables while loading, preventing users from hammering it and creating duplicate requests.

This version waits for the full response before displaying it. The next section upgrades it to streaming, which makes the experience feel much faster.

Without streaming, users stare at "Thinking..." for 2-5 seconds while the full response generates. With streaming, they see text appearing word by word in real time. The total time is the same, but the perceived speed is dramatically better. Streaming is what makes AI chatbots feel responsive.

First, update the backend to stream:

// server/routes/chat-stream.ts (OpenAI streaming)
router.post('/api/chat/stream', async (req, res) => {
  const { messages } = req.body;

  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  try {
    const stream = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [
        { role: 'system', content: 'You are a helpful assistant. Be concise and friendly.' },
        ...messages.slice(-20),
      ],
      max_tokens: 500,
      stream: true,
    });

    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content;
      if (content) {
        res.write(`data: ${JSON.stringify({ content })}

`);
      }
    }

    res.write('data: [DONE]

');
    res.end();
  } catch (error: any) {
    res.write(`data: ${JSON.stringify({ error: 'Stream failed' })}

`);
    res.end();
  }
});

Then update the frontend to consume the stream:

// Updated sendMessage function with streaming
async function sendMessage() {
  const trimmed = input.trim();
  if (!trimmed || isLoading) return;

  const userMessage: Message = { role: 'user', content: trimmed };
  const updatedMessages = [...messages, userMessage];
  setMessages(updatedMessages);
  setInput('');
  setIsLoading(true);

  // Add an empty assistant message that we will fill incrementally
  setMessages(prev => [...prev, { role: 'assistant', content: '' }]);

  try {
    const res = await fetch('/api/chat/stream', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ messages: updatedMessages }),
    });

    const reader = res.body?.getReader();
    const decoder = new TextDecoder();

    if (!reader) throw new Error('No reader available');

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const text = decoder.decode(value);
      const lines = text.split('
').filter(line => line.startsWith('data: '));

      for (const line of lines) {
        const data = line.slice(6); // remove 'data: '
        if (data === '[DONE]') break;

        try {
          const parsed = JSON.parse(data);
          if (parsed.content) {
            setMessages(prev => {
              const updated = [...prev];
              const lastMsg = updated[updated.length - 1];
              if (lastMsg.role === 'assistant') {
                lastMsg.content += parsed.content;
              }
              return updated;
            });
          }
        } catch {
          // Skip malformed chunks
        }
      }
    }
  } catch {
    setMessages(prev => {
      const updated = [...prev];
      const lastMsg = updated[updated.length - 1];
      if (lastMsg.role === 'assistant' && !lastMsg.content) {
        lastMsg.content = 'Sorry, something went wrong. Please try again.';
      }
      return updated;
    });
  } finally {
    setIsLoading(false);
  }
}

The key idea: we add an empty assistant message to the list immediately, then append each streamed chunk to it as it arrives. The user sees the response building up character by character. This is the same pattern that ChatGPT, Claude.ai, and every major AI chat product uses.

One gotcha: Server-Sent Events (SSE) can be finicky with certain proxy configurations and hosting platforms. If you are deploying behind a reverse proxy like Nginx, make sure it is configured to pass through SSE streams without buffering. On Vercel, streaming works out of the box with Edge Functions.

This is where most tutorials stop. But if you skip this section, you will either ship a fragile feature that breaks under real usage or wake up to a surprise API bill. Neither is fun.

Error Handling

AI APIs fail in specific, predictable ways. Handle each one:

• Rate limiting (HTTP 429): The API is throttling you. Show the user a "please wait" message and implement exponential backoff on retries.
• Context length exceeded (HTTP 400): The conversation is too long. Trim older messages and retry, or start a new conversation.
• API key invalid (HTTP 401): Your key is wrong or expired. Log the error (do not expose it to the user) and show a generic error message.
• Server error (HTTP 500/503): The AI provider is having issues. Retry once after a short delay. If it fails again, show an error and let the user try later.
• Network timeout: Set a reasonable timeout (30 seconds for non-streaming, 60 seconds for streaming) and show a clear message when it is hit.

Cost Controls

AI API costs can escalate quickly if you do not set boundaries. Here is a practical approach:

// Simple per-user rate limiting
const userRequestCounts = new Map<string, { count: number; resetAt: number }>();

function checkRateLimit(userId: string, maxPerHour: number = 30): boolean {
  const now = Date.now();
  const record = userRequestCounts.get(userId);

  if (!record || now > record.resetAt) {
    userRequestCounts.set(userId, { count: 1, resetAt: now + 3600000 });
    return true;
  }

  if (record.count >= maxPerHour) {
    return false;
  }

  record.count++;
  return true;
}

Beyond rate limiting, implement these cost controls from day one:

• Set max_tokens on every request. Cap response length to prevent unexpectedly long (and expensive) replies. 500 tokens is reasonable for most chatbot responses.
• Limit conversation length. Only send the last 10-20 messages to the API. Longer conversations cost more per request because of input tokens.
• Set spending alerts. Both OpenAI and Anthropic let you set monthly spending limits and email alerts. Set them lower than you think you need.
• Use the cheapest model that works. GPT-4o-mini and Claude Haiku handle most chatbot use cases well. Only upgrade to larger models if the cheaper one is not good enough for your specific use case.
• Monitor usage daily for the first few weeks after launch. Check the provider's dashboard to understand your actual cost per conversation.

For a Kenyan startup, an unexpected $500 API bill hits differently than it does for a Silicon Valley company. Set your limits conservatively, monitor closely, and scale up only when you have the revenue to support it.

A chatbot that says "I am a helpful assistant" is a tech demo. A chatbot that knows your product and helps your users is a feature. The difference is entirely in the system prompt and the context you provide.

Write a specific system prompt. Replace the generic "helpful assistant" prompt with something tailored to your app:

const systemPrompt = `You are the support assistant for ShopEasy, an e-commerce
platform popular in Kenya. You help customers with:
- Tracking their orders
- Understanding our return policy (30 days, item must be unused)
- Finding products on the site
- M-Pesa payment questions

Our delivery times:
- Nairobi: 1-2 business days
- Other major cities: 3-5 business days
- Rural areas: 5-10 business days

If the customer has an issue you cannot resolve (refunds, account problems,
damaged items), ask them to email support@shopeasy.co.ke with their order number.

Be friendly and concise. Respond in the same language the customer uses.
If you are not sure about something, say so honestly.`;

This system prompt turns a generic chatbot into a genuinely useful support tool. It knows the business context, has specific policies to reference, and has clear boundaries for what it can and cannot handle.

Add context from your database. For the next level, inject user-specific data into the system prompt. If the user is logged in, include their recent orders, their account status, or their subscription tier. The chatbot becomes dramatically more useful when it knows who it is talking to.

Consider RAG for product knowledge. If your chatbot needs to answer questions about a large product catalog or extensive documentation, simple system prompts are not enough. This is where Retrieval-Augmented Generation comes in. Check our guide to RAG for the full picture.

Start simple. Ship the basic chatbot with a good system prompt. Gather real user questions for a week. Then improve the system prompt based on what users actually ask, not what you assume they will ask. The real-world feedback loop is where the chatbot becomes genuinely valuable.