What tech stack do I need for an AI feature?
Less than most diagrams show. For one AI feature you need a model API called from your own server, the database you already have, a queue for slow jobs, caching, and a log of every prompt, answer and cost. Add a vector database or an agent framework only when a real problem asks for it.
OpenAI’s Batch API gives a 50% cost discount compared with its synchronous APIs, and each batch completes within 24 hours.
The layers of an AI tech stack
- Model API: OpenAI, Anthropic or Google. Call it from your server or an edge function, never from the browser, so the key stays secret and you can set limits per user.
- Your data: if the feature answers from your own content, store embeddings next to the rest of your data. Postgres with pgvector does this. Supabase’s docs put it plainly: “The best vector database is the database you already have.”
- A queue: anything that takes more than a few seconds, like summarizing a document, runs as a background job, with retries.
- Caching: OpenAI and Anthropic both cache repeated prompt prefixes. OpenAI turns it on by default for supported models. Anthropic’s cache lasts 5 minutes by default. Its cache reads cost a tenth of the base input price on most models, a twentieth on Sonnet 5.5 and Opus 5.5, and a fortieth on Fable and Mythos 5.1 and 5.2. Cache your own answers too when the same question comes up often.
- Logs: store the prompt, the model, tokens used, cost, time taken and whether the user accepted the answer. Without this you can’t debug a bad answer or explain a bill, and sooner or later you will be asked to do both.
- Guardrails: check the output against a schema, moderate what users send, cap spend per user, and decide what the app shows when the API is down or slow. That is easier to decide on a calm afternoon than during the outage.
Batch what doesn’t need an answer now
Many AI jobs are not live chats: tagging old records, writing summaries overnight, translating a catalogue. Old records are patient. Sent through OpenAI’s Batch API, these jobs cost half as much and finish within 24 hours. That one choice often matters more to the bill than which model you pick.
Privacy settings to check
- OpenAI says data sent to its API is not used to train its models unless you opt in.
- OpenAI keeps abuse-monitoring logs for up to 30 days by default.
- Zero data retention exists, but only after approval by OpenAI, and not for every feature.
- Tell your users which provider sees their data, and don’t send what the feature doesn’t need.
An example: EZ Chat
EZ Chat is my own app, live on the Overwolf store. It lets online gamers translate chats with other players into the language they choose. The front end is TypeScript and TailwindCSS on the Overwolf API. The back end runs on Supabase Edge Functions (Deno) with PostgreSQL migrations. The translation comes from the OpenAI API, and paid subscriptions run through Tebex, with the payment and subscription webhooks validated.
Translation is a single, well-defined task, so the stack around the model can stay small.
What you can skip at the start
- A separate vector database. Add one when Postgres search is measurably too slow, not before.
- An agent framework. Most features are one API call with a good prompt and checked output. It makes a dull diagram, which I count as a feature.
- Fine-tuning. Try better prompts and examples first.
- Hosting your own model. It moves the cost from the API bill to servers and staff time.
When a feature should not use AI
If the answer must be the same every time, like prices, tax, permissions or a calculation, write normal code. If a wrong answer could hurt someone or break a contract, keep a person in the loop or don’t use AI at all. And if a search box with good filters solves the problem, ship that.
Getting a second opinion
I sell this, so read it with that in mind. The Limited plan is $1,995 a month. A stack choice like these is one task on it, and build work, fixes and reviews are tasks too. If you already have an engineer you trust, ask them to sketch the layers above for your feature first. You may not need me.
Read next
Sources
- OpenAI API docs, Batch, seen 17 Nov 2026
- OpenAI API docs, Prompt caching, seen 17 Nov 2026
- OpenAI API docs, Your data, seen 17 Nov 2026
- Claude docs, Prompt caching, seen 17 Nov 2026
- Supabase docs, AI and vectors, seen 17 Nov 2026