2026
RAG Production App
The same question a RAG demo answers, asked under production constraints: what happens when ingestion fails halfway, when the free tier is cold, and when someone uploads the same document twice.
cat problem.md
The problem
A retrieval demo runs once, on a warm machine, with a small file. Production is the opposite: requests arrive cold, embedding a large PDF takes minutes, and any step can fail partway through and leave the vector store half-written.
Free-tier hosting makes all of that sharper. Render's free instance sleeps and has 512 MB of memory, which is less than a naive embedding batch will use on a long document.
The goal was to keep a real RAG pipeline honest under those limits rather than to make the happy path faster.
tree --architecture
Architecture
Streamlit UI (Streamlit Cloud)
|
| POST /ingest POST /query
v
FastAPI backend (Render free tier)
|
+-- load + chunk PDF
| sentence-aware, 1000 / 200 overlap
|
+-- embed -> FastEmbed bge-small-en-v1.5
| local ONNX, batch_size 8
|
+-- upsert -> Qdrant Cloud
| UUID5 ids, cosine, dim 384
|
+-- answer -> Groq llama-3.1-8b-instant
|
+-- /api/inngest
durable functions, retries
throttle 2/min, rate limit 1 per 4hThe HTTP path and the durable Inngest path run the same steps; only the execution guarantees differ.
| layer | technology | responsibility |
|---|---|---|
| Frontend | Streamlit on Streamlit Cloud | Upload, question box, cold-start handling |
| Backend | FastAPI + Uvicorn on Render | Ingest and query endpoints, Inngest function registration |
| Embeddings | FastEmbed, BAAI/bge-small-en-v1.5 | Local ONNX inference, 384 dimensions, no embedding API key |
| Vector store | Qdrant Cloud | Cosine similarity search over a free 1 GB cluster |
| Orchestration | Inngest | Durable steps with retries, throttling, and per-source rate limits |
| Generation | Groq, llama-3.1-8b-instant | Answers grounded in the top-k retrieved chunks |
git log --decisions
Key decisions
Ingestion is two durable steps, not one function
Loading and chunking is one step, embedding and upserting is another. Inngest checkpoints between them, so a failure during embedding retries from the chunks it already has instead of re-reading and re-splitting the PDF.
Chunk ids are derived, not random
Each chunk id is a UUID5 of the source plus its index. Re-ingesting the same document overwrites the same points rather than duplicating the corpus, which makes ingestion idempotent for free.
Throttling and rate limiting are part of the workflow
Two ingests per minute, and one per source every four hours. Declared on the Inngest function rather than bolted on at the edge, so the limit holds regardless of how the event was triggered.
Embedding batches are deliberately small
A batch size of eight keeps peak memory under Render's 512 MB ceiling. Larger batches are faster right up until the container is killed mid-document, which costs far more than the throughput gained.
The UI waits for the backend to wake up
A free Render instance sleeps. The Streamlit client polls the health endpoint for up to four minutes with a visible message instead of throwing a connection error at a visitor who did nothing wrong.
One Qdrant client for the process
A module-level singleton reuses the connection across steps rather than opening a client per call, which matters when durable steps run repeatedly inside the same worker.
cat outcome.md
Outcome
- Live on Streamlit Cloud with the API on Render, using Qdrant Cloud's free 1 GB cluster and Groq's free inference tier.
- Ingestion is idempotent and resumable, and the whole pipeline runs with no embedding API key and no paid service.
- Both execution paths are registered: direct HTTP endpoints for the UI, and event-driven Inngest functions at /api/inngest for durable runs.