Guide
Building a RAG pipeline on live web search
Retrieval-augmented generation usually means a vector database. For questions about the current world, the web is the index. A complete pipeline in about 40 lines.
· 3 min read · Usefully team
Retrieval-augmented generation (RAG) is simple in principle: find relevant text, put it in the prompt, and ask the model to answer from it. Most RAG tutorials start with chunking and a vector database, which is right for your own documents. But a lot of questions are about the world outside: a library's latest release, a regulation that changed last month, a competitor's pricing page. For those, the web is the index, and you don't have to build or refresh it.
Usefully puts web search and models behind the same API key and balance, so a web RAG pipeline is two API calls.
The shape of the pipeline
- 1Rewrite the user's question into a search query (optional, but it helps with chatty questions).
- 2Search with
POST /v1/search, asking for page text withinclude_content. - 3Select the best few results and trim them to a budget.
- 4Answer with a model, instructed to use only the numbered sources and cite them.
The code
import os, requests
from openai import OpenAI
KEY = os.environ["USEFULLY_API_KEY"]
BASE = "https://api.usefully.info/v1"
client = OpenAI(base_url=BASE, api_key=KEY)
def search(query: str, domains: list[str] | None = None) -> list[dict]:
body = {"query": query, "num_results": 6, "include_content": True}
if domains:
body["include_domains"] = domains
r = requests.post(f"{BASE}/search", json=body,
headers={"Authorization": f"Bearer {KEY}"}, timeout=30)
r.raise_for_status()
return r.json()["results"]
def to_query(question: str) -> str:
reply = client.chat.completions.create(
model="anthropic/claude-haiku-5-5",
max_tokens=40,
messages=[{"role": "user", "content": f"Write a short web search query for: {question}\nReturn the query only."}],
)
return reply.choices[0].message.content.strip()
def answer(question: str) -> str:
results = search(to_query(question))
sources = "\n\n".join(
f"[{i}] {r['title']} ({r['url']}, {r.get('published_at') or 'undated'})\n"
f"{(r.get('content') or r['snippet'])[:4000]}"
for i, r in enumerate(results, 1)
)
reply = client.chat.completions.create(
model="anthropic/claude-sonnet-5-5",
messages=[
{"role": "system", "content": "Answer using only the sources. Cite like [2]. If they don't answer it, say so."},
{"role": "user", "content": f"Question: {question}\n\nSources:\n{sources}"},
],
)
return reply.choices[0].message.contentDesign choices that matter
- A small model for the query, a stronger one for the answer. Rewriting a query is easy; synthesizing sources is not. Splitting the work keeps cost down.
- Cap each source. Trimming page text to a few thousand characters keeps the prompt focused and the cost predictable. Five trimmed sources usually beat twenty full pages.
- Pass dates. Each result can carry
published_at. Including it lets the model prefer recent sources and say how current its answer is. - Restrict domains when you can. For product documentation or official guidance,
include_domainskeeps forum posts and content farms out.exclude_domainsdoes the opposite. - Use freshness for news.
"freshness": "week"or"month"drops stale pages from time-sensitive questions.
Mixing in your own documents
Web search doesn't replace your private knowledge base; it complements it. A common setup retrieves from both, labels each source ([internal-3], [web-2]), and lets the model combine them. Keep the citation format consistent so your UI can render links for either kind.
Cost and observability
Search is billed per query and the model per token, both from the same prepaid balance. In Logs you'll see three entries per question (query rewrite, search, answer), each with its cost and latency, which makes it easy to see which step dominates. Failed searches and failed model calls aren't charged.
Handling no good results
Sometimes the search simply doesn't find anything useful: the question is too vague, too new, or about something private. Two safeguards help. First, keep the instruction that tells the model to say when the sources don't answer the question, and make your UI treat that answer as a normal outcome rather than an error. Second, if the top results all have low relevance or are off-topic, try one rewritten query before answering. A single retry with a more specific query fixes many misses without letting cost run away.
Also decide how to handle questions that don't need the web at all, such as greetings or questions about your own product. A cheap classification step before searching can skip the search for those and save a query.
Search parameters are in the search guide. For a deeper look at citations, read citing sources the user can check. Try a query on the search page first.