Skip to content
Free$1 of credit for every new organization
All articles

Guide

Building a RAG pipeline on live web search

Retrieval-augmented generation usually means a vector database. For questions about the current world, the web is the index. A complete pipeline in about 40 lines.

· 3 min read · Usefully team


Retrieval-augmented generation (RAG) is simple in principle: find relevant text, put it in the prompt, and ask the model to answer from it. Most RAG tutorials start with chunking and a vector database, which is right for your own documents. But a lot of questions are about the world outside: a library's latest release, a regulation that changed last month, a competitor's pricing page. For those, the web is the index, and you don't have to build or refresh it.

Usefully puts web search and models behind the same API key and balance, so a web RAG pipeline is two API calls.

The shape of the pipeline

  1. 1Rewrite the user's question into a search query (optional, but it helps with chatty questions).
  2. 2Search with POST /v1/search, asking for page text with include_content.
  3. 3Select the best few results and trim them to a budget.
  4. 4Answer with a model, instructed to use only the numbered sources and cite them.

The code

import os, requests
from openai import OpenAI

KEY = os.environ["USEFULLY_API_KEY"]
BASE = "https://api.usefully.info/v1"
client = OpenAI(base_url=BASE, api_key=KEY)

def search(query: str, domains: list[str] | None = None) -> list[dict]:
    body = {"query": query, "num_results": 6, "include_content": True}
    if domains:
        body["include_domains"] = domains
    r = requests.post(f"{BASE}/search", json=body,
                      headers={"Authorization": f"Bearer {KEY}"}, timeout=30)
    r.raise_for_status()
    return r.json()["results"]

def to_query(question: str) -> str:
    reply = client.chat.completions.create(
        model="anthropic/claude-haiku-5-5",
        max_tokens=40,
        messages=[{"role": "user", "content": f"Write a short web search query for: {question}\nReturn the query only."}],
    )
    return reply.choices[0].message.content.strip()

def answer(question: str) -> str:
    results = search(to_query(question))
    sources = "\n\n".join(
        f"[{i}] {r['title']} ({r['url']}, {r.get('published_at') or 'undated'})\n"
        f"{(r.get('content') or r['snippet'])[:4000]}"
        for i, r in enumerate(results, 1)
    )
    reply = client.chat.completions.create(
        model="anthropic/claude-sonnet-5-5",
        messages=[
            {"role": "system", "content": "Answer using only the sources. Cite like [2]. If they don't answer it, say so."},
            {"role": "user", "content": f"Question: {question}\n\nSources:\n{sources}"},
        ],
    )
    return reply.choices[0].message.content

Design choices that matter

  • A small model for the query, a stronger one for the answer. Rewriting a query is easy; synthesizing sources is not. Splitting the work keeps cost down.
  • Cap each source. Trimming page text to a few thousand characters keeps the prompt focused and the cost predictable. Five trimmed sources usually beat twenty full pages.
  • Pass dates. Each result can carry published_at. Including it lets the model prefer recent sources and say how current its answer is.
  • Restrict domains when you can. For product documentation or official guidance, include_domains keeps forum posts and content farms out. exclude_domains does the opposite.
  • Use freshness for news. "freshness": "week" or "month" drops stale pages from time-sensitive questions.

Mixing in your own documents

Web search doesn't replace your private knowledge base; it complements it. A common setup retrieves from both, labels each source ([internal-3], [web-2]), and lets the model combine them. Keep the citation format consistent so your UI can render links for either kind.

Cost and observability

Search is billed per query and the model per token, both from the same prepaid balance. In Logs you'll see three entries per question (query rewrite, search, answer), each with its cost and latency, which makes it easy to see which step dominates. Failed searches and failed model calls aren't charged.

Handling no good results

Sometimes the search simply doesn't find anything useful: the question is too vague, too new, or about something private. Two safeguards help. First, keep the instruction that tells the model to say when the sources don't answer the question, and make your UI treat that answer as a normal outcome rather than an error. Second, if the top results all have low relevance or are off-topic, try one rewritten query before answering. A single retry with a more specific query fixes many misses without letting cost run away.

Also decide how to handle questions that don't need the web at all, such as greetings or questions about your own product. A cheap classification step before searching can skip the search for those and save a query.

Search parameters are in the search guide. For a deeper look at citations, read citing sources the user can check. Try a query on the search page first.

More articles

Get started

Your first request is five minutes away.

  • $1 of free credit to start
  • No card, no subscription
  • Keep the SDK you already use
  • Real people at support@usefully.info
Building a RAG pipeline on live web search · Usefully