Guide
Migrating a LangChain app to Usefully
LangChain's OpenAI and Anthropic chat models both work with Usefully by changing the base URL. A step-by-step migration, including switching models per chain.
· 3 min read · Usefully team
LangChain apps tend to accumulate providers: an OpenAI model for one chain, an Anthropic model for another, maybe a Gemini model someone tried last month, each with its own key and its own bill. Moving them to Usefully consolidates that into one key and one balance, adds failover between providers, and gives you per-request costs in one place. The migration is mostly configuration.
Step 1: create a key
Create an API key in the dashboard, named for the app and environment, and put it in your environment as USEFULLY_API_KEY. Set a daily spend limit while you test.
Step 2: switch ChatOpenAI
ChatOpenAI from langchain-openai accepts a base URL. Point it at Usefully and use a full Usefully model ID. Because every model is served through the OpenAI-compatible API, the same class can run Anthropic and Google models too.
import os
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
base_url="https://api.usefully.info/v1",
api_key=os.environ["USEFULLY_API_KEY"],
model="anthropic/claude-sonnet-5-5", # or "openai/gpt-5", "google/gemini-2.5-flash"
)
print(llm.invoke("Say hello in five words.").content)Step 3 (optional): keep ChatAnthropic
If parts of your app rely on ChatAnthropic, you can keep it. Usefully also speaks the Anthropic Messages API at /v1/messages, and accepts the x-api-key header the Anthropic client sends. The Anthropic client adds /v1/messages itself, so the base URL has no /v1:
from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(
base_url="https://api.usefully.info",
api_key=os.environ["USEFULLY_API_KEY"],
model="anthropic/claude-sonnet-5-5",
)Step 4: centralize model choice
Most LangChain apps create chat models in several places. Replace them with one factory, so switching a chain's model is a one-word change and the base URL lives in one place:
MODELS = {
"router": "anthropic/claude-haiku-5-5",
"answer": "anthropic/claude-sonnet-5-5",
"extract": "openai/gpt-5-mini",
}
def chat_model(role: str, **kw) -> ChatOpenAI:
return ChatOpenAI(base_url="https://api.usefully.info/v1", api_key=os.environ["USEFULLY_API_KEY"],
model=MODELS[role], **kw)
chain = prompt | chat_model("answer", temperature=0) | parserStep 5: check what still works the same
- Streaming (
.stream(),.astream()) works unchanged. - Tool calling (
.bind_tools()) works with models that support tools. Check the models page filter. - Structured output: test
.with_structured_output()on each model you use; support varies by model. - Callbacks and tracing are client-side and unaffected.
Step 6: compare and cut over
Run your existing tests or a set of real inputs through the migrated app. Then look at the dashboard: Logs show each request's model, provider, tokens, cost and latency, and spend by model shows which chains cost the most. It's common to find a chain using a large model for a task a smaller one handles fine.
What you can delete afterwards
- Separate provider API keys in your environment and secret manager.
- Hand-written retry and provider-switching code that the gateway now handles (keep basic SDK retries).
- Per-provider cost tracking scripts, if Logs and spend charts cover what you needed.
A note on embeddings and other components
This guide covers chat models. If your app also uses an embedding model or a vector store, migrate those separately and on their own schedule; there's no need to move everything at once. LangChain lets you mix providers freely, so a chain can use a Usefully chat model alongside your existing retriever.
Rolling out gradually
For a production app, switch one chain at a time. Start with a low-risk chain, compare its outputs and costs for a few days, then move the next one. Keeping the model factory from step 4 means each switch is a small, reviewable change, and rolling back is just as small.
LlamaIndex, the Vercel AI SDK and other frameworks work the same way. See the quickstart and API reference, or ask at support@usefully.info.