Skip to content
Free$1 of credit for every new organization
All articles

Guide

Migrating a LangChain app to Usefully

LangChain's OpenAI and Anthropic chat models both work with Usefully by changing the base URL. A step-by-step migration, including switching models per chain.

· 3 min read · Usefully team


LangChain apps tend to accumulate providers: an OpenAI model for one chain, an Anthropic model for another, maybe a Gemini model someone tried last month, each with its own key and its own bill. Moving them to Usefully consolidates that into one key and one balance, adds failover between providers, and gives you per-request costs in one place. The migration is mostly configuration.

Step 1: create a key

Create an API key in the dashboard, named for the app and environment, and put it in your environment as USEFULLY_API_KEY. Set a daily spend limit while you test.

Step 2: switch ChatOpenAI

ChatOpenAI from langchain-openai accepts a base URL. Point it at Usefully and use a full Usefully model ID. Because every model is served through the OpenAI-compatible API, the same class can run Anthropic and Google models too.

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.usefully.info/v1",
    api_key=os.environ["USEFULLY_API_KEY"],
    model="anthropic/claude-sonnet-5-5",   # or "openai/gpt-5", "google/gemini-2.5-flash"
)
print(llm.invoke("Say hello in five words.").content)

Step 3 (optional): keep ChatAnthropic

If parts of your app rely on ChatAnthropic, you can keep it. Usefully also speaks the Anthropic Messages API at /v1/messages, and accepts the x-api-key header the Anthropic client sends. The Anthropic client adds /v1/messages itself, so the base URL has no /v1:

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    base_url="https://api.usefully.info",
    api_key=os.environ["USEFULLY_API_KEY"],
    model="anthropic/claude-sonnet-5-5",
)

Step 4: centralize model choice

Most LangChain apps create chat models in several places. Replace them with one factory, so switching a chain's model is a one-word change and the base URL lives in one place:

MODELS = {
    "router":  "anthropic/claude-haiku-5-5",
    "answer":  "anthropic/claude-sonnet-5-5",
    "extract": "openai/gpt-5-mini",
}

def chat_model(role: str, **kw) -> ChatOpenAI:
    return ChatOpenAI(base_url="https://api.usefully.info/v1", api_key=os.environ["USEFULLY_API_KEY"],
                      model=MODELS[role], **kw)

chain = prompt | chat_model("answer", temperature=0) | parser

Step 5: check what still works the same

  • Streaming (.stream(), .astream()) works unchanged.
  • Tool calling (.bind_tools()) works with models that support tools. Check the models page filter.
  • Structured output: test .with_structured_output() on each model you use; support varies by model.
  • Callbacks and tracing are client-side and unaffected.

Step 6: compare and cut over

Run your existing tests or a set of real inputs through the migrated app. Then look at the dashboard: Logs show each request's model, provider, tokens, cost and latency, and spend by model shows which chains cost the most. It's common to find a chain using a large model for a task a smaller one handles fine.

What you can delete afterwards

  • Separate provider API keys in your environment and secret manager.
  • Hand-written retry and provider-switching code that the gateway now handles (keep basic SDK retries).
  • Per-provider cost tracking scripts, if Logs and spend charts cover what you needed.

A note on embeddings and other components

This guide covers chat models. If your app also uses an embedding model or a vector store, migrate those separately and on their own schedule; there's no need to move everything at once. LangChain lets you mix providers freely, so a chain can use a Usefully chat model alongside your existing retriever.

Rolling out gradually

For a production app, switch one chain at a time. Start with a low-risk chain, compare its outputs and costs for a few days, then move the next one. Keeping the model factory from step 4 means each switch is a small, reviewable change, and rolling back is just as small.

LlamaIndex, the Vercel AI SDK and other frameworks work the same way. See the quickstart and API reference, or ask at support@usefully.info.

More articles

Get started

Your first request is five minutes away.

  • $1 of free credit to start
  • No card, no subscription
  • Keep the SDK you already use
  • Real people at support@usefully.info
Migrating a LangChain app to Usefully · Usefully