Skip to main content
GPTRouter is an open source LLM API Gateway that offers a universal API for 30+ LLMs, vision, and image models, with smart fallbacks based on uptime and latency, automatic retries, and streaming. This notebook covers how to get started with using LangChain + the GPTRouter I/O library.
  • Set GPT_ROUTER_API_KEY environment variable
  • or use the gpt_router_api_key keyword argument
pip install -qU  GPTRouter

from langchain_community.chat_models import GPTRouter
from langchain_community.chat_models.gpt_router import GPTRouterModel
from langchain.messages import HumanMessage

anthropic_claude = GPTRouterModel(name="claude-instant-1.2", provider_name="anthropic")

chat = GPTRouter(models_priority_list=[anthropic_claude])

messages = [
    HumanMessage(
        content="Translate this sentence from English to French. I love programming."
    )
]
chat(messages)

AIMessage(content=" J'aime programmer.")

​
GPTRouter also supports async and streaming functionality

from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler

await chat.agenerate([messages])

LLMResult(generations=[[ChatGeneration(text=" J'aime programmer.", generation_info={'finish_reason': 'stop_sequence'}, message=AIMessage(content=" J'aime programmer."))]], llm_output={}, run=[RunInfo(run_id=UUID('9885f27f-c35a-4434-9f37-c254259762a5'))])

chat = GPTRouter(
    models_priority_list=[anthropic_claude],
    streaming=True,
    verbose=True,
    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
)
chat(messages)

 J'aime programmer.

AIMessage(content=" J'aime programmer.")
Edit this page on GitHub or file an issue.
Connect these docs to Claude, VSCode, and more via MCP for real-time answers.