AnyRouter

LlamaIndex

Use LlamaIndex with AnyRouter. Connect OpenAILike (Python) or the OpenAI LLM (TypeScript) to power RAG and agents on free or paid models.

Drive your LlamaIndex indexes and agents with any model in AnyRouter's catalog. In Python, use LlamaIndex's OpenAILike class — its integration for OpenAI-compatible servers — because the plain OpenAI class only accepts OpenAI's own model names. In TypeScript, the @llamaindex/openai LLM takes a custom baseURL.

Before you start

  • A LlamaIndex project in Python or TypeScript.
  • An AnyRouter API key (prefix sk-ar-). Create one in the dashboard.

Set it up

Pick your language, install the integration, then configure the base URL, your key, and a model from the catalog. Use the LLM in your index querying or agent loops.

pip install llama-index-llms-openai-like
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="anyrouter/free",
    api_base="https://anyrouter.dev/api/v1",
    api_key=os.environ["ANYROUTER_API_KEY"],  # sk-ar-...
    is_chat_model=True,
    context_window=128000,
)

response = llm.complete("Explain semantic search in one sentence.")
print(response.raw.model)  # the model that served the request
print(response)

is_chat_model=True sends requests to the Chat Completions endpoint. Set context_window to the context length of the model you use (listed on its catalog page) so LlamaIndex sizes prompts correctly.

Verify

Export your key, then run the snippet. A printed completion confirms LlamaIndex is routing through AnyRouter.

export ANYROUTER_API_KEY="sk-ar-your-key-here"

Every request also shows up in Logs with the model, provider, tokens, and latency.

Free models

Set the model to anyrouter/free and AnyRouter routes the request to a free model that is healthy right now — no credits used. Free models have no daily cap on any plan. Details on Free Tier.

Use anyrouter/auto to let AnyRouter pick from the full catalog, or any provider/model id from the catalog to pin one model.

Bring your own keys

Already pay for OpenAI, Anthropic, Google, or another provider? Add those keys to AnyRouter and your requests use them, with no credits deducted. See Get a provider API key and BYOK.

Troubleshooting

Confirm the key starts with sk-ar- and is passed as api_key (Python) or apiKey (TypeScript). Manage keys in the dashboard.

The Python OpenAI class from llama_index.llms.openai rejects model ids that are not OpenAI's own. Use OpenAILike from llama-index-llms-openai-like as shown above.

Pass a full catalog id — anyrouter/free, anyrouter/auto, or a provider/model id such as z-ai/glm-4.7-flash. Valid ids live in the catalog.