AI Agents — LLM providers

The Agent runtime is provider-agnostic. You compose the kernel + rules + runtime context once, and one of four built-in providers translates it to the vendor’s native format. If you want a fifth, you register it — no core changes required.

Built-in providers

Why OpenRouter is one of the four: it exposes an OpenAI-compatible API over dozens of open-source models with a single billing account. You get Llama, DeepSeek, Mistral, and Qwen without integrating four separate providers.

Configuration

Each provider is bootstrapped from an env var at startup:

ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
GOOGLE_API_KEY=...
OPENROUTER_API_KEY=sk-or-...

A missing API key does not fail startup — the provider is registered as unavailable, and registry.get() throws ProviderUnavailableError when you try to use it. You can boot with only one provider configured (common in dev).

At startup, the api logs the list of available providers:

[ai-agents] LLM providers registered. Available: anthropic, openai

Cost estimation

Every provider has a per-model rate card with four dimensions:

• input_per_token
• output_per_token
• cache_read_per_token
• cache_write_per_token

Rate cards are a snapshot — vendors change them occasionally. See the provider comment in your fork for the last update date, and verify against the vendor’s pricing page before committing to a customer price.

const cost = provider.estimateCost('claude-sonnet-5', {
  input_tokens: 500,
  output_tokens: 200,
  cache_read_tokens: 12_000,
  cache_write_tokens: 3_000,
});
// Returns USD as a float.

If the model is not in the rate card, UnknownModelError is thrown. This protects against operator error at creation time — the Agent create endpoint (Phase 1.i) validates the provider + model combination before persisting.

Tool mapping

All four providers accept the same tool declaration format (the one from the §04 spec). Each provider’s adapter translates the declaration to the vendor’s native shape at runtime:

{ "name": "tool_0", "description": "...", "input_schema": { ... } }

{
  "type": "function",
  "function": { "name": "tool_0", "description": "...", "parameters": { ... } }
}

{
  "functionDeclarations": [
    { "name": "tool_0", "description": "...", "parameters": { ... } }
  ]
}

You don’t write these directly — you declare your tool once (see the Tools guide) and the provider handles the translation.

Bring your own provider

If you want to run a model that’s not in the four built-ins (custom hosted, Cohere, Mistral direct, an internal proxy), register your provider at boot:

import { LLMProvider, LLMProviderRegistry } from '@gosendapi/ai-agents';
 
class CohereProvider implements LLMProvider {
  readonly name = 'cohere';
 
  isAvailable(): boolean { return this.client !== null; }
 
  estimateCost(model: string, usage: TokenUsage): number { /* ... */ }
 
  mapToolsToNativeFormat(tools: ToolDeclaration[]): unknown { /* ... */ }
 
  async runOnce(input: NormalizedInput): Promise<NormalizedResponse> {
    // Call Cohere API. Normalize response to NormalizedResponse.
  }
}
 
// In a module init:
registry.register(new CohereProvider({ apiKey: process.env.COHERE_API_KEY }));

Rules:

• name must be unique. Registering a name that already exists throws ProviderAlreadyRegisteredError. To override a built-in intentionally, use registry.replace().
• The runOnce output must be a valid NormalizedResponse with correct usage fields — the platform depends on those numbers to compute AgentRun.costUsd.
• Throw ProviderApiError from runOnce for HTTP errors, timeouts, or config problems. Never let a raw vendor error bubble up — the operator debugging your provider deserves a normalized error.

Errors and debugging

Every error links back to this guide with #anchor fragments so operators can navigate directly.

What is not implemented yet

Phase 1.b delivers the registry + providers + rate cards + tool mapping. Coming next:

• The runtime loop (F1.d) actually invokes the providers — until then, runOnce is callable but there’s no queue worker feeding it.
• Cost tracking on AgentRun.costUsd — populated by the loop, not by this module.
• Streaming SSE for dev debug — Phase 1.d.
• Prompt caching measurement across the four providers — done in Phase 1.d with real conversations.
• A public endpoint to list available providers — Phase 1.i.

Watch the changelog for updates.