Using one LLM in a product is straightforward. You create an account, save an API key, add a payment method, and start sending requests.
Using three models creates a different kind of problem.
Three models can mean three accounts, three sets of credentials, three invoices or credit balances, three dashboards, and three different ways to configure rate limits. Add a fourth provider and somebody has to learn another billing panel, map another error format, and remember where the production limit is configured.
This is the problem LLM gateways promise to solve: one endpoint, one key, one place to observe traffic, and sometimes one bill. The word sometimes matters. A gateway can unify the technical interface without unifying invoices. It can remove a platform fee while giving your team a new service to operate. It can make models easier to switch while adding another company to the data path.
I compared four gateways that represent the main approaches particularly well: OpenRouter, LiteLLM, Vercel AI Gateway, and Cloudflare AI Gateway. The market is broader than these four - Helicone, Portkey, and other products solve overlapping problems - but the economic choices are remarkably consistent.
Prices and product details were verified on 22 July 2026. Gateway pricing, model catalogues, and limits change frequently. Follow the linked vendor pages before making a purchasing decision.
One product, several models, too many control panels
The first version of a multi-model integration often looks harmless. One model writes content, another handles complex reasoning, and a smaller model classifies requests. Each provider’s SDK works, and direct billing appears cheaper because there is no intermediary.
The operational cost arrives later.
- Finance reconciles several suppliers and payment schedules.
- Engineers manage separate production and development keys.
- Rate limits live in different dashboards and use different units.
- Model names, request fields, streaming events, and errors differ.
- Usage data must be combined before anyone knows what one customer or feature costs.
- A provider outage becomes application-specific incident logic.
None of these problems is impossible. The question is whether solving them is part of your product’s competitive advantage.
An LLM gateway moves this work into a shared layer. Your application calls the gateway, and the gateway calls the selected provider. That shared layer can be operated by a vendor, hosted in your cloud account, or implemented inside your own backend.
What an LLM gateway actually does
“One API for many models” is only one gateway function. In practice, four separate jobs are often bundled under the same name.
API normalization gives the application a stable endpoint and a mostly consistent request shape. Changing the model can become a configuration change instead of a new SDK integration.
Credential and billing aggregation decides who owns provider accounts and who pays the providers. A hosted gateway may resell inference through prepaid credits. A bring-your-own-key gateway uses your provider credentials, leaving the direct provider bill in place.
Routing and resilience decide where a request goes. The gateway can retry another deployment, prefer a cheaper host, enforce a regional policy, or fall back to another model when the first one fails.
Observability and policy make usage visible in one place. This can include latency, token counts, cost attribution, budgets, rate limits, caching, logging, and data-retention rules.
These jobs do not have to come from the same product. You can normalize APIs in your code, route through a hosted gateway, and export usage to a separate observability service. Bundling is convenient, but it also determines where your dependencies live.
The three ways gateways make the economics work
Most gateway products fit one of three economic models.
1. A credit intermediary
You pay the gateway, and the gateway pays the model providers. The gateway gives you one key and one balance, then funds its service through a fee, a margin, enterprise pricing, or negotiated provider economics.
This model removes the most administrative work. It also puts the gateway between your company and the provider commercially and technically.
2. A self-hosted BYOK gateway
You run the gateway and insert your own provider keys. The gateway unifies the endpoint, policies, and logs, but every provider continues billing you directly.
There may be no percentage fee on tokens, but there is infrastructure and engineering work. Your team owns upgrades, availability, secrets, scaling, and incidents.
3. A hosted gateway financed through a larger platform
The platform can offer the gateway with no token markup because it earns money from other services, higher plans, payment processing, enterprise agreements, or the broader customer relationship.
This can look like the best of both worlds. The trade-off is deeper reliance on one platform and the possibility that premium features sit outside the free gateway layer.
Cloudflare combines the first and third models. Its core gateway works with your provider keys for free, while Unified Billing replaces those keys with Cloudflare credits and a percentage fee.
OpenRouter: maximum catalogue, minimum setup
OpenRouter is the clearest version of the credit-intermediary model. Create an account, buy credits, generate one key, and use a hosted API covering more than 400 models from more than 70 providers as of the verification date.
The underlying inference rate is passed through without a token-price markup. OpenRouter instead charges 5.5% when credits are purchased, with a $0.80 minimum. Crypto credit purchases carry a 5% fee. This distinction matters: the model can still appear at its provider list price while your total acquisition cost is higher because funding the balance has a fee.
Pay-as-you-go is primarily a prepaid-credit relationship. You can top up manually or automatically, and usage is deducted from the balance. The pricing page lists invoicing options for Enterprise, but a normal credit purchase should not be described as a postpaid consolidated invoice.
OpenRouter also supports BYOK. You can connect direct provider credentials, control their priority, and decide whether shared OpenRouter capacity may be used as fallback. There is a catch: OpenRouter still charges a 5% BYOK fee after a free allowance. Its official documentation currently disagrees about that allowance. The pricing page describes $25,000 of list-price inference per month without a fee on pay-as-you-go, while the BYOK guide describes the first one million BYOK requests per month. If BYOK economics affect your decision, confirm the current rule with OpenRouter rather than building a forecast from either page alone.
Routing is a major part of the product. OpenRouter can fall back between providers hosting the same model, apply provider order and data policies, and favor price or throughput. Its activity view centralizes usage by model, provider, and key.
The convenience adds a new party to the request path. OpenRouter says it logs request metadata but does not log prompts and completions by default. The selected inference provider still receives the request, and its retention policy still applies.
Best fit: teams that value fast access to a very broad catalogue and want to eliminate most provider onboarding work.
Main trade-off: the credit-purchase fee grows with spend, and the product adds both a commercial and technical dependency.
LiteLLM: control in exchange for operational ownership
LiteLLM takes the opposite approach. It is an open-source gateway and Python SDK that presents a common interface for more than 100 LLM APIs. You deploy the proxy, configure provider credentials, and give applications LiteLLM virtual keys instead of exposing upstream keys.
The open-source core does not add a percentage to token usage. Providers bill your existing accounts, so direct contracts, committed spend, regional deployments, and negotiated rates remain available. The accounting problem also remains: five providers can still mean five supplier relationships and five bills.
In return, LiteLLM gives you a control plane you can adapt. It supports virtual keys, budgets, spend tracking, rate limits, model access rules, retries, fallbacks, and several load-balancing strategies. An internal model alias can represent multiple provider deployments, allowing the application to use a stable name while operations changes the route behind it.
Self-hosting can start small. A Docker container can run on a VPS, Render, Railway, or another container host. The current production deployment guide also documents Helm on EKS, GKE, and AKS, plus Terraform modules for AWS on ECS Fargate and Google Cloud on Cloud Run.
Production ownership is more substantial than running one container. LiteLLM’s high-availability architecture uses at least two proxy replicas behind a load balancer, PostgreSQL for keys and spend data, Redis for shared routing state, an explicit migration job, health checks, secret management, monitoring, and pinned images. Someone must patch, scale, back up, and respond to that stack.
There is also an important data-boundary correction. Hosting LiteLLM yourself removes a third-party gateway operator from the request path. It does not keep prompts inside your infrastructure when the configured destination is OpenAI, Anthropic, Google, or another external provider. Only a model hosted inside your own boundary can do that.
LiteLLM’s repository combines MIT-licensed core code with separately licensed enterprise functionality and paid support. “No gateway markup” is accurate for the open-source core. “Free production gateway” ignores infrastructure, engineering, and any enterprise capabilities your organization requires.
Best fit: teams that need provider contracts, infrastructure control, custom policies, or an internal gateway shared across many products.
Main trade-off: billing stays fragmented, and your team becomes the gateway’s platform and on-call team.
Vercel AI Gateway: zero markup inside a platform relationship
Vercel AI Gateway is a hosted service with a surprisingly aggressive pricing model: provider-list-price rates with no Vercel token markup. It works through the AI SDK as well as OpenAI- and Anthropic-compatible interfaces. An application can call the HTTPS API with a key even when that application is not hosted on Vercel.
Every Vercel team starts with $5 of monthly gateway credit after its first request. Once the team purchases paid credits, the monthly free credit stops and the account becomes pay-as-you-go. Vercel says payment-processing fees may still apply, so “zero markup” should not be translated into “every payment is fee-free.”
BYOK is also available without an added gateway fee. This lets a company use existing provider contracts and credits. Vercel tries the customer credential first and can retry with system credentials if it fails. Because that fallback is charged to Vercel credits, a team must keep a gateway credit balance even when using BYOK. That behavior improves availability, but it can complicate procurement and cost attribution if a company expects all traffic to stay on its direct agreement.
Billing deserves precise language. The normal path uses pre-purchased credits. Vercel says Enterprise customers can arrange a consolidated invoice across providers in their routing pool. “One Vercel bill” is therefore possible, but postpaid invoicing is not the default settlement model for every account.
The gateway supports provider ordering, automatic selection based on recent uptime and latency, timeouts, same-model provider fallback, and ordered fallback models. Its observability covers requests, spend, tokens, providers, projects, and time to first token. Vercel also advertises Zero Data Retention routing, no-training controls, and provider allowlists.
The service is not self-hostable. Its direct financial fee may be zero, but the architectural price is reliance on Vercel’s gateway, account, policy model, and future product decisions.
Best fit: teams that want a hosted gateway with broad model access, strong routing, and no current token markup.
Main trade-off: the economics depend on a continuing platform relationship, and BYOK fallback can still consume Vercel credits.
Cloudflare AI Gateway: BYOK and unified billing in one service
Cloudflare is the only product in this comparison that makes the billing model an explicit choice.
With BYOK, you connect provider credentials and continue paying those providers directly. Cloudflare’s core gateway features are currently free on all plans, including dashboard analytics, caching, and rate limiting. There is no Unified Billing credit fee on BYOK traffic.
With Unified Billing, Cloudflare supplies the credential and deducts third-party inference from prepaid Cloudflare credits. Provider inference prices are passed through without markup, while Cloudflare adds 5% when credits are purchased. Buying $100 of credits therefore costs $105. Workers AI models are an exception: models with the @cf/ prefix follow Workers AI pricing rather than Unified Billing.
This creates a useful per-provider choice. A company can keep a negotiated direct contract for one provider, use Unified Billing for occasional access to another, and route both through Cloudflare’s observability and policy layer. The trade-off is operational clarity: finance and engineering must understand which route uses which balance and supplier bill.
Cloudflare also exposes spend limits, caching, logs, rate limits, and dynamic routing. Dynamic routes can branch on metadata, split traffic for experiments, apply budget rules, and select a fallback. The public Unified Billing limit is currently 200 requests per 60 seconds per gateway; that limit does not apply to BYOK traffic.
“The gateway is free” needs two qualifications. First, Cloudflare says some future features may become premium, while Guardrails can generate Workers AI charges and Logpush requires Workers Paid. Second, request and response bodies are stored in gateway logs by default. Teams can disable payload storage with a header while retaining metadata, but sensitive-data handling must be configured deliberately.
Best fit: teams that want hosted gateway controls and the ability to choose between direct provider billing and one Cloudflare balance.
Main trade-off: there are two financial paths to understand, Unified Billing has a 5% credit fee, and the service remains Cloudflare-hosted.
| Gateway | Deployment | Credentials | Default billing model | Gateway charge | Operational owner |
|---|---|---|---|---|---|
| OpenRouter | Hosted | OpenRouter key or BYOK | Prepaid credits; Enterprise invoicing options | 5.5% ($0.80 minimum) on credit purchases; 5% BYOK fee after a documented but currently inconsistent free allowance | OpenRouter |
| LiteLLM | Self-hosted | Provider keys behind LiteLLM virtual keys | Separate provider bills | No per-token fee for the open-source core; infrastructure and optional enterprise costs remain | Your team |
| Vercel AI Gateway | Hosted | Vercel key or BYOK | Prepaid credits; Enterprise invoice option | No Vercel token markup; payment-processing fees may apply | Vercel |
| Cloudflare AI Gateway | Hosted | Provider keys or Cloudflare Unified Billing | Separate provider bills with BYOK, or Cloudflare credits | Free core gateway; 5% on Unified Billing credit purchases | Cloudflare |
One invoice or no gateway fee: choose which problem to keep
“One bill” sounds binary, but four different financial documents are easily confused:
- a prepaid credit purchase;
- a card receipt or account charge;
- a direct provider invoice based on monthly usage;
- a negotiated consolidated enterprise invoice.
OpenRouter pay-as-you-go, Vercel’s default flow, and Cloudflare Unified Billing all rely on credits. The exact tax document, currency treatment, VAT handling, purchase order support, and payment terms can matter more to finance than the API design. Public product pages do not answer every country- and company-specific procurement question. Confirm those details before making the gateway your only production route.
The percentage cost is easier to see with simple examples:
| Credits purchased | OpenRouter at 5.5% | Cloudflare Unified Billing at 5% |
|---|---|---|
| $100 | $5.50 fee | $5 fee |
| $1,000 | $55 fee | $50 fee |
| $10,000 | $550 fee | $500 fee |
These are gateway-funding fees, not complete inference bills. They exclude payment-processing charges, taxes, model-specific prices, and negotiated contracts. They also assume the published percentages apply to the full purchase.
At low spend, the fee may be cheaper than the engineering time required to integrate and reconcile several suppliers. At high spend, a few percentage points become a visible infrastructure line item. Direct provider discounts or committed-spend agreements can widen that difference further.
BYOK reverses the trade-off. You preserve provider economics and direct invoices, but you do not get one supplier relationship. The gateway unifies traffic, not accounting.
Self-hosting is not free
Self-hosting replaces a variable gateway fee with a mixture of fixed and operational costs.
For a prototype, the stack might be one small VPS, a Docker container, and provider keys stored as environment secrets. That is inexpensive and easy to understand. It also creates one gateway process whose failure can stop every AI feature.
For production, cost categories grow:
- compute for multiple gateway replicas;
- a load balancer and TLS termination;
- PostgreSQL for keys, configuration, and spend data;
- Redis for shared rate-limit and routing state;
- log storage, metrics, tracing, and alerts;
- backups, secret management, security updates, and incident response;
- engineering time for upgrades, migrations, provider changes, and capacity planning.
You can host these pieces on AWS, Google Cloud, Azure, Kubernetes, a container platform, or your own infrastructure. Hosting choice changes the invoice, not the ownership.
Consider a deliberately simple example: $50 per month of infrastructure plus two engineering hours at an assumed loaded cost of $100 per hour. The gateway costs $250 per month before on-call work. That equals a 5% fee on $5,000 of credits or a 5.5% fee on roughly $4,545.
This is not a LiteLLM price quote. Real infrastructure could be cheaper for a small internal tool or much more expensive for a multi-region regulated service. The example shows why “no markup” is not enough to compare architectures. The relevant number is total cost of ownership.
Self-hosting becomes more attractive when the organization already operates the necessary platform, needs custom policies, or saves enough through direct contracts to fund the work. It becomes less attractive when the gateway is an isolated snowflake owned by one engineer.
Switching models, routing traffic, and surviving provider failures
A gateway can reduce the amount of code needed to change models, but it cannot make models interchangeable.
The useful pattern is to give the application a stable internal name:
support-fast
-> primary: fast model through provider A
-> provider fallback: the same model through provider B
-> model fallback: a different model through provider CThe application asks for support-fast. The platform team can change the underlying route without editing every caller. The same boundary can enforce a budget, send premium customers to a stronger model, keep European traffic on an approved endpoint, or split a small percentage of requests into an experiment.
There are several distinct failure controls:
Retries repeat an attempt, usually after a transient error. They need limits because repeated work can increase latency and sometimes cost.
Provider fallback sends the same model to a different host or deployment. This is the least disruptive fallback when multiple providers serve compatible versions.
Model fallback changes the model. It improves availability but can change quality, context limits, tool behavior, structured output, safety filters, and price.
Load balancing distributes normal traffic across endpoints based on capacity, latency, cost, rate limits, or policy. It is not only an outage mechanism.
The unified API covers the common path. Provider-specific capabilities still leak through: tool definitions, JSON-schema support, prompt caching, batch APIs, file handling, reasoning controls, moderation, token accounting, and streaming events do not behave identically. A base-URL change can move the request, but proper model switching still requires evaluation and contract tests around the behavior your product depends on.
When building your own gateway makes sense
“Build our own gateway” can describe two very different projects.
A thin internal adapter is a small module inside the application. It maps internal model names to providers, normalizes a narrow response shape, records usage, and implements one or two fallbacks. This can be the right answer when one product uses a few models and the team wants to avoid an external dependency.
A production gateway platform owns authentication, virtual keys, tenant budgets, distributed rate limits, routing state, retries, streaming, audit logs, secrets, dashboards, migrations, and high availability. Rebuilding all of that is rarely a small project.
Custom work is easier to justify when at least one of these is true:
- routing logic is proprietary and directly affects product quality or margin;
- compliance requires a data path or approval workflow commercial gateways cannot provide;
- direct provider contracts create meaningful savings at your traffic level;
- an existing platform team can operate the service without creating a one-person dependency;
- the company needs one internal AI control plane across many products and business units.
If the requirement is merely “change model names without touching business logic,” build a narrow adapter or use an established gateway. Do not accidentally create a platform team because three SDKs have different method names.
Which approach fits which product?
| Situation | Practical starting point | Why |
|---|---|---|
| Prototype or side project | OpenRouter or Vercel AI Gateway | The cost of operating infrastructure is usually larger than the gateway fee or platform dependency. |
| Early SaaS using several providers | OpenRouter, Vercel, or Cloudflare Unified Billing | One balance and centralized usage reduce administrative work while the architecture is still changing. |
| Growing product with existing provider accounts | Vercel BYOK, Cloudflare BYOK, or LiteLLM | Direct contracts and rate limits start to matter; choose whether the gateway should remain hosted. |
| Regulated or data-sensitive system | Self-hosted LiteLLM or a carefully contracted hosted gateway with explicit retention policies | The decision depends on the complete data path, logs, provider terms, regions, and operational capability - not the word “self-hosted” alone. |
| High-volume platform with negotiated provider spend | LiteLLM or a focused internal gateway layer | Avoiding a percentage fee may fund the infrastructure, but only if the team can operate it reliably. |
| Team already committed to Vercel or Cloudflare | Evaluate that platform’s gateway first | Existing identity, procurement, observability, and support relationships can be more valuable than a marginal feature difference. |
Before choosing, ask six questions:
- Do we need one endpoint, one supplier, or both?
- Are prepaid credits acceptable to finance, or do we require postpaid invoices and purchase orders?
- Do we already have provider discounts or committed spend?
- Who will patch and support a self-hosted gateway at 3 a.m.?
- Which companies may process or log prompts and responses?
- Which provider-specific features must remain available when we switch models?
The answers usually eliminate more options than a feature checklist does.
Decide what you want to own
A free intermediary does not exist. There is only a choice about what you pay with.
OpenRouter asks for a percentage in exchange for the broadest catalogue in this comparison and very little setup. LiteLLM removes the per-token gateway fee but returns infrastructure, provider accounts, and separate bills to your team. Vercel currently passes through model prices without a token markup, funded within a larger platform relationship. Cloudflare lets you choose between free BYOK gateway features and Unified Billing with a 5% credit-purchase fee.
The cheapest line item is not automatically the cheapest system. A 5% fee can be excellent value if it saves days of engineering and months of accounting friction. It can also become unnecessary spend when traffic reaches hundreds of thousands of dollars and the company already has a platform team and direct contracts.
Start with the problem you actually need to remove: integration work, outages, fragmented usage, or fragmented billing. Then choose the gateway whose remaining cost you are prepared to own.