Get in touch
I'm always excited to take on new projects and collaborate with innovative minds.
Phone
I'm always excited to take on new projects and collaborate with innovative minds.
Most AI applications are tightly coupled to a single provider. This guide shows how to build a production-ready AI Gateway in ASP.NET Core that supports OpenAI, Claude, Gemini, and Ollama through a unified abstraction layer, enabling provider switching, fallback strategies, cost optimization, and better maintainability.
Most AI tutorials assume a single provider.
Production systems don't.
Different models have different strengths.
GPT-4o is excellent for reasoning and code generation.
Claude performs exceptionally well with large documents and long-context analysis.
Gemini offers strong multimodal capabilities and cost-effective models for high-volume workloads.
Ollama enables local inference and offline development.
The challenge isn't choosing the perfect provider.
The challenge is building an application that can switch providers without rewriting business logic.
If your controllers directly depend on OpenAI, you've already introduced vendor lock-in.
In this article, we'll build an AI Gateway in ASP.NET Core that abstracts multiple providers behind a common interface.
The result is a flexible architecture that allows provider changes through configuration rather than code changes.
Our architecture looks like this:
Client
│
ASP.NET Core API
│
AI Gateway
┌────────┬─────────┬─────────┐
│ │ │ │
OpenAI Claude Gemini Ollama
│ │ │ │
GPT-4o Sonnet 2.0 Flash Llama 3.2
Instead of coupling application logic to a specific provider, the gateway becomes the only component responsible for provider selection and communication.
Everything else remains provider-agnostic.
Many projects start like this:
Controller
↓
OpenAI SDK
It works.
Until requirements change.
Consider these scenarios:
Suddenly the application must support multiple providers.
If provider-specific code exists throughout the codebase, the migration becomes expensive.
A better design looks like this:
Controller
↓
IAIProvider
↓
Provider Factory
↓
OpenAI
Claude
Gemini
Ollama
Now provider selection becomes an implementation detail.
Controllers remain unchanged.
Business logic remains unchanged.
Only the gateway evolves.
Every provider must expose the same interface.
public interface IAIProvider
{
Task<ChatResponse> GenerateAsync(
ChatRequest request);
}
This is the most important architectural decision in the entire solution.
The interface defines what the application needs.
Not how the provider works.
OpenAI, Claude, Gemini, and Ollama all have different APIs.
They all return different payloads.
They all expose different features.
The gateway hides those differences.
The rest of the application interacts with a single contract.
Without a common interface:
Controller
↓
OpenAI Logic
Claude Logic
Gemini Logic
Ollama Logic
The controller becomes responsible for provider management.
This violates separation of concerns.
With a shared interface:
Controller
↓
IAIProvider
The controller doesn't know which provider generated the response.
And it shouldn't.
This follows the Dependency Inversion Principle and keeps the system maintainable.
Once multiple providers exist, something must decide which provider to use.
This is where the Factory Pattern fits naturally.
Request
↓
Provider Factory
↓
Selected Provider
The factory resolves the correct implementation based on configuration or business rules.
Provider selection is rarely random.
Examples:
Default Provider
↓
OpenAI
Useful for most applications.
Premium User
↓
GPT-4o
Free User
↓
Gemini Flash
This balances quality and cost.
Development Environment
↓
Ollama
No API costs.
No internet dependency.
Excellent for local development.
Document Analysis
↓
Claude Sonnet
Provider selection becomes a business decision rather than a technical limitation.
Hardcoded provider selection is another form of lock-in.
Avoid:
new OpenAIProvider()
Instead:
{
"AI": {
"DefaultProvider": "OpenAI"
}
}
The active provider becomes configuration.
Switching providers becomes a deployment decision rather than a development project.
Register every provider with ASP.NET Core.
IAIProvider
↓
OpenAIProvider
ClaudeProvider
GeminiProvider
OllamaProvider
The factory receives provider implementations through Dependency Injection.
Benefits include:
Adding a new provider should require registration, not refactoring.
One of the biggest challenges is response normalization.
Each provider returns different structures.
OpenAI:
{
"choices": [...]
}
Claude:
{
"content": [...]
}
Gemini:
{
"candidates": [...]
}
Ollama:
{
"response": "..."
}
The rest of the application shouldn't care.
Create a common response model.
public class ChatResponse
{
public string Answer { get; set; }
public Usage Usage { get; set; }
public string FinishReason { get; set; }
public string Provider { get; set; }
public long Latency { get; set; }
}
Now every provider returns the same shape.
This simplifies the entire application.
Provider APIs fail in different ways.
Common failures include:
429 Too Many Requests
401 Unauthorized
Request exceeded timeout
Selected model unavailable
The gateway should normalize these failures into consistent application exceptions.
Clients should never need provider-specific error handling.
Provider abstraction enables intelligent failover.
Example:
OpenAI
↓
Failure
↓
Retry
↓
Claude
↓
Failure
↓
Gemini
↓
Success
This can dramatically improve reliability.
However, fallback is not always appropriate.
Examples:
A different provider may produce acceptable results.
Examples:
Different providers may generate different answers.
Consistency may be more important than availability.
Fallback should always be a conscious business decision.
Provider abstraction creates a unique opportunity.
We can compare providers objectively.
Track:
Provider
Model
Prompt Tokens
Completion Tokens
Cost
Latency
Success
Failure
Over time this data becomes extremely valuable.
Questions become easy to answer:
Without metrics, provider selection becomes guesswork.
Not every request requires the most expensive model.
Smart routing can dramatically reduce costs.
| Task | Recommended Model |
|---|---|
| Summarization | Gemini Flash |
| Code Generation | GPT-4o |
| Long Documents | Claude Sonnet |
| Offline Development | Ollama |
This is where an AI Gateway starts creating business value.
Instead of selecting one provider globally, the application routes requests to the most appropriate model.
Better results.
Lower costs.
Greater flexibility.
An AI Gateway centralizes provider management.
That also makes it the ideal place to enforce security.
Never hardcode API keys.
Use:
Configuration files should not contain production credentials.
Prevent arbitrary model execution.
Only allow approved models.
Example:
GPT-4o
Claude Sonnet
Gemini Flash
Llama 3.2
Reject everything else.
Some providers may be reserved for specific workloads.
The gateway should enforce authorization policies.
Different providers have different quotas.
Independent rate limiting prevents one provider from affecting others.
This is one of the most overlooked benefits.
Instead of calling external APIs:
OpenAIProvider
can be replaced with:
FakeProvider
during tests.
Unit tests become:
Business logic can be validated without consuming tokens.
These mistakes appear frequently in production systems.
Creates vendor lock-in.
Makes upgrades difficult.
Creates maintenance nightmares.
Prevents future flexibility.
Reduces reliability.
Makes optimization impossible.
The reference implementation includes:
The objective is not merely to support multiple providers.
The objective is to make provider selection a business decision rather than an architectural constraint.
Include the following visuals.
Show the gateway routing requests to multiple providers.
Demonstrate:
POST /api/chat
Visualize:
Request
↓
Factory
↓
Selected Provider
Compare normalized responses from different providers.
Show:
Provider
Latency
Tokens
Cost
Status
These screenshots help communicate the operational benefits of the gateway.
Building AI applications around a single provider is easy.
Building systems that can evolve as providers, models, pricing, and business requirements change is significantly harder.
An AI Gateway introduces a layer of abstraction that protects the rest of the application from provider-specific concerns.
By standardizing contracts, centralizing provider selection, normalizing responses, implementing fallback strategies, and collecting operational metrics, you create a system that remains flexible as the AI ecosystem evolves.
Switching providers solves flexibility, but modern AI applications need more than text generation—they need the ability to interact with external systems.
In the next article, we'll explore Function Calling in .NET and show how AI can safely invoke tools, APIs, and business logic.
Your email address will not be published. Required fields are marked *