The OpenAI GPT models can achieve a lot and integrating their capabilities into your own applications can be a huge gamechanger.
That also means that your application must be able to communicate with the OpenAI API either from OpenAI directly or on Azure, and that means that your application must send your API key along. This might be fine for a desktop or mobile app, but when you have a single page application running in the browser, like Angular, React, Vue or Blazor WebAssembly, you really don’t want to expose your API key to the world. But even in the cases where you have a controlled environment, you still might want to keep the API key on the server side at a single point to be able to exchange it when the need arises without redeploying all applications.
This is where a proxy comes in handy, and in this article I want to show how to easily set up such an environment and explain the subtle differences between the official API directly from OpenAI and the one you can deploy on your Azure OpenAI resource.
The server side Proxy
As our main server-side technology stack is .NET, we want to use an ASP.NET Core Web API to host our proxy. The proxy itself leverages the Yarp library.
In this example, I completely realy on the the appsettings.json configuration of Yarp, but it is of course possible to configure the proxy programmatically and also use dynamic values for the settings, i.e. choose an API Key based on the user or tenant, for indivial billing.
Differences between OpenAI and Azure OpenAI
There a mainly two subtle differences between the OpenAI API and the Azure OpenAI API.
First of all, the paths are different. The OpenAI API uses the path /v{API_VERSION}/chat/completions while the Azure OpenAI API uses /openai/deployments/{DEPLOYMENT_NAME}/chat/completions?api-version={API_VERSION}. Usually, the client SDK you use to access the service will take care of this, but when you use a proxy, you need to know that there are differences and how to correctly transform the paths.
The second one is even more subtle. OpenAI wants the API Key in the Authorization header, while Azure OpenAI wants it in the Api-Key header. Since the client will not pass the API Key along, as we specifically want to hide it from the client, we need to make sure to set the correct header for the service in our Proxy.
It might be likely that you only use either OpenAI or Azure OpenAI and do not regularly switch between the two, however in my example I will enable both approaches. You can either take the code as is to be able to use both, or only use one configuration based on the service you want.
The Code
First of all, we follow the Getting Started guide from Yarp: We install the NuGet package Yarp.ReverseProxy and add the following code to our Startup.cs:
var builder - WebApplication.CreateBuilder(args);
// ... Set up Services ...
builder.Services
.AddReverseProxy()
.LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));
// ...
var app = builder.Build();
// ... Set up request pipeline ...
app.MapReverseProxy();
// ...
app.Run();
Code language: C# (cs)And this already brings us to our configuration in appsettings.json. We distinguish the service based on the base path, so when we access /OpenAI we go to OpenAI and when we use /AzureOpenAI we use the Azure service.
"ReverseProxy": {
"Routes": {
"openai": {
"ClusterId": "OpenAI",
"Match": {
"Path": "/openai/{**remainder}"
},
"Transforms": [
{ "PathRemovePrefix": "/openai" },
{ // OpenAI wants the API Key in the Authorization header like a Bearer token
"RequestHeader": "Authorization",
"Set": "Bearer {YOUR OPENAI API KEY}"
}
]
},
"azureopenai": {
"ClusterId": "AzureOpenAI",
"Match": {
"Path": "/azureopenai/{**remainder}"
},
"Transforms": [
{ "PathPattern": "/{**remainder}" },
{ // Azure OpenAI wants the API Key in the Api-Key header
"RequestHeader": "Api-Key",
"Set": "{YOUR AZURE OPENAI API KEY}"
}
]
}
},
"Clusters": {
"OpenAI": {
"Destinations": {
"openai": {
"Address": "https://api.openai.com/"
}
}
},
"AzureOpenAI": {
"Destinations": {
"azureopenai": {
// Azure OpenAI uses a custom domain name for each service deployment, so
// you need to adjust the address here
"Address": "https://{YOUR CUSTOM NAME}.openai.azure.com/"
}
}
}
}
}
Code language: C# (cs)The Client
For this example I use the Semantic Kernel SDK from Microsoft to use the AI services in my Blazor WebAssembly application.
Configuration
By default, the SDK can talk to both services but the Url’s are hard-coded and not easy to exchange. Also, while there is the property BaseUrl of the Semantic Kernel configuration that you can change, and it works for Azure OpenAI, it does not work with OpenAI directly because there is no way to pass the Url to the OpenAI connector.
So we need to extend the configuration a bit to be able to switch between using the services directly and going through our Yarp proxy. First of all, I extend the SemanticKernelOptions class to add new properties to it:
public class ProxyOpenAIOptions : OpenAIOptions
{
public bool IsOpenAI { get; set; }
public bool UseProxy { get; set; }
public string ProxyAddress { get; set; }
}
Code language: C# (cs)We could use the BaseUrl property of the OpenAIOptions to distinguish between the services, like the samples of Semtantic Kernel do, but I want to be a bit more explicit about it to have better readable code. Also, one could argue that setting the ProxyAddress would be enough and omit the UseProxy property, but since I want to determine the proxy address dynamically based on the base url of my Blazor WASM application, I also want to be explicit about the fact that I want to use a proxy.
Then, we need to apply the configuration. In our Program.cs I add the following code:
builder.Services.Configure<ProxyOpenAIOptions>(o => {
// In this example, I have one setting to switch beween OpenAI and Azure OpenAI
var service = builder.Configuration.GetValue<string>("AIService");
// We bind the section based on the switch above
builder.Configuration.GetSection(service).Bind(o);
if (o.UseProxy)
{
o.ProxyAddress = builder.HostEnvironment.BaseAddress
+ (builder.HostEnvironment.BaseAddress.EndsWith("/") ? String.Empty : "/")
+ service + "/";
}
});
Code language: TypeScript (typescript)If the UseProxy setting is there, we set the ProxyAddress to the base address of our Blazor WASM application plus the service name, i.e. OpenAI or AzureOpenAI. This is the path we configured in our Yarp proxy.
So our configuration appsettings.json from our Blazor WebAssembly application looks like this:
{
"AIService": "OpenAI", // here you can switch between the services
"OpenAI": {
"Model": "gpt-4",
"ApiKey": "FooBar", // Must not be empty, can be anything
"UseProxy": true,
"IsOpenAI": true
},
"AzureOpenAI": {
"Model": "gpt-4-32k-auto-update", // This is the DEPLOYMENT_NAME
"ApiKey": "FooBar", // Must not be empty, can be anything
"BaseUrl": "https://{YOUR_CUSTOM_NAME}.openai.azure.com/", // the library needs a value here, when using a proxy it does not need to be a valid domain
"UseProxy": true,
"IsOpenAI": false
}
}
Code language: C# (cs)Usage
When we build the Semantic Kernel, we add a service connector for OpenAI or Azure OpenAI based on the configuration. This connector either uses an internal http client or a custom http client that we can provide. In our case, we use a normal http client for direct access to the services and a custom http client when we use the proxy.
Remember that we are in a Blazor WebAssembly client application here, so the usage of the HttpClient with new is fine here as it only wraps the browser’s fetch API. If you use the Semantic Kernel in a server-side application, you should follow best practices on HttpClient usage. For that you could configure a named http client based on the configuration and use the HttpClientFactory to get a re-usable http client for each Kernel you build.
private readonly ProxyOpenAIOptions _options;
private readonly ILogger _logger;
/// ...
private IKernel BuildKernel()
{
var builder = new KernelBuilder()
.WithLogger(_logger);
// create a custom http client that uses the proxy if need be
var httpClient = _options.UseProxy
? new HttpClient(new ProxyClientHandler(new Uri(_options.ProxyAddress)))
: new HttpClient();
// Add the configured service connector with our prepared http client (proxy or not)
if (_options.IsOpenAI)
{
builder.WithOpenAIChatCompletionService(_options.Model, _options.ApiKey, httpClient: httpClient);
}
else
{
builder.WithAzureChatCompletionService(_options.Model, _options.BaseUrl!, _options.ApiKey, httpClient: httpClient);
}
var kernel = builder.Build();
// ... Add your plugins here ...
return kernel;
}
Code language: C# (cs)The missing piece is the ProxyClientHandler class, which will modify the request Url to point to the proxy:
public class ProxyClientHandler : HttpClientHandler
{
private Uri _baseUri;
public ProxyClientHandler(Uri baseUri)
{
_baseUri = baseUri;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (request.RequestUri is not null)
{
// Make path and query relative and append to base url
var path = request.RequestUri.PathAndQuery;
if (path.StartsWith("/")) path = path[1..];
request.RequestUri = new Uri(_baseUri, path);
}
return base.SendAsync(request, cancellationToken);
}
}
Code language: C# (cs)Summary
We talked about why it might be a good idea to hide your OpenAI or Azure OpenAI API Keys from the client application. For that we can pipe the requests to the AI service though a simple proxy that will sneak in the API Key on the server side.
In this example I showed how to use the Yarp reverse proxy to do that.
We also added a bit of code to a Blazor WebAssembly application and use the Microsoft Semantic Kernel SDK to consume the AI Models through our proxy.
With a setup like this, we can easily switch between using the services directly and going through the proxy by just changing the configuration. This is especially useful when you want to use the proxy only in certain environments, e.g. in production, but every developer has its own API key and can directly access the service locally.
