Run your GPT-4 securely in Azure using Azure OpenAI Service

Running the latest GPT-4 models in a controlled, secure, and governed environment is mission-critical when we think about real-world scenarios where we want to leverage the capabilities of OpenAI but hide our service instance from the public internet.

This post demonstrates how to deploy the Azure OpenAI service and a GPT-4 model into a private network infrastructure using an Azure Private Endpoint, individual private DNS records, and restrict access to the Azure OpenAI Service using Network Security Groups. We will use an Azure Container App for demonstration purposes, which acts as a frontend to access our GPT-4 model.

For managing the infrastructure, we’ll use HashiCorp Terraform. You can find the entire source code in the corresponding GitHub repository at https://github.com/thinktecture-labs/azure-openai-virtual-network-integration

Azure OpenAI and GPT-4 model deployment with Terraform

We can describe Azure OpenAI using the existing azurerm_cognitive_account resource provided by the AzureRM provider for Terraform.

resource "azurerm_cognitive_account" "openai" 
  name                  = "azopenai-$var.suffix"
  location              = data.azurerm_resource_group.rg.location
  resource_group_name   = data.azurerm_resource_group.rg.name
  kind                  = "OpenAI"
  sku_name              = "S0"
  custom_subdomain_name = "azopenai$var.suffix"
  public_network_access_enabled = false
}
Code language: C# (cs)

At the point of writing this article, GPT-4 is available in a private preview and limited to certain Azure Regions. You should check the corresponding page of the Azure documentation to verify if GPT-4 is available in your region of choice. 

To deploy the GPT-4 model, we use the azurerm_cognitive_deployment resource as shown in the following snippet:

resource "azurerm_cognitive_deployment" "gpt4" {
  name                 = "mygpt4"
  cognitive_account_id = azurerm_cognitive_account.openai.id
  model {
    format  = "OpenAI"
    name    = "gpt-4"
    version = "0314"
  }

  scale {
    type = "Standard"
  }
}
Code language: TypeScript (typescript)

If you want to deploy other LLMs, such as GPT-3.5 Turbo, you can use the azurerm_cognitive_deployment with corresponding property values.

Create a Private Endpoint for Azure OpenAI with Terraform

So far, we have seen how to describe Azure OpenAI and the GPT-4 model deployment using Terraform. By setting public_network_access_enabled to false, we’ve already disabled public network access to our service. However, we want to integrate the service instance with a private network environment. To do so, we create a Private Endpoint and deploy it into the backend subnet of our Azure Virtual Network.

resource "azurerm_private_endpoint" "endpoint" {
  name                = "pe-azopenai-${var.suffix}"
  location            = data.azurerm_resource_group.rg.location
  resource_group_name = data.azurerm_resource_group.rg.name
  subnet_id           = azurerm_subnet.backend.id

  private_service_connection {
    name                           = "psc-azopenai"
    private_connection_resource_id = azurerm_cognitive_account.openai.id
    is_manual_connection           = false
    subresource_names              = ["account"]
  }
}
Code language: C# (cs)

Finally, we must take care of the name resolution (DNS). Calls from our Azure Container App should resolve the FQDN of our Azure OpenAI instance to the private IP address associated with the Private Endpoint we looked at a few seconds ago.

To do so, we create a private DNS zone for privatelink.openai.azure.com with an A record that resolves our individual service URL to the private IP address of our Private Endpoint.

resource "azurerm_private_dns_zone" "dns_zone" {
  name                = "privatelink.openai.azure.com"
  resource_group_name = data.azurerm_resource_group.rg.name
}

resource "azurerm_private_dns_a_record" "dns_a" {
  name                = azurerm_cognitive_account.openai.custom_subdomain_name
  zone_name           = azurerm_private_dns_zone.dns_zone.name
  resource_group_name = data.azurerm_resource_group.rg.name
  ttl                 = 10
  records             = [azurerm_private_endpoint.endpoint.private_service_connection.0.private_ip_address]
}
Code language: Python (python)

Lastly, we link the Private DNS Zone to our Virtual Network.

resource "azurerm_private_dns_zone_virtual_network_link" "network_link" {
  name                  = "vnet-link"
  resource_group_name   = data.azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.dns_zone.name
  virtual_network_id    = azurerm_virtual_network.vnet.id
}
Code language: Python (python)

Restrict access to Azure OpenAI with an Azure Network Security Group

Our sample application will also be deployed to Azure Container Apps (which are integrated with the Virtual Network infrastructure and resist in the apps subnet).

We want to allow only connections from this subnet to access Azure OpenAI sitting behind the private endpoint in the backend subnet.

That being said, we will first create a new azurerm_network_security_group that comes with two security rules:

1. The prevent-all rule: This rule will turn off all TCP inbound traffic for the backend subnet with  destination port 443 
2. The allow-apps-to-openai-https rule: This rule allows inbound TCP traffic from the apps subnet to the IP address of our Private Endpoint (sitting in the backend subnet) with destination port 443

resource "azurerm_network_security_group" "backend" {
  name                = "nsg-backend"
  resource_group_name = data.azurerm_resource_group.rg.name
  location            = data.azurerm_resource_group.rg.location

  security_rule {
    name        = "allow-apps-to-openai-https"
    description = "Allow HTTPS from the apps subnet"
    access      = "Allow"
    direction   = "Inbound"
    priority    = 150
    protocol    = "Tcp"

    source_address_prefix = azurerm_subnet.apps.address_prefixes[0]
    source_port_range     = "*"

    destination_port_range     = "443"
    destination_address_prefix = "${azurerm_private_endpoint.endpoint.private_service_connection.0.private_ip_address}/32"
  }

  security_rule {
    name        = "prevent-all"
    description = "Deny all inbound traffic"
    direction   = "Inbound"
    access      = "Deny"
    priority    = 200
    protocol    = "Tcp"

    source_address_prefix = "*"
    source_port_range     = "*"

    destination_port_range     = "443"
    destination_address_prefix = azurerm_subnet.backend.address_prefixes[0]
  }
}
Code language: Python (python)

---

Finally, we must associate the Network Security Group (NSG) with our backend subnet. We do so by using an azurerm_subnet_network_security_group_association resource:

resource "azurerm_subnet_network_security_group_association" "backend" 
  subnet_id                 = azurerm_subnet.backend.id
  network_security_group_id = azurerm_network_security_group.backend.id
}
Code language: Python (python)

Deploying the infrastructure with Terraform

The sample repository contains everything you need to provision the entire infrastructure and a sample application.

There are two Terraform projects in the repository:

• global: This project will deploy an Azure Resource Group and an Azure Container Registry (ACR)
• workload: This project will deploy the infrastructure we looked at in this article, including the Azure Container App (ACA)

To streamline the entire deployment process, you can use the deploy.sh, located in the repository’s root directory.
The script will

1. Deploy the global infrastructure
2. Build the container image for the sample application and persist it in the Azure Container Registry (ACR)
3. Deploy the workload infrastructure, including the deployment of the container image built in Step 2 to Azure Container Apps (ACA)

---

Conclusion

As shown in this article, we can integrate Azure OpenAI in existing or new private network infrastructures and use proven techniques like Private Endpoints and Network Security Groups to control who and how the service can be accessed.

Because Azure OpenAI is built on top of existing ARM entities, we can provision and maintain Azure OpenAI and model deployments, such as GPT-4, without introducing new tools or techniques.