Thinktecture Logo

Run your GPT-4 securely in Azure using Azure OpenAI Service

Author: Thorsten Hans • Published: 19.07.2023 • Category: AI, Azure OpenAI, OpenAI

Running the latest GPT-4 models in a controlled, secure, and governed environment is mission-critical when we think about real-world scenarios where we want to leverage the capabilities of OpenAI but hide our service instance from the public internet.

This post demonstrates how to deploy the Azure OpenAI service and a GPT-4 model into a private network infrastructure using an Azure Private Endpoint, individual private DNS records, and restrict access to the Azure OpenAI Service using Network Security Groups. We will use an Azure Container App for demonstration purposes, which acts as a frontend to access our GPT-4 model.

Azure OpenAI VNet Integration

For managing the infrastructure, we’ll use HashiCorp Terraform. You can find the entire source code in the corresponding GitHub repository at

Azure OpenAI and GPT-4 model deployment with Terraform

We can describe Azure OpenAI using the existing azurerm_cognitive_account resource provided by the AzureRM provider for Terraform.

resource "azurerm_cognitive_account" "openai" 
  name                  = "azopenai-$var.suffix"
  location              = data.azurerm_resource_group.rg.location
  resource_group_name   =
  kind                  = "OpenAI"
  sku_name              = "S0"
  custom_subdomain_name = "azopenai$var.suffix"
  public_network_access_enabled = false
Code language: C# (cs)

At the point of writing this article, GPT-4 is available in a private preview and limited to certain Azure Regions. You should check the corresponding page of the Azure documentation to verify if GPT-4 is available in your region of choice. 

To deploy the GPT-4 model, we use the azurerm_cognitive_deployment resource as shown in the following snippet:

resource "azurerm_cognitive_deployment" "gpt4" {
  name                 = "mygpt4"
  cognitive_account_id =
  model {
    format  = "OpenAI"
    name    = "gpt-4"
    version = "0314"

  scale {
    type = "Standard"
Code language: TypeScript (typescript)

If you want to deploy other LLMs, such as GPT-3.5 Turbo, you can use the azurerm_cognitive_deployment with corresponding property values.

Create a Private Endpoint for Azure OpenAI with Terraform

So far, we have seen how to describe Azure OpenAI and the GPT-4 model deployment using Terraform. By setting public_network_access_enabled to false, we’ve already disabled public network access to our service. However, we want to integrate the service instance with a private network environment. To do so, we create a Private Endpoint and deploy it into the backend subnet of our Azure Virtual Network.

resource "azurerm_private_endpoint" "endpoint" {
  name                = "pe-azopenai-${var.suffix}"
  location            = data.azurerm_resource_group.rg.location
  resource_group_name =
  subnet_id           =

  private_service_connection {
    name                           = "psc-azopenai"
    private_connection_resource_id =
    is_manual_connection           = false
    subresource_names              = ["account"]
Code language: C# (cs)

Finally, we must take care of the name resolution (DNS). Calls from our Azure Container App should resolve the FQDN of our Azure OpenAI instance to the private IP address associated with the Private Endpoint we looked at a few seconds ago.

To do so, we create a private DNS zone for with an A record that resolves our individual service URL to the private IP address of our Private Endpoint.

resource "azurerm_private_dns_zone" "dns_zone" {
  name                = ""
  resource_group_name =

resource "azurerm_private_dns_a_record" "dns_a" {
  name                = azurerm_cognitive_account.openai.custom_subdomain_name
  zone_name           =
  resource_group_name =
  ttl                 = 10
  records             = [azurerm_private_endpoint.endpoint.private_service_connection.0.private_ip_address]
Code language: Python (python)

Lastly, we link the Private DNS Zone to our Virtual Network.

resource "azurerm_private_dns_zone_virtual_network_link" "network_link" {
  name                  = "vnet-link"
  resource_group_name   =
  private_dns_zone_name =
  virtual_network_id    =
Code language: Python (python)

Restrict access to Azure OpenAI with an Azure Network Security Group

Our sample application will also be deployed to Azure Container Apps (which are integrated with the Virtual Network infrastructure and resist in the apps subnet).

We want to allow only connections from this subnet to access Azure OpenAI sitting behind the private endpoint in the backend subnet.

That being said, we will first create a new azurerm_network_security_group that comes with two security rules:

  1. The prevent-all rule: This rule will turn off all TCP inbound traffic for the backend subnet with  destination port 443 
  2. The allow-apps-to-openai-https rule: This rule allows inbound TCP traffic from the apps subnet to the IP address of our Private Endpoint (sitting in the backend subnet) with destination port 443
resource "azurerm_network_security_group" "backend" {
  name                = "nsg-backend"
  resource_group_name =
  location            = data.azurerm_resource_group.rg.location

  security_rule {
    name        = "allow-apps-to-openai-https"
    description = "Allow HTTPS from the apps subnet"
    access      = "Allow"
    direction   = "Inbound"
    priority    = 150
    protocol    = "Tcp"

    source_address_prefix = azurerm_subnet.apps.address_prefixes[0]
    source_port_range     = "*"

    destination_port_range     = "443"
    destination_address_prefix = "${azurerm_private_endpoint.endpoint.private_service_connection.0.private_ip_address}/32"

  security_rule {
    name        = "prevent-all"
    description = "Deny all inbound traffic"
    direction   = "Inbound"
    access      = "Deny"
    priority    = 200
    protocol    = "Tcp"

    source_address_prefix = "*"
    source_port_range     = "*"

    destination_port_range     = "443"
    destination_address_prefix = azurerm_subnet.backend.address_prefixes[0]
Code language: Python (python)

Finally, we must associate the Network Security Group (NSG) with our backend subnet. We do so by using an azurerm_subnet_network_security_group_association resource:

resource "azurerm_subnet_network_security_group_association" "backend" 
  subnet_id                 =
  network_security_group_id =
Code language: Python (python)

Deploying the infrastructure with Terraform

The sample repository contains everything you need to provision the entire infrastructure and a sample application.

There are two Terraform projects in the repository:

  • global: This project will deploy an Azure Resource Group and an Azure Container Registry (ACR)
  • workload: This project will deploy the infrastructure we looked at in this article, including the Azure Container App (ACA)

To streamline the entire deployment process, you can use the, located in the repository’s root directory.
The script will

  1. Deploy the global infrastructure
  2. Build the container image for the sample application and persist it in the Azure Container Registry (ACR)
  3. Deploy the workload infrastructure, including the deployment of the container image built in Step 2 to Azure Container Apps (ACA)


As shown in this article, we can integrate Azure OpenAI in existing or new private network infrastructures and use proven techniques like Private Endpoints and Network Security Groups to control who and how the service can be accessed.

Because Azure OpenAI is built on top of existing ARM entities, we can provision and maintain Azure OpenAI and model deployments, such as GPT-4, without introducing new tools or techniques.

Aktuelle Research-Insights unserer Experten für Sie

Lesen Sie, was unsere Experten bei ihrem Research bewegt und melden Sie sich zu unserem kostenlosen Thinktecture Labs-Newsletter an.

Labs-Newsletter Anmeldung

Thorsten Hans

I am a passionate software developer and started my professional career more than 20 years back now. Through the years, I have learned, used, and mastered several technologies. It is that passion that lets me share my experience with others as part of my daily job. Here at Thinktecture, I work as a Consultant. I help customers to leverage the power of container technologies and Microsoft Azure to either move their existing application into the cloud or to build their new service on a large scale. Due to my community contributions, I am awarded as Microsoft MVP (Azure) and Docker Captain. If you have any questions, feel free to reach out and contact me at You can also follow me on Twitter at @ThorstenHans and consult my blog at

More about me →