Secure Azure Open AI using Azure Front Door WAF

 8 min to read.

Abstract

Azure Open AI adoption for application innovation is in full speed. With the same speed security teams are analyzing and doing VAPT on Azure Open AI endpoints. Recently a customer’s security team gave big list of 30+ vulnerabilities for their Azure Open AI endpoint.

So obvious question from application team “what can we do for security of Azure Open AI?”

This article step by step guide is for using Azure Front Door Web Application Firewall (WAF) with Azure Open AI.

The Challenge

In my opinion performing VAPT on direct open AI endpoint and calling it unsecure is incorrect approach. If an application team develops and host APIs on say Azure App Service/ AKS/ ARO/ Container Apps; it will also have similar vulnerabilities. Because these are plain APIs and not security appliances. We need to use Security appliances in front of APIs to secure them.

This is similar type of scenario when you create Azure Open AI service in your Azure subscription. Therefore we need to address below security aspects for Azure Open AI endpoint –

       1.      OWASP common vulnerabilities protection
2.      Azure Open AI endpoint should not be hit directly from end client applications like web app/ mobile app.
3.      If possible, remove public access to Open AI endpoint completely.

Solution

The standard approach to protect Azure Open AI APIs will be to use Web Application Firewall (WAF).

WAF on Azure Native technology offering is either Application Gateway or Azure Front Door. I will use Azure Front Door in this article.

Lets start.

Create Azure Open AI Service

Please don’t ask “how do I get access to Open AI in my Azure subscription?”. It is not in my control and I can’t even influence it. Follow the general procedure mentioned here.

Once you have access to Azure Open AI follow the below procedure to create it.

How-to:Create and deploy an Azure OpenAI Service resource - Azure OpenAI | MicrosoftLearn

From this link make sure that you choose “option 1: Allow all networks”. Don’t follow “specific networks only” and “disable network access” scenario for now.

Now Azure Open AI service instance is created. Next step will be to create deployment. Click on “Create New Deployment -> Add details as shown with model selected as gpt-35-turbo. Click on Create to complete model deployment.

Once model is created, select it and click on “Open in Playground”. Then click on “View Code” under “Chat Session” section. A pop up will appear showing Azure Open AI endpoint, and an access key as shown below –

Lets try Azure Open AI endpoint in Postman and get the output. To run in postman I used below details –

Body added in postman is as follows –

{

    "messages": [

        {

            "role": "system",

            "content": "You are an AI assistant that helps people find information."

        },

        {

            "role": "user",

            "content": "What is Azure OpenAI?"

        }

    ]

}

Rest of the setting with header in Postman as below. As you can see the outcome is 200 successful response.

Here we have used Azure Open AI endpoint directly in postman. As a next step lets create Azure Front Door and configure Azure Open AI as a backend to Azure Front Door.

Create Azure Front Door Service

On Azure portal click on Create -> In Search box type FrontDoor and Select FrontDoor Service creation option. Then on offerings page select options “Azure Front Door” and “Custom Create” as shown below – [click on image to get better view].

On Profile page under Basic tab select tier as “premium” and Resource group, service name as per your choice. Refer details as below. I could have chosen Standard tier however later in the future I am going to show AFD + Azure OpenAI with private endpoint which will be supported in Premium tier of AFD. [click on image to get better view].

Keep Secrets tab empty for now. On “Endpoint” tab click on “ Add an endpoint” button. Provide name as you wish. The click on Add. Refer to below screenshot – [click on image to get better view].

Then click on “Add a route” option in “Endpoint” tab. Add name as “myRoute”. Then click on “Add a new Origin Group”. Name it as “myOriginGroup”. Then click on “Add a Origin” button as shown below –

On the add Origin screen, I selected origin type as “custom” as Azure OpenAI does not appear inside the Azure front door origin types list. As of now I have not enabled the private endpoint for Azure Open AI. Refer to the screenshot below for rest of the inputs. Then click on “add” to finalize on Origin creation.

You will be redirected back to Origin group screen. Here the screen asks about enabling health probe. In Azure Front Door Health Probes can’t work with Authentication, however Azure Open AI endpoint API need API Key header for authentication. Also my intent is to show how we can WAF protection for Open AI. There are other ways by which you can check Azure Open AI endpoint health status. Therefore here I will disable the Health Probe. Click on Add button to finish Origin group creation.

This will bring back to add route screen. Keep Origin Path blank, Forwarding protocol to “match incoming requests”, Caching option is unchecked. Then click on add to finish route adding.

On the same page click on “Add Policy” button. Provide the name as “myPolicy”, select Azure OpenAI domain listed under “Domains” dropdown. Under WAF Policy option, click on Create New and name it as MyWAFPolicy. Check the option “add bot protection”. Finally click on Save to complete WAF policy creation.

Then click on “review and Create” button to finish Azure front door service creation process.

Output

Now we have Azure Front Door WAF ready with Azure Open AI endpoint as backend. Lets try to access Open AI endpoint using Azure Front Door URL via Postman.

My original Azure Open AI Endpoint is  - https://kunalgenai.openai.azure.com/openai/deployments/kunaldeploy1/chat/completions?api-version=2023-03-15-preview  

With Azure Front Door it will be - https://kunalopenaiafdep-hgc2ecbybxeranac.z01.azurefd.net/openai/deployments/kunaldeploy1/chat/completions?api-version=2023-03-15-preview

Try the Azure Front Door URL in postman, with same body, same headers, same api key and I see that result is successful. Refer below screenshot –

I have not tested with Azure Open AI with private endpoint. Please test it and add your experience in comments.

Conclusion

Hope this article helped you to secure Azure Open AI endpoint using Azure Front Door (AFD) (WAF).

Happy Generative AI..ing!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

AzureVirtual Machines should be connected to allowed multiple virtual networks.

Start stop multiple Azure VMs on schedule and save cost!

Export Azure VMs to CSV!

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!

Azure Virtual machines should be connected to an approved MULTIPLE virtual networks

 8 min to read.

Abstract

Virtual machines should be connected to an approved virtual network - This default Azure policy is fantastic. This policy checks if a VM is part of approved VNET; else it shows compliance message.

However it only offers VMs to be checked against single VNET name. In reality, we have flood of Azure VNETs across multiple Azure subscriptions.

So we need a policy that can check all Azure VMs against “multiple azure Virtual Networks”.

This article talks about creating a policy that allows to provision Azure VMs inside only allowed list of VNETs.

 

The Challenge

This policy will be built as custom. I want to do below –

Evaluate every NIC against the VNET names present in input parameter. Refer yellow and green highlight below. I need to –

1.       Loop using for or for-each

2.       Dynamically get values of all items present in a parameter after performing “split” on the input string.

 

  "mode": "All",

  "policyRule": {

    "if": {

      "allOf": [

        {

          "field": "type",

          "equals": "Microsoft.Network/networkInterfaces"

        },

        {

          "not": {

            "field": "Microsoft.Network/networkInterfaces/ipconfigurations[*].subnet.id",

            "contains": "[parameters('virtualNetworkIds')]"

          }

        }

      ]

    },

    "then": {

      "effect": "[parameters('effect')]"

    }

  },

  "parameters": {

    "effect": {

      "type": "String",

      "metadata": {

        "displayName": "Effect",

        "description": "The effect determines what happens when the policy rule is evaluated to match"

      },

      "allowedValues": [

        "Audit",

        "Deny",

        "Disabled"

      ],

      "defaultValue": "Audit"

    },

    "virtualNetworkIds": {

      "type": "String",

      "metadata": {

        "displayName": "Virtual network Names",

        "description": "Resource name of the virtual network. Example: Add , separated multiple values."

      }

    }

  }

}

The input parameter to this policy will be more than one names of VNETs. Unfortunately in Azure policy I could not find a way to iterate over input parameters by using for-each loop construct.

There is a Current function available however it can be used only when we are using Count and Where function. Also these functions are used over Field property, used as an array. These functions cant be used when input parameter is not an array.

Therefore we need a way to iterate over input parameters array and comparing every parameter value with one subnet id Field highlighted as green above.

Solution

If we see the outcome of below line  - "Microsoft.Network/networkInterfaces/ipconfigurations[*].subnet.id",

It will be resolved to id format of subnet similar to below - "/subscriptions/SubId/resourceGroups/rg--net/providers/Microsoft.Network/virtualNetworks/vnet-01/subnets/default"

So we just need to use expertise string functions lie split, combine, concat etc. in a such a way that we take out only VENT name from above string. Therefore lets write code to covert to string and then split so as to get VNET name. Final code is as below –

"[split(string(field('Microsoft.Network/networkInterfaces/ipconfigurations[*].subnet.id')),'/')[8]]"

Creating Custom Policy

Login to Azure portal -> In middle top search box type “Policies” -> Select Definitions -> Create New. Then add values as shown below. The name of the policy I have given as “Azure Resources should be connected to an approved virtual networks”. Code of policy to be added under section “POLICY RULE” can be taken from github link shared below.

Also whenever we create custom policy always get it added in new Category as “My Custom Policies”. Do not add any Role Assignment. Then save to finish policy creation wizard. [click to get better view.]

Open the newly created policy and click on Assign. [click to get better view].

On the Basics tab, make “policy enforcement” option as disabled. We want to just view the report of azure resources not deployed in approved VNET. If we enforce policy means it will not allow to provision new resources in any of the VNET other  than listed in parameters tab below. For testing purpose I disabled policy enforcement.

Under “Remediation” tab uncheck the option “create a managed identity”.

On Parameters tab, make sure that you add names of all VNETs against which you want to evaluate azure resources is added in below shown format only. Then Click on “Save” button for VNET names added and click on “Review and Create” to complete assignment process.

Output

Virtual Machines

I could see that Virtual Machines not listed under the VNETs I added as parameter. Refer screenshots [click to get better view] –

Same policy will also be automatically applied on Azure App Service configured with private endpoint. Refer below output –

Full and final working policy is present at this Github link –

kunalchandratre1/NicVnetPolicy:This repo has code related to Azure policy to identify Azure resources whichare not deployed in approved VNETs. (github.com)

Important –

This policy will not work for Azure Kubernetes Service, VM Scale Sets as their NIC resource provider format is different. However this policy should work as is for all types of private endpoints where NIC is create with resource provider format of Microsoft.Network/networkInterfaces/.

I have not tested with other types of private endpoint. Please test it and add your experience in comments.

Conclusion

Hope this article helped you to build custom policy and helped to achieve your compliance and governance. Let me know your views in comments section below to improve and what are your thoughts on this approach.

Happy Azure Governance!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Start stop multiple Azure VMs on schedule and save cost!

Export Azure VMs to CSV!

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!