Azure Firewall with Static Outbound Public IP – Azure NAT Gateway

12 min to read.

Abstract

Below conversation may come as a surprise for some organizations when they onboard on Azure Firewall – [Click on image to get better view]

Azure Firewall randomly selects attached public IPs for outbound SNAT connections.

Many times for outbound connections, organizations prefer to use static public IP for Outbound SNAT connections. This helps a lot in whitelisting at organization customer’s / partner’s end. As of today Azure Firewall do not offer this capability.

This article talks about; how can you make sure that traffic leaving Azure Firewall uses a static public outbound IP addresses or range of static public IP addresses.

The solution proposed below is being used in many large enterprises today successfully, for providing static outbound IP to Azure Firewall.

Let’s go!

Understanding inbound and outbound connections

In below diagram I have one Azure VNET in which I have deployed one windows server and Azure Firewall instance in dedicated subnet. Let us quickly understand what is inbound and outbound traffic flow.

The Azure VM do not have any public IP assigned to it. If I host a sample application on top of this Azure VM then public facing endpoint is public IP of Azure Firewall.

Inbound means traffic originating from internet and reaching to Azure VM through DNAT operation of Azure Firewall. Refer below diagram – [click on image to get better view].

Note – Remember, for the inbound connection there will be response sent. This is response traffic and not outbound traffic.

Outbound means traffic generated within Azure environment and going out to internet. Refer below diagram – [click on image to get better view].

Problem Statement

No Outbound Static IP

Now that inbound and outbound traffic is clear; in above diagram Partner DC may ask for public IP of your Azure environment to whitelist in their firewalls. So in that case we will need static public IP/ range of public Ips assigned to firewall.

This way outbound traffic generated with Azure [in our case inside Azure VM] to partner DC will be SNAT to static public IP.

However as stated above; Azure Firewall randomly picks up any public IP assigned to it for outbound SNAT. This proposes challenges to customers.

Limited SNAT ports

Another important point to consider is about SNAT ports. When traffic goes out from firewall, a port will be used to send the traffic outbound. These number of ports are always per public IP assigned to Azure Firewall. Azure Firewall offers 2048 ports per public IP assigned. This might be a lower number based on application nature.

Whenever one outbound connection is made, one port will used from Azure Firewall. The engaged port is not released until connection operation is completed. At the same time if Azure Firewall needs to make another outbound connection then another port will be used.

This means, at any given point, one instance of Azure Firewall service with one public IP attached, can make maximum 2048 outbound concurrent connections.

Azure Firewall support max 250 public Ips. So total SNAT ports available on Azure Firewall – 250*2048 = 512,000.

However, 250 public IP is still a big number. You do not attach so many public Ips to Firewall instance out of the blue. You generally attach per application. Bringing 250 applications behind single Azure Firewall instance may not be possible right away when you start Azure journey.

At the same time; your single application running behind Azure firewall may easily need concurrent a million outbound connections. Example, WebSocket based chat application, mobile app communication using websockets.

Therefore we need a way by which we can scale SNAT ports as well for outbound connections passing through Azure Firewall.

Verifying Random IP Behavior

I have below construct in my Azure Subscription. [Click on the image to get better view] –

1.      Created VNET with 3 subnets – Firewall Subnet, Web VM subnet, Jump VM Subnet

2.      Attached two public Ips to Azure Firewall.

3.      Created Web VM in Web Layer and did NOT attach any public IP.

4.      Create Route Table with below routes and attached to web layer subnet

a.      If destination is Jump Layer subnet – next hop VNET

b.      If destination is internet [0.0.0.0/0] – next hop AzureFirewall IP.

5.      Created Jump VM WITH Public IP; just to take RDP to web VM over its private ip.

Here are the screenshots of Azure Firewall IP configurations and UDR on Web Layer Subnet. [click to get better view].

I am going to add ifconfig.me domain inside Azure firewall to allow the outbound traffic. Therefore we need to add DNS settings on Azure Firewall. Below is the screenshot to add the same – [click to get better view].

We will be using ifconfig.me URL to get the IP. Therefore allow the same in Azure Firewall as shown below using application rules. [click to get better view].

Now when I run command Curl ifconfig.me/ip from putty of Web VM; we see the firewall IP as an output. This means because of UDR attached to web layer, internet traffic initiated from web vm is passing through Azure Firewall. Azure Firewall then sending to internet by SNAT [Source NAT] to its own one of the public IP.

From below screenshot you can clearly see that any random IP out of two public Ips attached 

to Azure Firewall, is being used while outbound traffic is SNAT from Azure Firewall. 

[click to get better view]. 

From above screenshot it is visible that out of the two public Ips random IP is being chosen for outbound access.

Here is the Deal…NAT Gateway!

Microsoft Azure releases Virtual Network NAT service that simplifies the outbound-only internet connectivity for virtual networks. When configured on a subnet, all outbound connectivity uses your specified static public IP addresses.

Many times this service is also referred to as NAT Gateway.

Per public IP attached to NAT gateway we get 64,000 outbound SNAT ports. One NAT gateway can have 16 public IP addresses attached.

So total outbound SNAT ports available with a NAT gateway = 16 * 64,000 = 1,024,000.

So NAT Gateway is our solution to get Static Public IP for outbound traffic flowing through Azure Firewall.

Create NAT gateway and associate to subnet of Azure Firewall. Assign a public IP to NAT gateway. So as shown in the below diagram, even if the Azure Firewall has 2 public IP addresses; traffic originating from Web VM always take NAT Gateway associated public IP as outbound IP. [click to get better view].

Create NAT GW and associate to AzureFirewallSubnet using below sample powershell commands. Make sure you replace Resource groups name, VNET Name and location as appropriate.

# Create public IP addresses

New-AzPublicIpAddress -Name public-ip-1 -ResourceGroupName AzFwStaticIPRG -Sku Standard -AllocationMethod Static -Location 'Central India'

# Create NAT gateway

$PublicIPAddress1 = Get-AzPublicIpAddress -Name public-ip-1 -ResourceGroupName AzFwStaticIPRG

New-AzNatGateway -Name firewall-nat -ResourceGroupName AzFwStaticIPRG -PublicIpAddress $PublicIPAddress1 -Location 'Central India' -Sku Standard

# Associate NAT gateway to subnet

$virtualNetwork = Get-AzVirtualNetwork -Name MyVNET -ResourceGroupName AzFwStaticIPRG

$natGateway = Get-AzNatGateway -Name firewall-nat -ResourceGroupName AzFwStaticIPRG

$firewallSubnet = $virtualNetwork.subnets | Where-Object -Property Name -eq AzureFirewallSubnet

$firewallSubnet.NatGateway = $natGateway

$virtualNetwork | Set-AzVirtualNetwork

 The Azure NAT Gateway I created has public IP address 52.x.x.20 and is visible on Azure portal as below – [click to get better view].

Similarly it is associated to AzureFirewallSubnet as shown below – [click to get better view].

Verifying Static Outbound IP Address

Logged in to webvm using SSH and ran curl ifconfig.me command. We can see that it is showing Nat Gateway IP. Using NAT Gateway, for outbound traffic passing through Azure Firewall NEVER selected random IP from Azure firewall. [click to get better view].

Conclusion

Hope this article helped to overcome behavior of random public IP selection of Azure Firewall for outbound traffic. NAT Gateway is fully managed service and helps to configure static public outbound IP for outbound traffic passing through Azure Firewall.

Happy Static IPs!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

Proven Azure Architecture Patterns using PaloAlto NGFW and F5 DDoS and WAF

4 VNETs and transitive Routing using Azure firewall

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Azure VM disk encryption, what should be my approach!

Bypass onpremises firewall to RDP or SSH into Azure VM 

How to Restrict Azure Firewall Access from Contributor, Owner Role?

 7 min to read.

Abstract

When I look at Azure Built In Role Documentation I always feel this is really big list of built in roles and hardly any Organization will require to create new Azure custom role.

However when you work with Azure Customers, your faith and beliefs are bound to fail. And if your customer is Bank, your concepts of Security will fail.

So my customer started big on Azure Firewall. It was maintained by their network administrators team. One fine day network admin discovered that the Azure Firewall is visible to all users with Contributor and Owner access.

Obviously the demand came that Owner/ Contributor should have full access EXCEPT Azure Firewall.

Damn Zero Trust!

Anyways, How to hide/ restrict Azure Firewall access from Azure Owner and Contributors? Read on…

Have you heard of Azure custom roles?

The solution is simple. We need Custom Owner and Contributor role that will have default access levels except for Azure Firewall.

Azure Built in Roles list is here - https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles.

If the Azure Built In Roles don’t meet the specific needs of your organization, you can create your own custom roles.

So we will need to build custom Owner and Contributor role that do not have access to Azure Firewall.

Always get Built in Role Definition first!

For simplicity I am going to build custom Contributor Role only. Same can be replicated for Owner Role too.

First let us grab the built in role definition of Contributor Role using Azure PowerShell.

If you do not have Azure PowerShell get it installed or use it from Azure Shell. I want to edit the role definition on my local laptop so I am using Azure PowerShell from my local machine.

First login to correct Azure Subscription and Azure AD. I have seen major bomb blasts due to wrong selection of Subscription and Azure AD Tenant.

Connect-AzAccount -Subscription YourSubscriptionID  -Tenant YourAzureADTenantName-Like-Something.onmicrosoft.com

Get Contributor Role definition in JSON format with below command –

Get-AzRoleDefinition -Name "Contributor" | ConvertTo-Json | Out-File "YourPath\Contributor.Json"

 

Make sure that correct Path is selected as per your choice in above command. The Contributor role definition JSON will be created at specified location and it looks similar to below screenshot. Open JSON in Visual Studio Code. [click to get better view].

Modify the Contributor Role to Restrict Azure Firewall access!

I copy pasted above Contributor Role JSON to another document in VS Code changed the name of file..

The name of the role I changed to “Contoso-Contributor.json”.

Now take quick steps as mentioned below –

1.      Remove the Id filed completely. When you create role in Azure PowerShell; ID filed automatically gets created for you.

2.      As this is going to be customer Azure Role; change “IsCustom” property to True.

3.      Change the description that suits to your needs. I changed it to – Contoso Contributor Grants full access to manage all resources, but does not allow you to assign roles in Azure RBAC, manage assignments in Azure Blueprints, or share image galleries. However  users with this role can't access Azure Firewall.

4.      To remove the firewall access we need to add below line in NotActions segment of JSON as below – Microsoft.Network/azurefirewalls/*"

When we added * above, this means all types of actions are removed / restricted for Azure Firewall Resources. Make sure that you add “comma” on second last line.

5.      Make sure you add the desired subscription at the bottom in “AssignableScopes” property. Unless subscription Id added the role will not be created. The format of subscription has to be as follows -

Refer to screenshot below – [click to get better view].

Create the Role Definition in Azure Subscription

Now run below PowerShell command to get the role created. Make sure you use the Path of your choice where Contoso contributor json definition is present.

New-AzRoleDefinition -InputFile "YourPath\Contoso-Contributor-NoFirewallAccess.json"

 

The output of the command will be shown as below – [click to get better view]

Assign the role to user

I have created a user named as Contoso User in My Azure AD as shown below – [Click to get better view]

Then go to Azure Subscription -> Subscription -> Select Subscription [which is used in AssignableScope] -> Access Control -> Add -> Role Assignment. Selected Contoso User and assigned custom contributor role – [click to get better view]

Verify Azure Firewall Access

So, I selected two users – One with Contributor role and other with Contoso-Contributor Role. When click on Azure Firewall; I can clearly see that even If the Contoso User has all the access same as normal Contributor; the Azure Firewall Resource is not visible – [click to get better view].

Conclusion

Hope this article helped to Custom Contributor role to restrict / hide Azure Firewall.

Happy access control!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

4VNETs and transitive Routing using Azure firewall

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!

Bypass onpremises firewall to RDP or SSH into Azure VM

Azure VMs – Export to CSV

Abstract

For human race, there are some common tasks in daily life which you must do. Without which humans can survive but you can’t say if they are “living the life”. For example, taking a bath is one of such tasks. You can survive without taking a bath but you will not like it. These tasks are those which bring “life” to humans daily lives and make them enjoy their stay on earth.

Out of these humans there is special category of humans – I call them “Humans who work in Software field”. They can be sub-categorized into below –

       1.     Software developers
2.      IT administrators
3.      Project managers
4.      Software Engineers
5.      Solution Architects

For all these sub-categories you can apply prefix such as Senior, Junior, Principal, Full Stack, Distinguished and in recent times we have a new addition to this prefix known as “CLOUD”. These are humans who can easily survive without bath but there is one thing without which none of these humans can survive and it is known as “Export to CSV”. It is more or equally important like eating food and drinking water for them.

No matter how many cutting edge features and service Microsoft Azure is bringing, we still feel the product or service is not complete unless you offer “the” functionality of “Export to CSV”. Surprisingly Azure VM export to csv do not exists on Azure portal and you need to write a PowerShell for this. Therefore my lazy followers and friends asked me to write PowerShell to export Azure VM to CSV. So here we are!

Let’s go!

Current state of feedback

Users have provided the feedback to bring the functionality of export to csv for all azure resources on the portal. It is on the roadmap. You can view the details here - https://feedback.azure.com/forums/216843-virtual-machines/suggestions/37934101-virtual-machine-list-export-to-csv.

Why yet another new script?

I spoke to multiple people who are dealing with Azure VMs daily basis and requirements from there for CSV was an eye opener. Many of the scripts available today provides very minimal details about VM when exported to CSV. Most of the Azure Administrator find them not so useful. So after a quick survey with few of Lazy followers I received below list as a top ask for Azure VM to CSV export. In the below list the top ask was to retrieve Azure VNET for Azure VM and Azure VNET subnet for Azure VM using PowerShell. I have addressed this as well in the script.

So, current script provides the CSV output of Azure VMs with great details. Refer below columns list –

Delete Azure File Storage Directory and files using PowerShell and secret sauce of recursion

Abstract

Come on guys!! Seriously?

After this successful post of Azure Virtual Machine – Frequently asked questions – not easily answered I got almost 54+ requests/ comments/ emails asking specifically for a blog post on sharing script for “Deleting Azure File Storage Directory using PowerShell”.

Not sure why people felt this is hard to write a PowerShell script. Well, this post is about the same topic and at the end of post you will know how to delete Azure file storage Directory and Files using PowerShell scripting.

You must have read the word in title “Secret sauce of Recursion”; about that later in the post below.

Lets go!

What is big deal in deleting Azure File Storage Directory and Files?

Well, you may think this as straight forward PowerShell. I also thought the same way until I actually tried for it.

As a best practice, before trying anything from PowerShell I always do the same task using Azure Portal. So for deleting Azure Files I went on to the portal. Created a storage account of type v2; and then created a File storage of 10GB as shown in the screenshot below. At this point the file share is empty.

Note – Focus on Blue box and blue underline; I wrote this blog at 2AM in the night. This is how I am getting attached to my blog after my wife.

Then I created a directory and uploaded few files within directory and few files directly in share and few directories inside the root directory and then again few files.

Note - If you have not understood above sentence; then blame 2AM time.

So my directory structure looks as below on Azure File storage share –

Yellow boxes are directories, blue boxes are files and green box is the actual file share inside which all of this is present. I tried to delete the directory wwwroot from the portal UI and I got below message.

This means I will have to delete each and every file present in the directory of my file share. If there is cascading of directories [directory within directory] then each of the directory will have to be made empty first and then delete each of the directory one by one and then the actual directory I want to delete.

This is big deal task and no wonder people are finding it difficult in PowerShell. As of now there is no single command by which you can delete directory including everything within it. So if I want to retain my file share but delete everything within it; I must empty all the directories.

What is my way out?

Well, you need to traverse through each of the directory and make it empty. So you might be thinking for for-each loop in PowerShell to do it. You are correct. But only for-each or for loop will not help you win the war. You need super power called as “RECURSION”. Write a recursive function in PowerShell and we should be able to win the war.

Why people find it difficult to write RECURSIVE functions?

People of my generation [people who started career between 2006 to 1012 year]who had started their programming career in C, C++, micro-controller programming will never find recursion difficult. Because the very first program we had build was Factorial number and it is product of all integers from 1 to n. For example factorial of 5 is 5 x 4 x 3 x 2 x 1  = 120.

This classic problem can be solved by recursion effectively.

Recursion is basic programming technique you can use in many languages including PowerShell, in which a function/ method calls itself to solve some problem. A method/ function that uses this technique is called as “Recursive function”. There are many problems that can be solved only by recursion.

To empty azure file directories we need recursion in PowerShell.

I have seen most of the new comers who starting their career in IT fields directly jump to Artificial Intelligence and Machine learning programming. More of less basics of programming may get missed like data structures, recursions etc.. I asked the year of experience to many people who demanded to write this PowerShell to delete Azure files directories and files and no surprise most of them were in the range of 4 to 6 years of experience.

Anyways no hard feelings!

Let us build PowerShell to delete Azure file storage directories and files recursively.

Define variable and create Azure Storage context

I am using latest Azure PowerShell module.

We need to define the variables for storage account details and then create the context for storage account. Context is an important steps as for every operation you perform against any of the service in Azure storage context should be passed in PowerShell commands.

#define varibales

$StorageAccountName = "YourStorageAccountName"

$StorageAccountKey = "YourStorageAccountPrimaryKey"

$AzShare = "kunalshare – you write your file share name here"

$AzDirectory = "LatestPublish"

#create primary region storage context

$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey

$ctx.ToString()

#Check for Share Existence

$S = Get-AzStorageShare -Context $ctx -ErrorAction SilentlyContinue|Where-Object {$_.Name -eq $AzShare}

Get the directory name and call for delete function

Now as per the directory hierarchy in above diagram; you can see that inside File Share I have a root directory called as “LatestPublish” and all other files/ directories are present inside it.

So I want to delete “LatestPublish” folder. So let us retrieve the reference for the folder and then call delete recursive function.

# Check for directory

$d = Get-AzStorageFile -Share $S -ErrorAction SilentlyContinue|select Name

if ($d.Name -notcontains $AzDirectory)

{

    # directory is not presetn; no action to be performed

   

}

else

{

    $dir = Get-AzStorageFile -Share $s -Path $AzDirectory   

    RemoveFileDir $dir $ctx #Calling function to remove directory.

}

Recursive function to remove directories

In above code of getting root directory we are calling a function named as RemoveFileDir. Let us write this function.

function RemoveFileDir ([Microsoft.Azure.Storage.File.CloudFileDirectory] $dir, [Microsoft.Azure.Commands.Common.Authentication.Abstractions.IStorageContext] $ctx)

{  

    $filelist = Get-AzStorageFile -Directory $dir

   

    foreach ($f in $filelist)

    {

       

        if ($f.GetType().Name -eq "CloudFileDirectory")

        {

            RemoveFileDir $f $ctx #Calling the same function again. This is recursion.

        }

        else

        {

            Remove-AzStorageFile -File $f           

        }

    }

    Remove-AzStorageDirectory -Directory $dir

   

}

In above function, we are receiving the root directory object to be deleted and storage context as parameter. Then we use for loop to traverse through the received directory object and see if there any more directories. If there is a directory found then we call the same function in recursion mode; until we reaches to last directory/ file present in the hierarchy. Once we reach to last item then we delete it.

Download entire source code

Download the entire source code of deleting Azure file directories from GitHub - https://github.com/kunalchandratre1/DeleteAzureFilesDirectoriesPowerShell

Bonus tip

If you delete file share from the azure portal; everything within that gets deleted automatically and you don’t have to indulge yourself in to PowerShell. However if you want to delete selected directories and files from File share then PowerShell is the only way or do it manually from portal.

Conclusion

Hope this blog helped you t get working Azure PowerShell script for deleting Azure file storage directory and files recursively. Please feel free to add/ update your comments. Thanks.

Happy recursion!! 

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefitted, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.