Thursday, May 20, 2021

PaloAlto NGFW, F5 WAF and DDoS - Proven Azure Architecture Patterns

14 min to read.

Abstract

I have been part of many Azure Landing Zone implementations in last 1.5 years. Many organizations are already invested on below security devices –

1.      F5 WAF – Web Application Firewall

2.      F5 DDoS – Distributed Denial of Service

3.      PaloAlto NGFW – Next-Generation Firewalls

Enterprises have inclination to use same devices on Azure as well. Not only because they are invested in licenses costs but also on skillset cost. There are dedicated [paid] teams managing these devices and logging, monitoring, SOC, incident management, dashboards everything is already streamlined for them.

This article talks about Azure Architecture patterns I have seen across many organizations Azure Deployment using F5 WAF, DDoS and PaloAlto NGFW.

Note - Below I am considering incoming traffic from internet and outgoing to internet. I am not talking about Azure-OnPremises connectivity scenario in below architecture patterns.

Also recommendations as per my experience; deploy the security devices in dedicated VNET and configure all applications in separate VNET [Hub-Spoke!].

Azure Architecture Pattern 1

Refer to first pattern on deploying F5, PaloAlto devices in combination on Azure – [Click on image to get better view].



Pattern 1 Description

1.      The traffic from internet lands on public IP attached to External Standard Load Balancer [Layer 4] of Azure in front of F5 DDoS VMs.

2.      On External Azure Standard load balancer you can configure inbound public Ips. These public Ips can be added as per number of applications sitting behind your internet DMZ.

3.      DDoS Azure VMs are configured as Active-Passive or Active-Standby. At any given point only single Azure VM of F5 DDoS is serving the incoming requests from internet.

4.      PaloAlto NGFW VM-Series Azure VMs are configured as active-passive. It has an Azure Standard internal load balancer in front of it for load balancing and HA achievement. For monitoring purpose it is required to know incoming source IP at firewall; therefore we did not perform SNAT on DDoS. However at PaloAlto VM-Series firewall on Azure we can perform SNAT [if need be].

5.      F5 WAF VMs are present behind the PaloAlto NGFW. F5 WAF is also configured in Active-Active configuration. It has an internal load balancer in front of it for load balancing and HA achievement.

6.      F5 WAF machines do not have any public IP assigned.

Where do we perform SSL Offload?

To inspect incoming traffic; SSL offload is mandatory. Without SSL Offload packet can’t be inspected/ verified if it is valid traffic or malicious traffic. Therefore in above Azure Architecture pattern we should perform SSL Offload on F5 DDoS VMs. Post which you can inspect traffic on PaloAlto NGFW and F5 WAF.

What if I want to achieve end to end SSL on incoming traffic?

Perform SSL Offload on F5 DDoS Azure VMs. Then let traffic be inspected on F5 DDoS, PaloAlto NGFW and F5 WAF Azure VMs. On F5 WAF itself post inspection you re-configure the leaving traffic with new SSL certificate. So after F5 WAF when traffic goes to application VMs then it will be accessed over internal private IP communication; over HTTPS. This is end to end SSL.

How my outbound traffic “generated within my Azure environment” will flow?

In above diagram we have configured/ assigned public IP directly to one of the Azure PaloAlto VM. So traffic initiated from Application VMs directly goes to Azure PaloAlto VM and then it goes out to internet. In this traffic flow, F5 WAF and F5 DDoS Azure VMs do not come in to picture.

Make sure you attach separate public IP to one of NIC of each PaloAlto Azure VMs.

In case you want to ask 3rd party company to whitelist your outgoing IP in their firewall; then both Ips of firewall will be required to whitelisted.

Note - If you fear of exhausting SNAT ports for Azure PaloAlto VM then you can also configure Azure NAT gateway in same subnet of PaloAlto NGFW external/ internet interface and use it for all outbound traffic. I have not tried this but should be pretty straight forward.

Azure Architecture Pattern 2

Many times customers prefer to use separate public Ips for inbound and outbound traffic. Separate outbound traffic public IP helps in whitelisting at their customers end. Example, WoodGrove org has api which will be called from Azure environment of Contoso company. So WoodGroove will ask Contoso to provide public IP range from which the api will be called. In this case Contoso will share the public IP to WoodGroove, specifically attached for outbound traffic “generated within Azure”.

In above pattern #1, we have public IP attached at the F5 DDoS Azure LB and also to the PaloAlto NGFW VM-Series. Sometimes customer do not want any public to be attached other than entry point. This can be addressed using below pattern. [Click on image to get better view].



Pattern 2 Description

1.      The traffic from internet lands on public IP attached to External Azure Standard Load Balancer [Layer 4] of Azure in front of F5 DDoS Azure VMs.

2.      On External Azure Standard load balancer you can configure inbound public Ips. These public Ips can be added as per number of applications sitting behind your internet DMZ.

3.      DDoS Azure VMs are configured as Active-Passive or Active-Standby. At any given point only single Azure VM of F5 DDoS is serving the incoming requests from internet.

4.      PaloAlto NGFW Azure VMs are configured as active-passive. It has an Azure Standard internal load balancer in front of it for load balancing and HA achievement.

5.      F5 WAF VMs are present behind the PaloAlto NGFW. F5 WAF is also configured in Active-Active / Active-passive configuration. It has an internal load balancer in front of it for load balancing and HA achievement.

6.      F5 WAF and PaloAlto NGFW VMs do not have any public IP assigned.

7.      Standard Azure load balancer attached in front of F5 DDoS also has outbound rules configured. To know more refer - https://docs.microsoft.com/en-us/azure/load-balancer/outbound-rules#scale.

8.      SSL Offload still happens at F5 DDoS Azure VMs.

9.      There is no SNAT performed for incoming traffic at DDoS therefore source IP is visible in PaloAlto NGFW Azure VMs.

 

How my outbound traffic “generated within my Azure environment” will flow?

In above diagram we have configured/ assigned public IP to External Azure Standard Load Balancer; present in front of F5 DDoS Azure VMs. Therefore we will need to configure UDR [Azure Route Tables] to make outbound traffic flow in below sequence –

App VM -> F5 WAF -> PaloAlto NGFW -> F5 DDoS -> Azure Standard LB -> Internet

In this case, outbound traffic flowing through F5 WAF and DDoS do not add any value. However as customer requirement is not to allow any public IP other than entry point; we will have to make traffic flow through each of the device. Here F5 WAF and DDoS will act only as pass through for traffic and adding extra hops.

Azure Architecture Pattern 3

When I implemented Azure Landing Zones at financial organizations many of them asked a variation in above patterns with Proxy deployment on Azure for Outbound traffic. So for outbound traffic below can be another architecture pattern where traffic flows as - >

App VMs-> Proxy VMs -> PaloAlto NGFW VMs -> Internet.

Same approach can also be used for pattern #2 above. Below is architecture for Pattern # 1 with Proxy – [Click on image to get better view].



Azure Architecture Pattern 4

While we can have entry point on Azure Internet DMZ Zone through F5 DDoS BIG IP on Azure VMs; we can achieve the high availability WITHOUT USING Azure Standard Load Balancer as well.

Refer to below diagram for the same - [Click on image to get better view].



In above diagram assuming we want to retain Source IP to Firewall level; we can configure DDoS F5 Azure VMs in Active-Passive.

In the diagram, for F5, in the event of a failover, the IP configuration is deleted from active device and recreated on that standby device’s network interface. So your public IP [virtual IP] on which traffic lands remain same irrespective of which VM is serving the requests.

This failover is API call based failover and well explained here - Azure(f5.com).

Same API based failover can also be achieved for F5 WAF device. Also you can keep adding secondary public IP addresses per application being onboarded behind this DMZ zone.

Azure Architecture Pattern 5

I have many organization using PaloAlto Azure VMs Firewall for outbound and inbound combined had to use bigger size VMs. There is another deployment pattern I have seen where separate Azure PaloAlto Firewalls are used for inbound and outbound.

In this pattern we will need to perform mandatory SNAT at F5 WAF level to allow return traffic reach to correct destination of F5 WAF and outbound traffic generated within app layer to reach to outbound PaloAlto Azure VM. Refer to below diagram for details - [Click on image to get better view].



Here we are having two public Ips attached to one of the NIC of each of outbound PaloAlto NGFW. So that based on current active VM the outbound traffic flow outside to internet.

Remember you will need UDR configured in app layer in such a way that traffic destined to internet will flow to PaloAlto NGFW Azure VMs and rest of the traffic should flow to F5 WAF.

Is there a way to run F5 BIG-IP DDoS on Azure on entry point in Active-Active with No SNAT?

If you want to preserve incoming source IP till firewall/ app layer then SNAT should not be used. If we plan to deploy F5 DDoS in Active-Active then SNAT is required. Otherwise return traffic does not understand which was VM devices to be used for return/ response traffic. In this case source IP of incoming traffic will not be visible to firewall. In such case many customers take below approach –

1.      Configure SSL Offload on F5 DDoS

2.      Configure F5 DDoS in Active-Active with SNAT.

3.      Add incoming Source IP in X-Forwarded-For [XFF] header.

This is fine if you firewall devices are able to block incoming malicious IP present in XFF. If not, then Active-Passive will make more sense.

Should I always have F5 DDoS on front irrespective of order of devices?

DDoS is for protection of traffic coming from internet. So for internet facing applications, you should always have DDoS in front of everything.

I see PaloAlto NGFW VM-Series on Azure is mentioned as Active-Standby / Active-Passive only? Can it work in Active-Active mode?

As of today PaloAlto VM-Series firewall on Azure can not work in Active-Active mode. Refer to the document for more information - VM-Series in High Availability (paloaltonetworks.com)

Do we need to perform SNAT on PaloAlto VM-Series Firewall?

If you don’t want to retain source IP forwarded from DDoS devices beyond PaloAlto NGFW Azure VMs then you can certainly perform SNAT on PaloAlto VM-Series Firewall Azure VMs. IF you want to retain source incoming IP till application layer then “Do not” perform SNAT on PaloAlto firewall device.

How to preserve Incoming Source IP of internet till Azure application layer VMs?

Many organizations require incoming source IP to be preserved till application layer/ firewall layer for monitoring/ business requirement/ logging/ audit purpose. If this is the case then you will need to configure the F5 DDoS Azure VMs in Active-Passive or Active-Standby mode.

This way Source IP of internet is not NATted on F5 DDoS Azure VMs and it is visible in PaloAlto NGFW layer. If SNAT is configured on DDoS then PaloAlto NGFW will never see real incoming source IP from internet.

Similarly to preserve the source IP beyond PaloAlto Azure firewall; you will need to avoid SNAT on PaloAlto Azure VM-Series and F5 WAF devices.

Refer to Pattern 5 which stated how can you have incoming source IP taken to App layer in XFF header through F5 WAF device. Or you can simply avoid SNAT n F5 WAF Azure VMs also and you will get source IP as internet IP in application layer..

Conclusion

There can be many combinations of the security devices of F5 and PaloAlto on Azure. The above mentioned architecture patterns I have seen at most of the places on Azure. Hope this article helped to design combination of F5, PaloAlto on Azure.

If you have any recommendations to make article better, reach out to me. I will be more than happy to update the article.

Happy NVAs on Azure!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.


Next Related Posts

Restrict Azure Firewall access from Owner and Contributors of Azure Portal

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Acomplex design – Site to Site VPN, VNET Peering and 4 VNETs – how to solve using Azure Firewall?

Azure VM disk encryption, what should be my approach!

Bypass onpremises firewall to RDP or SSH into Azure VM

Sunday, April 25, 2021

How to Restrict Azure Firewall Access from Contributor, Owner Role?

 7 min to read.

Abstract

When I look at Azure Built In Role Documentation I always feel this is really big list of built in roles and hardly any Organization will require to create new Azure custom role.

However when you work with Azure Customers, your faith and beliefs are bound to fail. And if your customer is Bank, your concepts of Security will fail.

So my customer started big on Azure Firewall. It was maintained by their network administrators team. One fine day network admin discovered that the Azure Firewall is visible to all users with Contributor and Owner access.

Obviously the demand came that Owner/ Contributor should have full access EXCEPT Azure Firewall.

Damn Zero Trust!

Anyways, How to hide/ restrict Azure Firewall access from Azure Owner and Contributors? Read on…



Have you heard of Azure custom roles?

The solution is simple. We need Custom Owner and Contributor role that will have default access levels except for Azure Firewall.

Azure Built in Roles list is here - https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles.

If the Azure Built In Roles don’t meet the specific needs of your organization, you can create your own custom roles.

So we will need to build custom Owner and Contributor role that do not have access to Azure Firewall.

Always get Built in Role Definition first!

For simplicity I am going to build custom Contributor Role only. Same can be replicated for Owner Role too.

First let us grab the built in role definition of Contributor Role using Azure PowerShell.

If you do not have Azure PowerShell get it installed or use it from Azure Shell. I want to edit the role definition on my local laptop so I am using Azure PowerShell from my local machine.

First login to correct Azure Subscription and Azure AD. I have seen major bomb blasts due to wrong selection of Subscription and Azure AD Tenant.

Connect-AzAccount -Subscription YourSubscriptionID  -Tenant YourAzureADTenantName-Like-Something.onmicrosoft.com

Get Contributor Role definition in JSON format with below command –

Get-AzRoleDefinition -Name "Contributor" | ConvertTo-Json | Out-File "YourPath\Contributor.Json"

 

Make sure that correct Path is selected as per your choice in above command. The Contributor role definition JSON will be created at specified location and it looks similar to below screenshot. Open JSON in Visual Studio Code. [click to get better view].



Modify the Contributor Role to Restrict Azure Firewall access!

I copy pasted above Contributor Role JSON to another document in VS Code changed the name of file..

The name of the role I changed to “Contoso-Contributor.json”.

Now take quick steps as mentioned below –

1.      Remove the Id filed completely. When you create role in Azure PowerShell; ID filed automatically gets created for you.

2.      As this is going to be customer Azure Role; change “IsCustom” property to True.

3.      Change the description that suits to your needs. I changed it to – Contoso Contributor Grants full access to manage all resources, but does not allow you to assign roles in Azure RBAC, manage assignments in Azure Blueprints, or share image galleries. However  users with this role can't access Azure Firewall.

4.      To remove the firewall access we need to add below line in NotActions segment of JSON as below – Microsoft.Network/azurefirewalls/*"

When we added * above, this means all types of actions are removed / restricted for Azure Firewall Resources. Make sure that you add “comma” on second last line.

5.      Make sure you add the desired subscription at the bottom in “AssignableScopes” property. Unless subscription Id added the role will not be created. The format of subscription has to be as follows -

Refer to screenshot below – [click to get better view].



Create the Role Definition in Azure Subscription

Now run below PowerShell command to get the role created. Make sure you use the Path of your choice where Contoso contributor json definition is present.

New-AzRoleDefinition -InputFile "YourPath\Contoso-Contributor-NoFirewallAccess.json"

 

The output of the command will be shown as below – [click to get better view]



Assign the role to user

I have created a user named as Contoso User in My Azure AD as shown below – [Click to get better view]



Then go to Azure Subscription -> Subscription -> Select Subscription [which is used in AssignableScope] -> Access Control -> Add -> Role Assignment. Selected Contoso User and assigned custom contributor role – [click to get better view]



Verify Azure Firewall Access

So, I selected two users – One with Contributor role and other with Contoso-Contributor Role. When click on Azure Firewall; I can clearly see that even If the Contoso User has all the access same as normal Contributor; the Azure Firewall Resource is not visible – [click to get better view].



Conclusion

Hope this article helped to Custom Contributor role to restrict / hide Azure Firewall.

Happy access control!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

4VNETs and transitive Routing using Azure firewall

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!

Bypass onpremises firewall to RDP or SSH into Azure VM

Tuesday, April 6, 2021

Transitive Routing and Site to Site VPN, Azure Firewall, VNET Peering, 4 VNETs

14 min to read.

Abstract

In earlier blog I talked about solving transitive routing problem in 3 VNETs. Recently I came across a situation where transitive routing was required across 4 layers.

A (On-premises DC) < site to site VPN> B (VNET with VPN GW) < peered to> C (Firewall VNET) < peered to> D (server app VNET).

Interested to know how to achieve transitive routing across 4 VNETs? Read on.

Problem Statement

Let us understand the problem better by using example. [Click to get better view]-



Requirements are as below -

1.      Contoso company has on-premises DC connected to Azure Landing Zone VNET using Site to Site VNET.

2.      Server Deployment should be done in separate VNET and all traffic should be monitored using Firewall

3.      The firewall must be placed in separate VNET for logical separation and more granular control.

4.      Communication between OnPremises DC server to Azure Server VNET must pass through S2S VPN, Firewall in both directions.

This is classic Transitive routing scenario in Azure, but a complex one. There are 4 networks and you need connectivity between first and fourth; without having them connecting directly.

Let us solve this one by one.

Solving on premises DC connectivity to Azure Landing Zone VNET

I don’t have any on premises site for demo. Therefore I created Azure VNET only and will be treating it as on premises site.

So we have 2 Azure VNETs across which we need connectivity. You can easily set it up using VNET to VNET connection or using VPN Gateway connection. However to replicate real world scenario we will create Site to Site VPN between On premises DC and Contoso Landing Zone Azure VNETs; using Azure VPN Gateway in each.

Connection between Server VNET to Firewall VNET

Server VNET needs to send data [ in our case we will just try RDP] to on premises DC VM. However they are not connected directly.

Also traffic to/ from server VNET has to be filtered through firewall. Therefore we created separate VNET for Server and firewall. I decided to use Azure firewall and the VNET is called as Transit VNET below.

As Server VNET just receive / send data to/ from Azure firewall; we need Standard VNET Peering between Server and Transit VNET.

Connection between Firewall VNET to Landing Zone VNET

Data from Server VNET VMs need to be sent to on premises. However it has to be passed from Azure firewall. Therefore we need Transit VNET to send data to on premises in reality.

So we want traffic from firewall VNET to reach to contoso on premises DC VNET using S2S VPN Gateways present in both VNETs.

For this we will need to setup VNET Peering with Remote Gateway option between Transit VNET and Landing Zone VNET. Only Standard VNET peering won’t work in this scenario.

Final Solution Architecture looks like below

[click to get better view].



Setting up Site to Site between Azure VNETs

We need to create two Local network gateway as shown in above diagram. You need to take care of below when you create S2S between Azure VNETs using Azure VPN Gateways.

1.      Create Contoso DC Local network gateway and assign public IP of Contoso DC VPN GW. Assign range of on premises DC private VNET.

2.      Create Contoso Zone Local Network Gateway and assign public IP of Contoso Zone VPN GW. Assign ranges of Server, Transit and Zone private VNET.

Refer below screenshots – [click to get better view]





Then to setup Site to Site IPSec tunnel follow the guide as describe here - Tutorial- Connect on-premises network to virtual network: Azure portal - Azure VPN Gateway | Microsoft Docs

Refer below screenshots – [click to get better view]





This completes the S2S connection between 2 Azure VNETs using Azure VPN Gateway.

Provision Azure Firewall and Add Network Rules

Create dedicated subnet for Azure Firewall in Transit VNET and then create Azure Firewall in Transit VNET.

We want RDP traffic from Server VNET to DC VNET and vice versa to allow through Azure Firewall. Therefore add below rules in Azure Firewall network rules. [click to get better view]



Setup VNET Peering between Server, Transit and Zone VNETs

First configure server VNET to Transit VNET peering as shown below. Here as both VNETs to dot have Azure VPN Gateway; so this will be standard VNET peering. [click to get better view]



Then configure Transit VNET to Zone VNET peering. Here Zone VNET has Azure VPN Gateway and we want traffic filtered from Transit VNET firewall to pass to on premises DC VNET over S2S.

Therefore use remote gateway setting in Transit VNET peer, and “Use this VNETs Gateway or Route server” setting in Zone VNET peer as shown. [click to get better view].



This step completes all the required peering setting as per architecture diagram.

Configure Azure Route Tables

From Server subnet we want traffic to go to Contoso DC and pass through Azure Firewall. Therefore we need to add below rules on server subnet –

1.      If destination is Contoso DC VNET then next hop is firewall IP

2.      If destination is Zone VNET then next hop is firewall IP

Then assign route table to server subnet as shown below. [click to get better view].



The traffic received on Contoso Zone GW and destined to server vnet should also be passed always to firewall. Therefore we need to add below rules on Zone Gateway Subnet route table –

1.      If destination is server VNET then net hop is firewall IP.

Then assign route table to Gateway subnet of Zone VNET. [click to get better view.]



This completes configuration of all Route tables.

Confirm the connectivity between Server and On Premises DC VNET

Login to Server VM using public IP. Then simply ping to on premises Dc VM. The ping should be successful as shown below. If we try to take RDP to On Premises DC VM over private IP from server VM; it should be successful as shown below. [click to get better view].



You can view the source address of server VM from event viewer as below – [click to get better view]



Similarly RDP from On Premises DC server to Server VM over private IP should also be successful.

Conclusion

Hope this article helped to design Transitive Routing across 4 VNETs.

Happy Peering!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

Transitive routing across 3 VNETs using Ubuntu VM

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!

Bypass onpremises firewall to RDP or SSH into Azure VM

Friday, March 19, 2021

Transitive Routing in Azure VNet Peering using Ubuntu VM IP Forwarder

8 min to read.

Abstract

Transitive routing requirement is common reality in every “Enterprise Grade” Azure Implementation!

Simplest way to connect two Azure VNet is to use Azure Vnet Peering. If I have 3 VNETs peered with each other; example A <> B <> C. I want Azure VM in A Vnet to talk to Azure VM in C Vnet then it wont work; because A and C VNet are not peered directly to each other; but through B Vnet.

Fact that VM in A vnet not able to reach VM in C Vnet is called as “Transitive Routing Problem”.

How do I solve it?

There are many solutions. Simplest way to solve transitive VNET peering problem would be, to create an “IP Forwarder using Azure Ubuntu VM” in B Vnet.

Let’s start!

Problem statement in detail




As you can see in above diagram I have 3 Azure Virtual Networks named as

1.      ContosoDC VNET

2.      UbuntuFwVNET  - you can also name it as Transit VNET.

3.      ContosoServer VNET

I have provisioned subnets as shown in the diagram. All VNETs are peered bi-directional as described below –

1.      Contoso Dc VNET < peered to> Transit VNET

2.      Contoso Server VNET < peered to> Transit VNET

Azure VNET Peering is bidirectional. Needless to say, if Contoso DC VNET peered to Transit VNET means; Transit VNET will also require peering to Contoso DC VNET to allow them communicate each other.

However, ContosoServer VNET and ContosoDC VNET are not peered with each other.

Note – None of these VNETs are having VNET Gateway or ExpressRoute Gateway in it. So when you peer make sure you choose None for gateway options as shown below [click to get better view].




I have 2 windows VMS and one Ubuntu VM [with Single NIC] in Transit VNET. One of the windows VM has public IP and rest of the two VMs do not have public IP.

Problem statement - I want to allow RDP from ContosoDCVM [10.11.0.4] to ContosoServer VM [10.10.0.4].

How to go to unknown destination - User Defined Routes - UDR

ContosoDC and Contoso Server VNET are not peered. This means both VNETs do not know each other’s address ranges.

When from an Azure VNET you want to send traffic to another unknown Azure VNET, you use Azure User Defined Route Tables aka UDR and provide the next hop to an appliance/ VM/ NVA/ Firewall/ Router/ or forwarder which will then take care of sending traffic to correct destination.

In our case the NVA is our ubuntu Azure VM. Therefore from both VNETs we need to send the traffic to Ubuntu VM IP. Then ubuntu VM will eventually DNAT [Destination NAT] the incoming traffic to its correct destination.

UDR on Contoso Server Subnet as below – [click to get better view]



UDR on Contoso DC Subnet as below – [click to get better view]





Configure Ubuntu as IP Forwarder

For this there are two settings. One on Network Interface of Ubuntu VM from Azure portal and one inside the OS of Ubuntu itself.

Go to NIC associated to Ubuntu VM and enable “IP Forwarding” as shown below [click to get better view] -  



Take SSH into Ubuntu VM and run below commands to enable the IP forwarding inside OS.

$ sudo vim /etc/sysctl.conf

Uncomment the below line – [remove #] and then press Esc and type :wq to save the file.

net.ipv4.ip_forward=1

Run below command to confirm if the change is correct – 

$ sudo sysctl -p

This completes the configuration of IP forwarding for Ubuntu VM.

NAT the incoming traffic

NATing is of two types.

Source NAT – SNAT – to be used when you want to change the sender’s address.

Destination NAT – DNAT – to be used when you want to change the destination address.

Example, Web application is hosted on behind the firewall. Firewall VM has public IP and actual application runs on web server which has private IP. In this case, incoming web request reaches to public IP of firewall and get forwarded to web server private IP. This is translation from public IP to private IP. This is DNAT.

In our case we need to perform DNAT as destination is not known to each of the VNET. Only Transit VNET knows both VNETs. Also in our case DNAT will be for private IP to private IP, as there is no public IP in picture here.

DNAT can be achieved using iptables in PREROUTING and POSTROUTING chaining. To know more refer this awesome link - https://www.karlrupp.net/en/computer/nat_tutorial.

In our case we need below DNAT chaining –

1.      PREROUTING - If source is 10.11.0.4 then DNAT to 10.10.0.4:3389

2.      PREROUTING - If source is 10.10.0.4 then DNAT to 10.11.0.4:3389

3.      POSTROUTING - For all DNAT, use IP address of Ubuntu VM as source

The respective commands are as follows – run it on Ubuntu VM –

$ sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -s 10.11.0.4 --dport 3389 -j DNAT --to-destination 10.10.0.4:3389

$ sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -s 10.10.0.4 --dport 3389 -j DNAT --to-destination 10.11.0.4:3389

$ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

These iptables configuration are not persistent over reboots. So we need to use “iptables-persistent” module to make it happen.

Run below commands – 

$ sudo apt-get update

$ sudo apt install iptables-persistent

On running above command you will have to enter “Yes” twice. This completes the installation of iptables-persistent module.

Then read all iptables configuration we perform and pipe [or copy paste] to rules files. Then add this rules files path in rc.local which eventually will make our iptables configuration persistent over reboot of ubuntu machine. Run below commands – 

$ sudo iptables-save > /etc/iptables/rules.v4

$ sudo vim /etc/rc.local

Add below line in /etc/rc.local file and then save the file by pressing Esc -> :wq.

/sbin/iptables-restore < /etc/iptables/rules.v4

Reboot the machine by typing below command – 

$ sudo reboot

This completes the configuration of iptables and NAT on ubuntu machine. 

Confirm the connectivity

Login to Contoso server VM over public IP. You can leverage Azure Bastion, Jump VM, or any other option for this. After login lets confirm if the traffic is flowing to ContosoDc VM through Ubuntu VM. For this I ran tracert command and output shows that the traffic does flow through ubuntu VM as shown below – [click to get better view] –



Similarly I could see return traffic from ContosoDC VM to ContosoServer VM was flowing through ubuntu VM – [click to get better view]



You should be able to take RDP between the VMs. Post RDP I checked the source in event viewer of both windows VM and I could see the source IP as ubuntu IP from which DNAT occurs.

Source IP of RDP can be viewed from below path in Event viewer - Event Viewer > Applications and Services Logs > Microsoft > Windows > TerminalServices-LocalSessionManager > Operational

Screenshot of ContosoDc machines – [click to get better view]



This concludes how an Azure Ubuntu VM can be configured as “IP Forwarder” to achieve Transitive routing in Azure VNET Peering.

What are the other ways to achieve transitive routing in Azure VNet peering?

What we did with Ubuntu is, we built IP forwarding out of it from scratch. This is readily available with leading firewall and routing devices such as PaloAlto, CheckPoint, Barracuda etc. The only difference is those devices will come with licenses cost. Ubuntu VM we built does not incur any licenses cost.

How to achieve Transitive routing in peered VNet using Azure Firewall?

This possible using Azure Firewall.

Deploy Azure firewall in Transit VNET. You just need to allow traffic from ContosoDC to Contoso Server VNET and vice versa as a Firewall Network rule. Configure the UDR on both VNETs with next hop to Firewall private IP and you are done.

Conclusion

Hope this article helped to design IP forwarder using simple Azure Ubuntu VM to solve your Transitive Routing needs.

Happy Peering!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts

Transitive Routing with 4 VNETs and Azure Firewall

Azure Virtual Machines – real world frequently asked questions – not easily answered.

Azure Migration frequently asked questions, not easily answered!

Azure VM disk encryption, what should be my approach!

Bypass onpremises firewall to RDP or SSH into Azure VM