Thursday, January 30, 2020

Azure VM Disk Encryption - Storage Side Encryption vs Azure Disk Encryption

11 min to read.

Abstract

Encryption is a vast and complex topic. No matter how much you try to make it easy; it turns into more complex subject. The encryption is very close to heart for security teams in any organization.

Especially for security people working for a Bank or Financial institutes, the encryption will always be favorite topic. The audit, compliance and security teams always tend to evaluate encryption in and out.

With more and more organizations moving their workloads to Azure Cloud; encryption becomes hot topic. The most common service used on Azure is Azure VMs.

I have been part of the discussions where “Encryption of Azure VMs and its disk” took more than 6 months to satisfy all queries of security teams.

As always, I don’t want every discussion to continue that long and this is what this blog post targets. In this blog I have added crucial information about “Azure VM Disk Encryption” that will help you to make decision faster and move toward the Azure journey quickly for large scale adoption.

I will TRY to explain the Azure VM encryption scenarios and common questions which are not provided by Vast documentation of Azure VM Disk Encryption present – here, here, here, here and here.

Before we go forward it is always best to start with FAQ on Azure IaaS Disk Encryption.

Lets go!

What to expect out of Azure VM encryption?

The topic of encryption vastly moves around two things –

1.      Encryption at rest
2.      Encryption in transit

For Azure VMs when we talk about encryption for Azure VMs, it is mainly applicable to “Encryption at rest”. The data stored on Azure VM disks should be encrypted in the major requirement of organizations. Apart from that other requirements are –

      1.       I want to use my secret keys for encryption
2.      When I backup VM, the backup should also be encrypted
3.      When I restore VM from backup; it should be encrypted
4.      When I perform DR replication for VM; it should be encrypted
5.      My VM stored data should not be readable unless decryption keys are provided
6.      If I download the encrypted Azure VM from Azure and provision in my on premises then data on VM should not be readable
7.      What is difference between Azure managed disk “Server Side Encryption” (SSE) and “Azure Disk Encryption” (ADE)? When to use what?

These are the common questions/ requirements I have seen most of the companies demand for Azure VM 
encryption. Let us see how Azure VM encryption options resolve it.

Encryption methods for Azure VM

Azure VM managed disks can be encrypted using two methods –
1.     Server Side Encryption
2.      Azure Disk Encryption

Server Side Encryption of Azure VM Managed Disks

Server side encryption [SSE] is default offering. All of your Azure VMs managed disks are always encrypted by default when they are stored on underlying storage. This is encryption at rest by the Azure itself. 

You don’t need any additional efforts to perform Server Side Encryption of Azure VM Managed disk. More importantly you can disable it as well. Server side encryption is not optional, and always provided behind the scene.

Azure Disk Encryption of Azure VM Managed Disks

Azure Disk Encryption [ADE] is optional. This method provides an extra layer of security over SSE. This encryption is performed at OS level of VM and hence there are many conditions where ADE is supported/ not supported. Where as SSE is always performed at backend storage level and has nothing to do with OS of VM being encrypted. 

So there are no non-supported scenarios for Server Side Encryption of Azure VM Managed Disks.

For Windows VM ADE is configured using BitLocker.
For Linux VM ADE is configured using DMCrypt.

What is fundamental difference in Server Side Encryption and Azure Disk Encryption?

Differences are best explained by diagrams. Refer to below diagram where SSE and ADE is performed in the context of Azure VM.



As you can see in the diagram, ADE is performed at VM OS level whereas SSE is performed at the storage level. All Managed disks of Azure VM are backed by Azure page blobs and this is where SSE is performed. As ADE is performed at OS level, we use tools such as BitLocker and DMCrypt.

In next section we will talk about common questions/ requirements talked about Azure VM Managed Disk Encryption and at last we will discuss the hottest topic ADE vs SSE and when to use what.

Can I use my own Key for Server Side Encryption and Azure Disk Encryption of Azure VM Managed Disks?

This scenario is called as Bring your own Key [BYOK] scenario.

BYOK – Also known as – Customer Managed Keys [CMK] – Can be used for SSE and ADE both. For SSE as of today it is in preview in some regions. CMK can be leveraged only if you use Azure Key Vault. You can’t bypass Azure Key Vault for CMK.

BYOK process for SSE or ADE is as follows –
        You bring your key in Azure Key Vault. We call it as Key encryption key [KEK].
        Azure Key Vault uses KEK to encrypt the Data Encryption Key[DEK] while stored in Key Vault.
        DEK is actually generated automatically internally, and used for SSE or ADE, and for actual encryption of underlying data at rest of Azure VMs.
        This scenario of using your own key, using Azure Key Vault is called as Customer Managed Key [CMK].
        Remember in CMK, your key is never used for actual encryption of the Azure VM disk stored data. Rather it is used to encrypt the Data Encryption Key. The DEK is the real key which encrypts the data stored on Azure VM disks.

Can I disable SSE for Azure VM Managed Disks encryption?

No. SSE is provided by default and you can’t opt out of it or disable it. Even if you don’t use CMK, Azure will always keep data encrypted based on system/ platform generated keys and data at rest is always encrypted.

If my Azure VM is encrypted using SSE and I download the VHD. Then using this VHD if we create a VM will it be encrypted and data on it will be non readable?

No. As soon as the data leaves the boundary of underlying storage, it is decrypted. Hence if you provision VM vhd or data disk vhd the data will be readable.

If my Azure VM is encrypted using SSE + CMK and I download the VHD. Then using this VHD if we create a VM will it be encrypted and data on it will be non readable?

No. As soon as the data leaves the boundary of underlying storage, it is decrypted. Hence if you provision VM vhd or data disk vhd the data will be readable.

Note – If you want to use SSE only as it avoid lot of operational overheads and fastest way to compliance; but worried what if someone downloads the VHD; then you can create Azure Custom Role in such a way that it restricts the download VHD completely. 

Then assign the custom role created to those users at subscription level who work on daily VM operations using Azure portal, CLI or PowerShell.

Contact me if you need such a custom role built in your azure subscription.

Can I opt out of ADE?

Yes. ADE is completely optional.

If my VM is encrypted using ADE + CMK and I download VHD, will it be encrypted?

Yes. The Data disk and OS disks will not be readable.

If my VM is encrypted using ADE + CMK, and I take backup of VM using Azure Backup vault. Will that VM backup be encrypted?

Yes. VM backup is also encrypted. If you restore such a VM from backup then restored VM is also encrypted using existing keys.

What is the biggest risks customers should be aware of using Customer Managed Key [CMK] for SSE or ADE with Azure Key Vault?

CMK is where customer brings their own key. If you use your key for SSE or ADE then management and lifecycle of key is in your hand.

If key is lost, means data is lost. There won’t be any recovery point/ option available, if the KEK used for encryption of DEK, is lost.

As a failsafe mechanism always enable Soft Delete on Azure Key Vault and never perform Hard delete on Azure Key Vault secrets, Keys and certificates.

What type of VM images are supported for SSE?

OS
Windows
Linux
Gallery image
Yes
Yes
Marketplace, like CIS Benchmarked images
Yes
Yes
Custom built using Sysprep or generalized images
Yes
Yes

Note – As SSE is performed at the backend store of managed disks; the image type of VM really doesn’t matter.


What type of Azure VM images are supported for ADE?


OS
Windows
Linux
Gallery image
Yes
Yes
Marketplace, like CIS Benchmarked images
Yes
No
Custom built using Sysprep or generalized images
No*
No*

Note - Custom images and marketplace images for Linux OS are supported for encryption on case by case basis.


Can I create template generalized image from an encrypted VM and use it further?

ADE VMs must be encrypted one by one, the generalization process breaks ADE, that’s not a supported approach.

Customers could reduce some time using automation tools to trigger the encryption process as soon as the VM provisioning completes and any possible OS configuration details (orchestrate the process), but at this moment it is not possible to create images with OS already encrypted using ADE, as mentioned the VMs should be encrypted one by one. 

What is the recommended approach for ADE?

  • Deploy the VM using an endorsed supported gallery images
  • Encrypt the VM prior installing any apps or performing customizations
  • Once the VM is encrypted then you can install apps and perform any needed customizations like hardening, Antivirus install, monitoring agent install and so on (making sure those customization will not break ADE pre-requisites).

My VM is ADE encrypted, will I be able to take individual Folder backups?

No. When VM is ADE encrypted then individual folder backup is not supported. You need to first decrypt the VMs and then take individual folder backups.

Above point is valid for Windows and Linux both.

When VM is ADE encrypted, always take entire VM backup.

When I have VM encrypted using SSE, and I take backup. Is my backup is also encrypted?

Yes.

When I have VM encrypted using ADE, and I take backup. Is my backup is also encrypted?

Yes.

When I restore VM from backup, which was encrypted using SSE or ADE, my newly restored VM is always encrypted?

Yes.

In which scenarios the data disks are formatted for ADE?

If you have a Linux VM already provisioned with Data disk having data stored on it and if you perform ADE [DM-Crypt], data disks will be formatted. Therefore take the backup first if you are already using VM and then perform ADE.

If you have a Windows VM already provisioned with Data disk having data stored on it and if you perform ADE [BitLocker], data disks will NOT be formatted. However, recommended to always take the backup first if you are already using VM and then perform ADE.

In any scenario, OS drive is never formatted.

If we add new disks to already encrypted VM, will it format existing encrypted data disks?

Whether your current VM is Linux or windows  and has encrypted data disks, then while encryption of newly added data disks never cause any format operation on data disks which are already encrypted.

For linux, Any data disks that are not encrypted earlier and you do encryption using ADE then always Format happens for the first time. For Linux if you want any disks not to be encrypted, then unmount it always before encryption, remount after encryption is complete.

For Windows, Any data disks that are not encrypted earlier and you do encryption using ADE then also Format never happens, as for Windows BitLocker works in background.

What are operational overheads for ADE?

        As custom image based VMs are not supported, total provisioning time of VM increases. Also all hardening steps are recommended to be performed post VM ADE encryption completion. This adds to time required for ready VM.

        For Linux, the data disks are always formatted when first time ADE encryption happens. So if VMs are already in use this proposes challenge.

        If VM has 2 disks out of which 1 need to be encrypted and other need is not to be encrypted; you have unmount non encryption required disks and mount them back after encryption. This process will be required to be carried out everytime new disk is added to VM.

        Individual folders can’t be backed up if VM is encrypted. You have to first decrypt VM and then take backup of individual folders. In this case you are forced to take entire VM backup even if it may not be business requirement.

        ADE encrypted VMs can be restored from backup only in the same region and subscription.

        During encryption operation if OS +data encryption is happening the minimum RAM required is 8GB. Post encryption RAM can be reduced, if business/ application demands lower VM configuration.

        Only RHEL 7 images with PAYG are supported. Existing BYOL for RHEL is not supported for ADE.

        For key vault accessibility, if VM is not having internet access [which is the case in most of the org] aditional network requirements - https://docs.microsoft.com/en-us/azure/virtual-machines/linux/disk-encryption-overview#networking-requirements

Can I use 3rd party encryption service?

You can, but it may not integrate well with native azure services such as Azure backup vault and site recovery DR solution.

ADE and SSE are well integrated with all native Azure services and recommended approach for disk encryption.

I want to use 3rd party KMS such Thales/ Gemalto for individual folder encryption. Can I do that?

You can certainly do that however please confirm from the 3rd party provider that encryption used for individual folders will work with other azure services such as –

  • Azure Backup for VMs
  • Site recovery services
  • Performance is guaranteed for encryption and decryption process.

What is the cost associated with SSE or ADE?

Both are free. You are charged only for Azure Key Vault operations, key storage if you are using CMK. If platform managed keys are used then there won’t be any charge.

When to choose between SSE and ADE?

ADE
SSE
This is most convenient to use and provides solution to most of the  compliance needs.
If you are finance org and goes through extreme stringent compliance audit then opt for this. Always verify if your Security teams are okay with SSE; if not then use ADE on top of it.
Operational overheads are almost zero in this approach.
This approach increases operational overheads drastically.
The temp disks of Azure VM is not encrypted. So if you app running on VM uses temp disk for any operation then that data will not be encrypted.
Temp disks of Azure VM is also encrypted.


Conclusion

Hope this article helped you to get answers you are looking for. If you have any such questions but not answered; add your comments and I will try to give answers for them.

Happy questioning!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Monday, December 2, 2019

Azure Migrate – Frequently asked questions – not easily answered

12 min to read.

Abstract

Migration are not new to humans. Migrations are happening since early days of human history. Almost every person on planet earth have migrated one or more times in their life. In today's world I see people are migrating for their Love and Marriages. It is mostly boys who migrate for their love and mostly girls migrate after their marriage. Love and Marriage both are complex! 

When you say you have done Love Marriage - that is a deadly complex combination!

In technology world also, migrations are not new. Migrations of applications is fundamental work to achieve core motive of migration – Betterment.

Workloads / applications will be going through Azure migration. Questions related to Azure migrate are unique and I see many enterprises get stuck just because they are not able to find the right answer.
This blog post dedicated to all those “real world frequently asked Azure migrate questions - not easily answered”.

I am strong believer in Diversity and Inclusion therefore - This FAQ is suitable for people who knows Azure and who don’t know Azure yet!

Lets go!

Note – This is second blog in the popular series of Azure VM Real world frequently asked questions – not easily answered.

Azure Migrate

I just love the Azure Migrate engineering team. They are really awesome guys and most talented. In technology field, believe it or not “Migrations are always complex”. Azure Migrate engineering team is making cloud migration tasks so simple with their hard work and dedication. Really impressed!

Azure migrate is fantastic tool to cater to all of your migration needs. The scope and scenario coverage is really awesome. Below are the tools that are present/ integrated in Azure Migrate which can be used for migration based on your scenario –



Below are the workload types supported by the Azure Migrate –
       1.    Servers – Virtualized, physical, other clouds to Azure
2.      Databases – almost all, Refer below screenshot in related questions below
3.      Data – Migrate to Hot, Cool and Archive tier of Azure Storage using Databox, Import/ Export
4.      Web applications – Migrate to Azure App Service
5.      Virtual Desktops – VDI migrations to Windows Virtual Desktop


I am not going to explain “what is Azure migrate?” We directly dive into questions in next section. As of today I have curated list of around 23+ questions based on experience which are other than official FAQ document

If you have any question that is still not answered, use “Contact Me” and I shall get answer for you related to Azure Migration. That’s a promise!

Wednesday, October 30, 2019

Assign and verify Microsoft Azure AD custom domain using Azure App Service Domains

10 minutes to read.

Abstract


Have you ever heard of the word Doppelganger? It means ghostly double or counterpart of living person. There is theory that says, In today’s world there are 7 people who look similar to you. There are 6.4 billions basepairs in Human DNA. So I don’t trust this theory. However humans made sure that when it comes to internet world, there is one thing always unique – domain names. Domain names are very critical and there have been many famous domain name battles on uniqueness. One of the famous domain name battle was “Microsoft vs. MikeRoweSoft”. What? You never heard of it? Go search the internet.

Domain names can make or break an identify for your business. Custom domain names are inevitable in today’s world. If you want to be successful with your business/ website then custom domain name is a must. Microsoft Azure services also need custom domain names so that instance of Microsoft Azure services names can be aligned with your business / company names.

Azure AD is central to any Microsoft Azure based environment. All subscriptions belong to Azure AD and Azure subscriptions related identity management is also managed through Azure AD. Any Azure AD created will always have form as “YourAADInstanceName.onmicrosoft.com”. So when you create a user in Azure AD the FQDN of Azure AD user will be “username@YourADInstanceName.onmicrosoft.com.” Example I have an Azure Active Directory named “SanganakAuthority.onmicrosoft.com” then user “kunal” will have FQDN as “kunal@SanganakAuthority.onmicrosoft.com”. This is definitely not a user friendly name. If I change it to kunal@SanganakAuthority.com then it appeals a lot.

So custom domain for Azure AD is good to have thing!

Best thing is you DON’T have to go for external domain registrars/ providers like GoDaddy, HostGator, Bluehost etc. Microsoft Azure provides domain names through Azure App Service domains and you can use it anywhere including Azure AD.

Let’s go!

Relationship in Domains Names, DNS Records, DNS Zones and equivalent Microsoft Azure services

This is important to know aspects such as Domain Names, DNS Zones and DNS Records concepts in general and in Microsoft Azure platform. Refer below important diagram.



Domain names are provided by domain name registrars. Owning the domain names gives you right to control the DNS hierarchy. These purchased domain names are hosted on DNS names servers in terms of DNS Records. The domain registrar has their own name servers or they also allow you to specify your own/ preferred name servers. You host the domain names on name servers as DNS Records like A records, CName record, Txt record and so on. The combination of Names servers and DNS records is called as DNS Zones.
For purchasing domain names Azure provides “Azure App Service domain” and for DNS Zones Azure provides service called as “Azure DNS”.

Sunday, October 13, 2019

Azure Cost Optimization - Effective use of Azure App Service and App Service Plans


Abstract


Azure cost optimization is becoming a reality and must do for every organization. Limiting cloud spend is top most priority for many organizations to continue the show. While Microsoft Azure drives organizations to perform “Digital Transformation” and “App Modernization”; the IT Heads are finding it difficult to keep the cost low for cloud services. This does not mean that Microsoft Azure do not offer you choices to bring your Azure cost down rather cloud approach is meant to bring your IT cost down. However if you are not doing smartly then obviously you are not getting most out of your Azure journey. I have seen mainly the ignorance from IT Heads, IT admins, Cloud Administrators and lack of knowledge [Most of the time I call it Laziness] causing much of the problems. Microsoft Azure gives you enough choices, ways and tools to make sure you optimize the Azure cost. Remember Azure Cost Optimization is a journey and you have to continuously monitor it.

As always, my dear [and of course lazy] followers keep asking to bring down Azure cost and what are the effective ways of doing it. For you guys [and girls] I have started a dedicated page for “Azure Cost Optimization”. On this page I will be adding the tricks and tips for Azure cost optimization. Link of this page can be found at the end of article or on the top.

Let’s go!

Why Azure App service for cost optimization?

How many of you have seen KeyNote of Satya Nadella held in July 2019 at Microsoft Inspire conference? If not go watch it. Refer to below screenshot –





It was stated that in next 5 years, 500 million apps/ applications will be created. When we talk about creating so many applications definitely major chunk will be web applications, REST API and web hosted applications. The question where will be those applications hosted. The service which is scalable, resilient and easy to configure will win the race. Azure App Service exactly fits into it. Azure App Service is a platform which makes it super easy for hosting web-based applications primarily. The current momentum of Azure App service is also amazing.


So obviously Azure App service is the primary way of hosting web applications and will increase exponentially in coming future. So in essence it is important for us to know how can we save cost when we host our applications on Azure App Service. 

The costliest resource on cloud – Compute

Microsoft Azure is made of 3 basic building blocks – Network, Storage and Compute. All services are built on top of these building blocks. For any computing, processing , analysis you need compute power. For Azure, “Compute” is the most costly resource as compared to Networking and Storage. If you save on Compute then you perform major cloud cost optimization.

In Azure App service, the compute, network and storage for your application is provided by Azure App Service Plans.

Understanding Azure App Service Plans