Showing posts with label Azure Files. Show all posts
Showing posts with label Azure Files. Show all posts

Friday, April 17, 2020

Azure Function for generating Azure AD Token using .NET core

10 min to read.


Abstract

Development is a struggle, a continuous struggle. With serverless this struggle is becoming more and more imminent. Developers are still trying to get comfortable to serverless technologies.

Azure Function is one of the key serverless offering today on Azure. Azure Function is becoming backbone for many serverless and micro service based applications.

Recently after writing Azure AD token generator using .NET Core; immediately got a lot of requests to make this code run in Azure Function and here we are!

So in this blog we will create Azure Function to generate Azure AD token using .NET core.

Let’s go!

Architecture

Refer to below diagram [Click to get better view] for overall components used for running this code in Azure Function.



To publish the code of Azure AD token generator using .NET core to Azure Function, I plan to use Visual Studio. Publish operation from Visual studio to Azure Function is deployed as .zip file.

When we create Azure Function, creating an Azure storage account is also important and mandatory step. When we publish code from Visual studio, behind the scene Azure Files share is created in associated storage account and .Zip is actually copied to File Share.

This File Share will then mapped to Azure Function compute behind the scene and then used for running the actual application from Bin folder as shown above.

To get administrative access to runtime of Function keys are used. These keys are stored securely in Blob Storage which also get automatically created in the associated Storage account.

Making Azure Function async

First create Azure function project as shown below in Visual Studio – [click to get better view]



If you have seen the GitHub code of Azure AD Token generator using .NET core, you will observe that entire code is built using Async mechanism and token is also received in async way. When we create Azure Function project by default Azure Function startup method is not async. So let us make it async as shown below – [Click to get better view]



Then copy the code from GitHub and remove all Console specific methods. Instead make all methods to log information as shown in below screenshot – [click to get better view]



Publishing from Visual Studio

After completing code changes, it is time to publish to Azure Function. Either you can create new Azure Function from the Azure portal and specify the same during Publish method or simply create new Resource Group, Function from Visual Studio publish screen itself. In this tutorial, I am creating new Azure Function from Azure Portal as shown below – [click to get better view]



I have created TimerTrigger function. Then click on Publish button of your Azure Function project from Solution Explorer and make sure you select existing Function to create Publish Profile as shown below – [click to get better view]



Then continue further to publish to Azure Function.

Benefits of running Azure function from package file

Now in above screenshot of Creating profile you might have observed as checkbox – “Run from a package file”.
This is really important. This directly sets the Azure Function to run from package file only. Means your Azure Functions runs directly from a deployment package file in your Function App. By running function from package file you –

       1.      Reduces the risk of file copy locking issues
2.      Improves performance of Function App drastically
3.      Code start time is reduced
4.      Unwanted files are automatically removed from final deployment package.

My current Function app is based on Windows and .NET core therefore in Architecture diagram you might have observed below setting –

WEBSITE_RUN_FROM_PACKAGE = 1

This setting tells Function app to run from package file. You can also view this setting from “Application Settings” tab in Azure Function as shown below – [click to get better view]



Other ways of deployment

I can also write the code Azure function directly in Azure portal editor. However I have seen lot of issues in adding Nuget package reference in Azure function. Deploying from Visual Studio[for .NET code], VS code [all other languages including .NET] makes it easy in all respect; as all dependent assemblies, files get packaged in .zip.

When you deploy the Azure Function from Visual Studio with .zip package deployment; automatically your Azure Function will become read only and will show message as shown below –

Your app is currently in read only mode because you are running from a package file. To make any changes update the content in your zip file and WEBSITE_RUN_FROM_PACKAGE app setting.

Well, this is fine because anyways deployment is governed by Visual Studio. So we don’t need to make any changes in the code from portal. So Function becoming read only is fine. Rest of the platform specific features never become read only.  So I will definitely recommend to deploy the Azure function using VS or VS Code or using DevOps.

Verifying the .zip package and Azure File Share

After deployment, you can verify that .zip package is actually present in Azure File Share of the storage account and it is mapped to D drive on Kudu Console of Azure Function. See below - [Click to get better view]



Running the Azure Function and view the Azure AD Token

Run the Function and you should be able to view the generated Azure AD Token as shown below – [Click to get better view] –



I created TimerTrigger Azure Function to get Azure AD token on timely basis. This is because my next blog is based on Azure API Management monitoring which requires Azure AD token generation on timely basis from Function. You can also use HttpTrigger for Azure Function to generate Azure AD Token. So stay tuned for next blog soon!

Conclusion

Hope this article helped you to get Serverless Azure Function for generating Azure AD token using .NET core. Let me know your views in comments section below to improve and what are your thoughts on this approach.

Happy token generating!!

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefited, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.

Next Related Posts








Wednesday, September 4, 2019

Azure DevOps – Build and Release pipeline to Azure File Storage


Abstract

Azure File Storage is an ubiquitous service. It is so useful that I have hardly seen an enterprise not making use of it. Recently Premium tier of File storage is introduced with IOPS as 1,00,000 as opposed to standard File storage which used to offer 1000 IOPS only. This premium tier now even makes Azure File Storage natural choice for high performance demanding applications.

I have been architecting many scenarios and implementations where I used Azure File Storage for running web applications. In such a scenario your application binaries, DLLs, application files [or jar, war files in case Java] are present on Azure File Storage and this Azure File Storage is mapped as a drive to Azure VMs. This Azure VM then runs web servers likes Tomcat, IIS and maps their website path to the drive mapped using Azure File Storage. And this works awesome!

In today’s enterprise world DevOps has become a regular practice. Naturally when you plan to deploy web applications you use pipelines in Azure DevOps. In above scenario you would need to create Build and Release pipeline that will deploy your application to Azure File storage. Unfortunately, there is no default task exist in Azure DevOps that can publish the build output to Azure File Storage. This is what I am going to build in this post.

Let’s go!

Setting up pre-requisites

For this blog post I am going to use .NET Core language web application as my sample. It doesn’t matter if you use any other language-based example. So I already have a DevOps Organization created and in the same I have created one Project named as “MyNETCoreApp” as shown below –



It doesn’t matter which option you select for Agile process while creating project in Azure DevOps. Selection of Agile process while creation of project in Azure DevOps will matter in other important cases of setting up your business process. Not in Build and Release pipeline. But that is another blog on another day. Not today!

Next part is about creating the project n Visual studio [or in your favorite IDE] and pushing the same project in this current Azure DevOps Project Repo. I have it done already because I think this is straight forward and no step by step guide is required. So My project Repo looks as follows –




So we are all set for now to create Build and Release pipelines to release to Azure File Share.

Don’t believe on your eyes - Clearing up the confusion

When I asked my regular blog followers for the current topic most of them responded saying “Azure DevOps already offers a way/task to release the code to Azure Files.”. I said where? How and when did this happen? Then my dearest followers [most of them - not all] sent below screenshot to me [without Red and Green highlights]–



They were not wrong because they had just a given a quick look to the heading. Red highlighted Heading is perfectly fine and it will make you believe that Azure DevOps do offer a task to publish and release pipeline to Azure File Share. However if you focus on description of the task Green highlighted; you will see that Azure DevOps File Copy tasks do not publish to Azure Files but to Azure Blob and Virtual Machine only. Confusion because of names!

I am sure you know the difference between Azure Blob [https] storage and Azure File Share [SMB] storage. So moral of the story – we don’t have any tasks default provided in Azure DevOps to release the code to Azure Files share.

High Level Solution

Wednesday, July 10, 2019

Delete Azure File Storage Directory and files using PowerShell and secret sauce of recursion


Abstract


Come on guys!! Seriously?

After this successful post of Azure Virtual Machine – Frequently asked questions – not easily answered I got almost 54+ requests/ comments/ emails asking specifically for a blog post on sharing script for “Deleting Azure File Storage Directory using PowerShell”.

Not sure why people felt this is hard to write a PowerShell script. Well, this post is about the same topic and at the end of post you will know how to delete Azure file storage Directory and Files using PowerShell scripting.
You must have read the word in title “Secret sauce of Recursion”; about that later in the post below.

Lets go!

What is big deal in deleting Azure File Storage Directory and Files?


Well, you may think this as straight forward PowerShell. I also thought the same way until I actually tried for it.
As a best practice, before trying anything from PowerShell I always do the same task using Azure Portal. So for deleting Azure Files I went on to the portal. Created a storage account of type v2; and then created a File storage of 10GB as shown in the screenshot below. At this point the file share is empty.



Note – Focus on Blue box and blue underline; I wrote this blog at 2AM in the night. This is how I am getting attached to my blog after my wife.

Then I created a directory and uploaded few files within directory and few files directly in share and few directories inside the root directory and then again few files.

Note - If you have not understood above sentence; then blame 2AM time.

So my directory structure looks as below on Azure File storage share –



Yellow boxes are directories, blue boxes are files and green box is the actual file share inside which all of this is present. I tried to delete the directory wwwroot from the portal UI and I got below message.



This means I will have to delete each and every file present in the directory of my file share. If there is cascading of directories [directory within directory] then each of the directory will have to be made empty first and then delete each of the directory one by one and then the actual directory I want to delete.

This is big deal task and no wonder people are finding it difficult in PowerShell. As of now there is no single command by which you can delete directory including everything within it. So if I want to retain my file share but delete everything within it; I must empty all the directories.

What is my way out?


Well, you need to traverse through each of the directory and make it empty. So you might be thinking for for-each loop in PowerShell to do it. You are correct. But only for-each or for loop will not help you win the war. You need super power called as “RECURSION”. Write a recursive function in PowerShell and we should be able to win the war.

Why people find it difficult to write RECURSIVE functions?


People of my generation [people who started career between 2006 to 1012 year]who had started their programming career in C, C++, micro-controller programming will never find recursion difficult. Because the very first program we had build was Factorial number and it is product of all integers from 1 to n. For example factorial of 5 is 5 x 4 x 3 x 2 x 1  = 120.

This classic problem can be solved by recursion effectively.

Recursion is basic programming technique you can use in many languages including PowerShell, in which a function/ method calls itself to solve some problem. A method/ function that uses this technique is called as “Recursive function”. There are many problems that can be solved only by recursion.

To empty azure file directories we need recursion in PowerShell.

I have seen most of the new comers who starting their career in IT fields directly jump to Artificial Intelligence and Machine learning programming. More of less basics of programming may get missed like data structures, recursions etc.. I asked the year of experience to many people who demanded to write this PowerShell to delete Azure files directories and files and no surprise most of them were in the range of 4 to 6 years of experience.

Anyways no hard feelings!

Let us build PowerShell to delete Azure file storage directories and files recursively.

Define variable and create Azure Storage context


I am using latest Azure PowerShell module.

We need to define the variables for storage account details and then create the context for storage account. Context is an important steps as for every operation you perform against any of the service in Azure storage context should be passed in PowerShell commands.

#define varibales
$StorageAccountName = "YourStorageAccountName"
$StorageAccountKey = "YourStorageAccountPrimaryKey"
$AzShare = "kunalshare – you write your file share name here"
$AzDirectory = "LatestPublish"


#create primary region storage context
$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$ctx.ToString()

#Check for Share Existence
$S = Get-AzStorageShare -Context $ctx -ErrorAction SilentlyContinue|Where-Object {$_.Name -eq $AzShare}

Get the directory name and call for delete function


Now as per the directory hierarchy in above diagram; you can see that inside File Share I have a root directory called as “LatestPublish” and all other files/ directories are present inside it.
So I want to delete “LatestPublish” folder. So let us retrieve the reference for the folder and then call delete recursive function.

# Check for directory
$d = Get-AzStorageFile -Share $S -ErrorAction SilentlyContinue|select Name

if ($d.Name -notcontains $AzDirectory)
{
    # directory is not presetn; no action to be performed
   
}
else
{
    $dir = Get-AzStorageFile -Share $s -Path $AzDirectory   
    RemoveFileDir $dir $ctx #Calling function to remove directory.
}

Recursive function to remove directories


In above code of getting root directory we are calling a function named as RemoveFileDir. Let us write this function.

function RemoveFileDir ([Microsoft.Azure.Storage.File.CloudFileDirectory] $dir, [Microsoft.Azure.Commands.Common.Authentication.Abstractions.IStorageContext] $ctx)
{  
    $filelist = Get-AzStorageFile -Directory $dir
   
    foreach ($f in $filelist)
    {
       
        if ($f.GetType().Name -eq "CloudFileDirectory")
        {
            RemoveFileDir $f $ctx #Calling the same function again. This is recursion.

        }
        else
        {
            Remove-AzStorageFile -File $f           
        }
    }
    Remove-AzStorageDirectory -Directory $dir
   
}


In above function, we are receiving the root directory object to be deleted and storage context as parameter. Then we use for loop to traverse through the received directory object and see if there any more directories. If there is a directory found then we call the same function in recursion mode; until we reaches to last directory/ file present in the hierarchy. Once we reach to last item then we delete it.

Download entire source code

Download the entire source code of deleting Azure file directories from GitHub https://github.com/kunalchandratre1/DeleteAzureFilesDirectoriesPowerShell

Bonus tip

If you delete file share from the azure portal; everything within that gets deleted automatically and you don’t have to indulge yourself in to PowerShell. However if you want to delete selected directories and files from File share then PowerShell is the only way or do it manually from portal.


Conclusion

Hope this blog helped you t get working Azure PowerShell script for deleting Azure file storage directory and files recursively. Please feel free to add/ update your comments. Thanks.


Happy recursion!! 

A humble request!

Internet is creating a lot of digital garbage. If you feel this a quality blog and someone will definitely get benefitted, don't hesitate to hit share button present below. Your one share will save many precious hours of a developer. Thank you.