Using SAS, renew SAS and REST API to Upload large files to Azure blob storage in parallel and async

First of all thanks for overwhelming response to my earlier blog post related to upload of large files to azure block blob in parallel and async.

At the bottom of this post you will find the link for downloading the code sample.

In current post I will extend the same code library to support REST Api to perform azure blob upload. Also, as a best practice you should always use SAS (Shared Access Signature) for performing any operation on blob storage. Therefore in current post I will extend the code to support SAS as well.

Why we need to renew or extend Azure SAS –

Using SAS with blob storage is great option to provide one more level of security to azure storage operations however, anyone who gets access to SAS url can perform malicious operations. Therefore keeping SAS expiry time to minimum is always a good practice. But again, if we are performing upload of fairly large files then keeping SAS expiry to minimum will not help.

For example, let’s consider a scenario. Let’s say I will upload a file of size 100MB in azure blob storage using REST Api and SAS. As a best practice and already stated in my previous blog post of uploading large files to azure block blob in parallel, the files to be uploaded to azure need to be sliced in chunks.

Now if I have set the expiry time to 5 minutes. Of course upload of 100MB size will not be completed in 5 minutes and hence my block upload will fail in between.

Solution -

To continue to upload large files to azure block blob seamlessly after SAS expiry time, I need to renew the SAS token again. This is what exactly I am doing in this blog post and code sample.

So essentially, I check the response status code of every block upload. If response status code Created means current block is uploaded and continue to next. If the response status code is received as 403 Forbidden means the upload of block is failed due to invalid SAS. This is where I mark the particular failed block as Not Added.  Once the loop is over, I retrieve list of failed blocks and perform the same operation with new SAS. To perform same operation of upload with new SAS I am using popular and basic concept you also might know called as RECUSRSIVE FUNCTION.

To keep the SAS expiry to minimum I have kept the duration of 2 minutes only. After expiry the new SAS gets generated for another 2 minutes. The SAS I am generating here is Ad Hoc SAS. If you are in need then you can also generate Stored Access Policy based SAS and use in the application. If you need sample with Stored Access Policy Based SAS then please let me know through my contact mail id.

Thank you for reading.

Important – Please suggest your Feedback/ Changes / Comments to the article to improve it.

Full code sample is present here – http://gallery.technet.microsoft.com/Using-SAS-renew-SAS-and-06160a3a

Cheers…