Azure : Blob Storage / Retrieval

This is a new post in a series of beginners articles on how to do things in Azure. This series will be for absolute beginners, and if you are not one of those this will not be for you.

You can find the complete set of posts that make us this series here :

http://sachabarbs.wordpress.com/azure/

This time we will look at how to use Azure blob storage for uploading things like files (images, word documents, whatever you like really).

Introduction To Blob Storage

What exactly is Blob Storage? Well it is actually very simple, it is Azure hosted storage that allows you to upload large amounts of unstructured data (typically binary data, so bytes) that may be shared publicly using http/https.

Typical usage may be:

  • Storing images
  • Storing documents
  • Storing videos
  • Storing music

A typical Azure blob service would make use of these components

image

Account

This would be your Azure storage account. You must create a storage account using the Azure portal, which you would do through the portal : https://manage.windowsazure.com (we will see more on this in just a minute)

Container(s)

These belong to Account, and are using to group blob(s). Each account can have unlimited number of containers, and each container may contain an unlimited number of blobs.

Blob(s)

Blobs represent the Azure hosted data that represent the originally uploaded binary data. It is the blobs that you would eventual end up loading when you share an Azure blob storage url.

There are 2 types of blob storage available

Block Blobs

These are binary blocks of up to 200GB, where you can upload 64MB at one time, Typically for a larger block, you would spit things up and upload them in chunks using multiple threads, and Azure will reassemble them making the available as a single blob.

Page Blobs

These can be up to 1TB and consist of a collection of 512 pages. You would set a maximum size when creating the page blob. I have not used these so much, and personally I think these are here to support other Azure features like Virtual Hard Drives (VHDs) which are stored as page blobs in Azure Storage.

 

Url Syntax

The actual blob url format is as shown below:

http://<storage account>.blob.core.windows.net/<container>/<blob>

 

How To Get An Azure Storage Account

The first thing you will need to do is create a storage account, this is easily achieved by using the portal. Go to the portal : https://manage.windowsazure.com, and then click new

image

Then pick storage, and go through creating a new Storage Account

image

 

Then once you have that you can open the newly created storage account, and click the dashboard, and that will show you the relevant connection strings which you may use in your application

image

Using The Storage Emulator

NOTE : If you just want to try things out without going through the process of creating a Storage Account, you can actually use the Storage Emulator, which you can do using a account connection string something like this:

<!-- TODO : This would need to change to live azure value when deployed -->
    <add key="azureStorageConnectionString" value="UseDevelopmentStorage=true;" />

Since I am writing the bulk of these posts on a train without any internet connectivity, I will be using the storage emulator in any demos in this post.

If you want to use the emulator, ensure that is running, and that you have enabled the storage emulator

image

image

 

Getting The Right NuGet Package

You will need to install this NuGet Package to work with Azure Blob Storage “WindowsAzure.Storage”

 

Creating A Container

This is done using the following sort of code

CloudStorageAccount storageAccount = CloudStorageAccount
    .Parse("UseDevelopmentStorage=true");
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("sachasContainer");
            

// Create the container if it doesn't already exist.
container.CreateIfNotExists();

container.SetPermissions(
                    new BlobContainerPermissions
                    {
                        PublicAccess = BlobContainerPublicAccessType.Blob
                    });

 

Uploading A File To Azure Blob Storage

This is done using the following code

CloudBlockBlob cloudBlockBlob = container.GetBlockBlobReference(
    "mandrillBlobUploaedToAzure.jpg");
cloudBlockBlob.Metadata["TypeOfImage"] = "Animal";

// Create or overwrite the "myblob" blob with contents from a local file.
using (var fileStream = System.IO.File.OpenRead(@"C:\Users\User\Pictures\mandrill.jpg"))
{
    cloudBlockBlob.UploadFromStream(fileStream);
    Console.WriteLine("Blob Url : {0}", cloudBlockBlob.Uri);
}

 

Listing All Blobs In A Container

This is done using the following code

container = blobClient.GetContainerReference("sachasContainer");

// Loop over items within the container and output the length and URI.
foreach (IListBlobItem item in container.ListBlobs(null, false))
{
    if (item.GetType() == typeof(CloudBlockBlob))
    {
        CloudBlockBlob blob = (CloudBlockBlob)item;

        Console.WriteLine("Block blob of length {0}: {1}", blob.Properties.Length, blob.Uri);

    }
}

Download Blobs

This is done using this sort of code

CloudBlockBlob blockBlob = container.GetBlockBlobReference("mandrillBlobUploaedToAzure.jpg");

// Save blob contents to a file.
using (var fileStream = System.IO.File.OpenWrite(@"C:\Users\User\Pictures\XXX.jpg"))
{
    blockBlob.DownloadToStream(fileStream);
}

Blob Metadata

Blobs support metadata via a dictionary which is available using the Metadata property which is a simple key/value pair.

 

How To Add Descriptive Metadata To A Blob

This is actually very easy, you just store more data in either a SQL Azure or SQL Azure table storage where you would include the blobs Url.  Job done

 

Deleting A Blob From A Container

This is done using this sort of code

blockBlob = container.GetBlockBlobReference("mandrillBlobUploaedToAzure.jpg");

// Delete the blob.
blockBlob.Delete();

Azure : SQL Azure

This is a new post in a series of beginners articles on how to do things in Azure. This series will be for absolute beginners, and if you are not one of those this will not be for you.

You can find the complete set of posts that make us this series here :

http://sachabarbs.wordpress.com/azure/

This time we will look at how to create a new SQL server database within Azure (in a later article we will loot using Microsofts NoSQL database “DocumentDB”)

Anyway so step 1, is to open up the portal

https://manage.windowsazure.com

 

From there you can click on “SQL Databases” and choose the “Create a SQL Database” hyperlink

image

From there you need to fill in your preferences within the wizard

image

image

Once this wizard has completed you will see a new database has been created

image

IMPORTANT : When the database is created, you will need to ensure that the standard port 1433 is opened. One of the easiest ways to do that, is to use the Azure portal to query the database (even though there is no tables in the database yet)

image

This little cheat will prompt you to open up the Firewall ports, which is great, lets just let the Azure portal do this work for us

image

So once the port is open, you will be redirected to an app in the browser (Silverlight app at present), that allows you to use your connection details you chose

image

When you successfully connect you should see something like this

image

Now there is no data in the SQL database yet. We could use this Silverlight app to add some tables, and data. However I would prefer to do that in Visual Studio, so lets go back to the portal, and open the connection strings, as shown below

image

image

We are interested in the ADO .NET one, where the part I have highlighted is the important part you need

image

SO grabbing the connection address to the Azure SQL server instance, lets connect via Visual Studio, and create a table

image

Once you have a connection in Visual Studio, lets create a new table using the context menu

image

When you are happy with the table, click the “Update” button which will push the changes to Azure. This is only a demo, for a real app you would likely have some sort of scripts, or would use the Entity Framework migrations facility to manage changes

image

image

So now lets check everything worked by connecting to the Azure SQL database from SQL server management studio.

image

As we can see we see the table we just created above

image

And lets also check the Azure portal query app

image

Yep, the table looks good, there is no data there yet, as expected for a new table. So lets now turn our attention to getting some data into the new table.

Lets use a new Entity Framework model to talk to the new SQL Azure database/table we just created.

image

I went with the defaults but you can choose what you like

image

This will result in a few files being created in the demo app, such as these, as well as an entry in the App.Config file to point to the SQL Azure database instance

image

And here is some code that will do some basic CRUD operation using the Entity Framework context that was created for us.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace SQLAzureTest
{
    class Program
    {
        static void Main(string[] args)
        {

            //insert
            using (var sachaAzureSQLEntities = new SachaAzureSQLEntities())
            {
                sachaAzureSQLEntities.Orders.Add(new Order()
                {
                    //note we are spelling this wrong so we can update it later
                    Description = "Buzz Lighyear toy",
                    Quanitity = 1
                });
                sachaAzureSQLEntities.SaveChanges();



                //select
                var order = sachaAzureSQLEntities.Orders.Single(
                    x => x.Description == "Buzz Lighyear toy");
                Console.WriteLine("Order : Id: {0}, Description: {1}, Quanity {2}",
                    order.Id, order.Description, order.Quanitity);

                //update
                order.Description = "Buzz Lightyear toy";
                sachaAzureSQLEntities.SaveChanges();

                var exists = sachaAzureSQLEntities.Orders.Any(
                    x => x.Description == "Buzz Lighyear toy");
                Console.WriteLine("Buzz Lighyear toy exists :  {0}", exists);

                order = sachaAzureSQLEntities.Orders.Single(
                    x => x.Description == "Buzz Lightyear toy");
                Console.WriteLine("Order : Id: {0}, Description: {1}, Quanity {2}",
                    order.Id, order.Description, order.Quanitity);



                //delete
                sachaAzureSQLEntities.Orders.Remove(order);
                sachaAzureSQLEntities.SaveChanges();

                Console.WriteLine("Orders count :  {0}", 
                    sachaAzureSQLEntities.Orders.Count());


            }

            Console.ReadLine();
        }
    }
}

And here is the results of this against the SQL Azure instance we just created.

image

Azure : Provisioning a Virtual Machine

This is a new post in a series of beginners articles on how to do things in Azure. This series will be for absolute beginners, and if you are not one of those this will not be for you.

You can find the complete set of posts that make us this series here :

http://sachabarbs.wordpress.com/azure/

This time we will look at how to create a new Virtual Machine using the Compute element of Microsoft Azure.

This again will be quite a screen shot heavy posting (its only a couple of the posts that will be like this, a lot of the subsequent ones will be much more codey, which is probably a good thing as there is a new version of the Azure portal waiting in the wings that has a different look and feel)

Anyway so step 1, is to open up the portal

https://manage.windowsazure.com

From there you will want to create a new Virtual machine, this can be done by clicking on the “New” button in the Azure portal.

image

From here you can choose what new thing you want to create in Azure. As this post is all about Virtual Machines, we will choose to create a Virtual Machine using the Azure compute element. This can be seen in the screen shot below. Azure comes with a whole load of pre canned images where the most common ones are shown in the list. You can also choose to examine more images, which is what we will be doing in  this post. So you would use the “More images” drop down item, as shown below

image

This will take you to a wizard that allows you to choose your Virtual Machine image and setup, and username/password

image

 

image

image

image

Once you have completed the wizard a new Virtual Machine, should be listed in the Azure portal for you. This is as shown below, for this posts example.

image

You can go into the Virtual Machine and view information about it, such as its public IP address, its DNS name etc etc

image

More importantly is the “Connect” button. When you click that a new Remote Desktop item will be created and downloaded for you. You can then use that to gain access to the Virtual Machine you setup.

One word of warning though is that the Status of the Virtual Machine MUST be “Running”, the status of the Virtual Machine can be seen in the screen shot above.

image

image

Here is me connected to a Virtual Machine (a standard Windows 2008 server), that I created another time.

image

Anyway hope that helps someone, another step along the Azure highway

Azure : How to publish a web site from VS2013 to Azure

This is the first post in a series of beginners articles on how to do things in Azure. This series will be for absolute beginners, and if you are not one of those this will not be for you.

You can find the complete set of posts that make us this series here :

 http://sachabarbs.wordpress.com/azure/

This time we will be looking at how to create and publish a web site to Azure using the Azure web portal (there is a new version of this coming soon, but for now it looks as shown below).

The first thing you will need is an Azure subscription, if you don’t have one of these you will need to get one. Once you have that you can sign into the portal.

Once you have signed into the Azure portal you simple need to create a new web site, I personally use the “Quick Create” option, and then you need to pick the Url and region for the new site. 

image

This will then create a new website place holder in Azure. You now need to go into the newly created place holder web site, which will take you into the Azure page for the new web site

image

From there you will be able to download the publish profile. This will be a single file which you should store somewhere where you will remember it.

image

Now you can go into Visual Studio 2013, and create a new website. I chose to go with one of the standard templates, where I simply altered the content a bit, such that I know the publishing step we will do later actually worked.

image

Once you are happy with your awesome web site you can right click and use the “Publish Web Site” menu.

image

This will then present you with some choices. You can use the “Import” option, and then browse to the publish profile settings file you downloaded in the Azure web portal step.

image

All you then need to do is follow the wizard through, and make sure you click “publish” at the end and you should then up with a lovely website hosted in Azure, as can be seen from the screen show below.

image

Not rocket science I know, but as I say this is for absolute Azure beginners.

PowerShell : Create MSMQ

AT work I use MSMQ I fair bit, and I also use NServiceBus a fair bit of late, which thankfully takes care of creating all the queues needed. But, for those time when you really need to make a bunch of queues, it can be a time consuming exercise, so I decided to see if my new found PowerShell skills could automate this process (remember I am still learning, so this may not be the best/most current way, in fact  know for Windows 8.1/Windows Server there is a later more groovy API. But for now this is what I came up with

[CmdletBinding()]
Param(
   [Parameter(Mandatory=$True,Position=1)]
   [string]$queueName,

   [Parameter(Mandatory=$True,Position=2)]
   [bool]$isTransactional,

   [Parameter(Mandatory=$True,Position=3)]
   [bool]$isJournalEnabled,

   [Parameter(Mandatory=$True,Position=4)]
   [string]$userName,

   [Parameter(Mandatory=$True,Position=5)]
   [bool]$isAdminUser
)


[Reflection.Assembly]::LoadWithPartialName("System.Messaging")



function printUsage() 
{
    Write-Host "Usage is CreateQueue.ps1 -queueName SomeQueuName 
		-isTransactional $true -isJournalEnabled $true 
		-userName mclocal\barbers -isAdminUser $true"
}


try {

    $fullQueueName = ".\private$\" + $queueName


    If ([System.Messaging.MessageQueue]::Exists($fullQueueName))
    {
        Write-Host($fullQueueName + " queue already exists")
    }
    else
    {
        $newQ = [System.Messaging.MessageQueue]::Create($fullQueueName, 
			$isTransactional)
        if ($isJournalEnabled)
        { 
            $newQ.UseJournalQueue = $True
        }

        
       
        if ($isAdminUser)
        { 
            Write-Host("ADMIN")
            $newQ.SetPermissions($userName, 
                [System.Messaging.MessageQueueAccessRights]::FullControl, 
                    [System.Messaging.AccessControlEntryType]::Allow)        
        }
        else
        { 
            Write-Host("NOT ADMIN")
            $newQ.SetPermissions($userName, 
                [System.Messaging.MessageQueueAccessRights]::GenericWrite, 
                [System.Messaging.AccessControlEntryType]::Allow)
            $newQ.SetPermissions($userName, 
                [System.Messaging.MessageQueueAccessRights]::PeekMessage, 
                [System.Messaging.AccessControlEntryType]::Allow)
            $newQ.SetPermissions($userName, 
                [System.Messaging.MessageQueueAccessRights]::ReceiveJournalMessage, 
                [System.Messaging.AccessControlEntryType]::Allow)
        }
    }
}
catch [Exception] {
   Write-Host $_.Exception.ToString()
  printUsage
}




















 

This will allow you to create a private queue of your choice of name, where you can also pick the following

  • Whether its a transactional queue
  • If journaling is enabled
  • If its an admin user queue

Hope it helps

Powershell To Clean Visual Studio Bin/Obj folders

One of my old WPF Disciple buddies William Kempf read my last post, and alerted me to yet another useful PowerShell Command to clean all the BIN and OBJ folders of a Visual Studio solution.

Here is Williams version

gci -inc bin,obj -rec | rm -rec -force

In Williams own words

That wipes out all of the “bin” and “obj” directories in the current directory and every subdirectory. Super useful to run in your workspace directory to get to a “clean” state, especially when someone messes up and there’s something that a Clean or Rebuild inside the IDE doesn’t catch.

For those of you reading that may not know, PowerShell supports command aliases, here it is rewritten again not using the aliases

Get-ChildItem -inc bin,obj -rec | Remove-Item -rec -force

NOTE : You should have this stored in a PowerShell file and place that file at the root of your solution (where the .sln file resides), and then run it when you want a proper clean (not the micky mouse one that VisualStudio does, and reports success too).

If you want to know a commandlet alias you can use the following PowerShell commandlet to find the full commandlet name:

Get-Alias gci

Which when run for this example will give something like this:

image

Powershell : Killing all processes of name

Now I am just starting with PowerShell, so I will likely come up with some ridiculously simple examples.

One of thing I really like about PowerShell is the ability to pipe things from one CmdLet  to another.

Imagine you want to get all instances of the the running process “notepad” and kill them.

This is easily achieved using the following code

Get-Process "notepad" | Stop-Process

 

Like  I say this is ridiculously simple, and hardy worthy of a blog post at all, but I aim to build a set of common tasks posts, and this will just form one of those.

So until next time