Archive

Windows Azure Platform

In certain circumstances, you might find yourself with a need to install SQL Server Express on one of your Windows Azure worker roles. Exercise caution here though folks: this is not a supported design pattern (remember, a restart of your role instance will cause all data to be lost).

It was however exactly what I needed for my scenario and I thought I’d share it in case it serves a purpose for you.

There are a couple of approaches you can take, of course, one of which is ‘startup tasks’ specified in the service definition files. However, these offered me limited configuration options because I needed to customise some of the command line arguments being passed to the installer based on values from the Role Environment itself.

The trickiest part was actually figuring out the correct command line parameters for SQL Server 2008 R2 Express, which to be honest wasn’t that fiddly at all. Here are the parameters you’ll need:

/Q/ACTION=Install/FEATURES=SQLEngine,Tools /INSTANCENAME=YourInstanceName
/HIDECONSOLE /NPENABLED=1 /TCPENABLED=1 /SQLSVCACCOUNT=”.\YourServiceAccount\” /SQLSVCPASSWORD=”YourServicePassword” /SQLSYSADMINACCOUNTS=”\.\ADMINACCOUNT” /IACCEPTSQLSERVERLICENSETERM S/INSTALLSQLDATADIR=”FullyQualifiedPathToFolder”

In the parameters above, we’re specifying a silentinstall with the /Qparameter, installing the SQL Database Engine and Management Tools (basic) with the /FEATURESparameter, setting the instance name, enabling named pipes and TCP, while setting service accounts and specifying the SQL data directory.

The next part then, is to actually build this as a command line and execute it in the cloud environment. How do we do this? Simples: we use System.Diagnostics to create a new Process()object and pass in a ProcessStartInfoobject as a parameter:

var taskInfo=new ProcessStartInfo
{
FileName=file,
Arguments=args,
Verb="runas",
UseShellExecute=false,
RedirectStandardOutput=true,
RedirectStandardError=true,
CreateNoWindow=false
};
//Start the process
_process=new Process(){StartInfo=taskInfo,EnableRaisingEvents=true};

For good measure, we’ll also redirect the standard and error output streams from the process so that we can capture those out to our log files:

//Log output
DataReceivedEventHandler outputHandler=(s,e)=>Trace.TraceInformation(e.Data);
DataReceivedEventHandler errorHandler=(s,e)=>Trace.TraceInformation(e.Data);

//Attach handlers
_process.ErrorDataReceived+=errorHandler;
_process.OutputDataReceived+=outputHandler;

Then, we’ll execute our task and ask the role to wait for it to complete before continuing with startup:

//Start process
_process.Start();
_process.BeginErrorReadLine();
_process.BeginOutputReadLine();

// Wait for the task to complete before continuing...
_process.WaitForExit();

Stick all of that into a method that you can re-use, and don’t forget to add parameters called fileand args(strings) that contain the path to the SQL Server Express installation executable and the command line arguments you want to pass in.

How to build your command line argument

If you’re wondering why I didn’t hardcode my command line options, it’s because up in Azure, the standard builds for web and worker roles don’t come preloaded with any administrative accounts – you have to specify those during design time. I actually ‘borrow’ the username of the Remote Desktop user (which is provisioned as an administrator for you when you ask to enable Remote Desktop).

I actually end-up with this quick-and-dirty snippet:

string file=Path.Combine(UnpackPath,"SQLEXPRWT_x64_ENU.exe");
string args=string.Format("/Q/ACTION=Install/FEATURES=SQLEngine,TOOLS/INSTANCENAME={2}/HIDECONSOLE/NPENABLED=1/TCPENABLED=1/SQLSVCACCOUNT=\".\\{0}\"/SQLSVCPASSWORD=\"{1}\"/SQLSYSADMINACCOUNTS=\".\\{0}\"/IACCEPTSQLSERVERLICENSETERMS/INSTALLSQLDATADIR=\"{3}\"", username,password,instanceName,dataDir);

So, ultimately, you’ll then want to wrap all of this up in to your role’s OnStart() method. Include a check to see whether SQL Express is already installed, too.

And, if you’re stuck trying to debug what’s going on with your otherwise silent installation, SQL Server Setup Logs are your friend. You’ll find them by connecting to your role via Remote Desktop and opening the following path:

%programfiles%\Microsoft SQL Server\100\Setup Bootstrap\Log\

Enjoy!

I’ve spoken with a lot of developers recently who haven’t yet adopted the Windows Azure platform, often because they think the process is difficult, time-consuming or requires some kind of advanced ninja training to get up and running.

This video will show you that it can be done in 90 seconds or less, without writing a single line of code!

In the screen cast, I’ll show you how to create a new Azure project in Visual Studio 2010, add a web role to the project, create a deployment package, upload it to the cloud, and then view it running in the cloud.

Before you begin though, head over to the Windows Azure site, and make sure you’ve activated a Windows Azure subscription. If you’re an MSDN or BizSpark subscriber, you get a free basic subscription anyway which is great for this demo. If you’re not, don’t worry, because until June 30th, Microsoft are giving you a free trial, too. Just get started at http://www.microsoft.com/windowsazure/free-trial/.

It’s seriously easy to do, so what are you waiting for – get going! :)


EDIT 30/03/2011:
In this video, I’ve shortened the “deployment” sequence to fit within the timeframe, but it should be noted it is normal for this part of the process to take anywhere between 15-30 minutes (while the Azure platform does what it does to spin up the resources it needs to run your solution). Thanks to all the watchers who pointed out that this fact was probably worth mentioning! :)

As always, feedback is appreciated and welcome.

I gave a talk at Microsoft BizSparkCamp in London on March 25th, and I thought I’d follow that up with a blog post summarising some of the main benefits of deploying your next solution on Windows Azure. These reasons all form part of what I call the “Lure of Azure”.

So here’s a taste of eight of the reasons I broadly covered in the talk:

Reason #1: Financial
Building, deploying and maintaining a Windows Azure solution is likely to cost you far less than acquiring and maintaining your own physical infrastructure. If high availability and redundancy are important to you, I can pretty much guarantee you can’t do it cheaper or quicker than with the Azure platform.

Reason #2: Forget hardware
Very few applications actually need to be run on dedicated physical hardware (“Co-location” etc). Windows Azure abstracts away all the hardware and gives you a platform upon which you deploy code only, and through configuration files, you can determine how much or how little of the resources will be available to your application. Let Windows Azure take away the strain of worrying about load-balancing and redundancy, as all that’s taken care of for you.

Reason #3: Consolidation
The Windows Azure platform allows you to consolidate all your logical services under one account, with one control panel. That means you can instantly provision SQL Azure databases, and make use of infinitely scalable storage resources on-demand from a single point of administration, quickly and easily. Managing Windows Azure applications that utilise SQL Azure, Windows Azure storage and Compute is orders of magnitude easier than maintaining the equivalent physical infrastructure yourself.

Reason #4: Scale up (and down)
If you’re going to buy a physical infrastructure, chances are you’ll over-specify and end up with a lot of spare capacity because you build-in a lot of what you don’t need all the time… I.E. you build to accommodate ‘peak’. That means you’d be paying for what you don’t need (or use) most of the time with any physical infrastructure. Windows Azure solutions can be scaled-up and down on demand, meaning you only ever pay for capacity when you need it.

Reason #5: Build flexibility
If you build on physical infrastructure, when you need more of something (or less, for that matter), you end up messing about with hardware. If you lease (as many of us do), then that means contract variations, change requests, maintenance periods and perhaps even down-time but generally always cost. The Windows Azure platform lets you do all this stuff on-demand, with ease.

Reason #6: Better global reach
Locating a physical data centre in one territory is one thing, but if you want truly global scale it pays to geographically distribute your resources to other territories as well. Co-location is an expensive way to do this, and then you have to think about how you’re going to replicate your data between all your data centres yourselves and the bottom line is: that’s tricky to say the least. Locating resources globally in Windows Azure is as easy as point-and-click.

Reason #7: If you’re a .NET house already, it’s even easier
Using Visual Studio 2010? Know C#? Most of your existing code can run in the cloud immediately with just a few minor tweaks. Go download the SDK and start today.

Reason #8: Flexibility to utilise all, or a part of the platform
Fed-up maintaining your own SQL database cluster? Running out of resources locally? Hosting company charging you too much for a SQL server database? Bung your database on SQL Azure and leave your app where it is. You can use the SQL Azure Migration Wizard to move your databases over to the cloud, then it’s just a simple matter of changing your connection strings in your code. Show me a simpler way of creating a triple-redundant SQL server instance!

There are many other reasons, of course, and I could expand on any of these along the way but the point of this post was to just get you thinking about some of the key advantages by summarising some of the points I discussed in my talk. Feel free to post comments!

A little while ago, I came across an excellent article by Maarten Balliauw in which he described a project he was working on to support FTP directly to Azure’s blob storage. I discovered it while doing some research on a similar concept I was working on. At the time of writing this post though, Maarten wasn’t  sharing his source code and even if he did decide to at some point soon, his project appears to focus on permitting access to the entire blob storage account. This wasn’t really what I was looking for but it was very similar…

My goal: FTP to Azure blobs, many users: one blob storage account with ‘home directories’

I wanted a solution to enable multiple users to access the same storage account, but to have their own unique portion of it – thereby mimicking an actual FTP server. A bit like giving authenticated user’s their own ‘home folder’ on your Azure Blob storage account.

This would ultimately give your Azure application the ability to accept incoming FTP connections and store files directly into blob storage via any popular FTP client – mimicking a file and folder structure and permitting access only to regions of the blob storage account you determine. There are many potential uses for this kind of implementation, especially when you consider that blob storage can feed into the Microsoft CDN…

Features

  • Deploy within a worker-role
  • Support for most common FTP commands
  • Custom authentication API: because you determine the authentication and authorisation APIs, you control who has access to what, quickly and easily
  • Written in C#

How it works

In my implementation, I wanted the ability to literally ‘fake’ a proper FTP server to any popular FTP client: the server component to be running on Windows Azure. I wanted to have some external web service do my authentication (you could host yours on Windows Azure, too) and then only allow each user access to their own tiny portion of my Azure Blob Storage account.

It turns out, Azure’s containers did exactly what I wanted, more or less. All I had to do was to come up with a way of authenticating clients via FTP and returning which container they have access to (the easy bit), and write an FTP to Azure ‘bridge’ (adapting and extending a project by Mohammed Habeeb to run in Azure as a worker role).

Here’s how my first implementation works:

A quick note on authentication

When an FTP client authenticates, I grab the username and password sent by the client, pass that into my web service for authentication, and if successful, I return a container name specific to that customer. In this way, the remote user can only work with blobs within that container. In essence, it is their own ‘home directory’ on my master Azure Blob Storage account.

The FTP server code will deny authentication for any user who does not have a container name associated with them, so just return null to the login procedure if you’re not going to give them access (I’m assuming you don’t want to return a different error code for ‘bad password’ vs. ‘bad username’ – which is a good thing).

Your authentication API could easily be adapted to permit access to the same container by multiple users, too.

Simulating a regular file system from blob storage

Azure Blob Storage doesn’t work like a traditional disk-based system in that it doesn’t actually have a hierarchical directory structure – but the FTP service simulates one so that FTP clients can work in the traditional way. Mohammed’s initial C# FTP server code was superb: he wrote it so that the file system could be replaced back in 2007 – to my knowledge, before Azure existed, but it’s like he meant for it to be used this way (that is to say, it was so painless to adapt it one could be forgiven for thinking this. Mohammed, thanks!).

Now I have my FTP server, modified and adapted to work for Azure, there are many ways in which this project can be expanded…

Over to you (and the rest of the open source community)

It’s my first open source project and I actively encourage you to help me improve it. When I started out, most of this was ‘proof of concept’ for a similar idea I was working on. As I look back over the past few weekends of work, there are many things I’d change but I figured there’s enough here to make a start.

If you decide to use it “as is” (something I don’t advise at this stage), do remember that it’s not going to be perfect and you’ll need to do a little leg work – it’s a work in progress and it wasn’t written (at least initially) to be an open-source project. Drop me a note to let me know how you’re using it though, it’s always fun to see where these things end up once you’ve released them into the wild.

Where to get it

Head on over to the FTP to Azure Blob Storage Bridge project on CodePlex.

It’s free for you to use however you want. It carries all the usual caveats and warnings as other ‘free open-source’ software: use it at your own risk.

If you do use it and it works well for you, drop me an email and it’ll make me happy. :)

Follow

Get every new post delivered to your Inbox.

Join 357 other followers