Let's rokk! [Tudor Cret's blog]

June 14, 2011

10 reasons to use Azure for your cloud apps

Filed under: Windows Azure — Tudor Cret @ 11:09 am
Tags: , ,

1. Familiarity of Windows

Based on Windows you can write .NET apps using C#/VB/C++ and ASP.NET/MVC/Silverlight.

Easy to migrate existing Windows applications.

2. 64-bit Windows Virtual Machines

Each instance of the app running in its own VM on the 64-bit Windows Server 2008 operating system. The hypervisor on which they run is designed specifically for the cloud. You don’t have to supply your own VMs or deal with managing and maintaining the OS because apps are developed using Web role instances or worker role instances that run in their own VMs

3. Azure SDK

You can run locally (on your PC) when developing and debugging an application and then move it to the cloud.

4. Scalability and flexibility

Using Azure, you can easily create applications that run reliably and scale from 10 to 10 thousand or even 10 million users — without any additional coding. Azure Storage provides scalable, secure, performance-efficient storage services in the cloud.

5. Cost benefits and pricing model

No costs for building and/or expanding on-premises resources, pay as you go model.

6. Data Center in the cloud

Relational database engine in the cloud offered by SQL Azure.You get high availability and reliability with redundant copies of your data and automatic failover.

7. Support resources

Because Azure uses the same familiar tools and technologies as other Windows platforms, you can take advantage of the well-established support structure within Microsoft and company-provided resources. Strong communities and world wide forums.

8. Interoperability

You can develop hybrid applications that allow your on-premises applications to use cloud services, such as the cloud database and storage services. Communications services work between on-premises applications and the cloud, as well as mobile devices (HTTP,XML,SOAP,REST). SDKs for Java,PHP,Ruby.

9. Security

Windows Azure AppFabric provides a powerful mechanism to secure your application and your communications to the app.

10. Something for everyone

Windows Azure can benefit hosting providers, ISVs, systems integrators, and custom software developers. Hosting providers can expand their services to areas where they don’t have existing infrastructure and add new services without more infrastructure investment. ISVs can use Azure to create, deploy, and manage Web apps and SaaS without large capital expenditures, and they can scale those applications more quickly and cost effectively. Systems integrators can take advantage of Azure’s ability to work with existing on-premise infrastructures. Custom software developers can create software solutions for customers who can’t afford the costs of in-house development, including hardware costs, and they can deliver their applications to customers as services without building and maintaining an expensive data center.

Advertisements

January 25, 2010

Behind ciripescu.ro, a Windows Azure cloud application (part 1)

Filed under: Technologies — Tudor Cret @ 2:18 pm
Tags: , , , , , , ,

Intro

January 4th marks an important step towards enabling Microsoft’s customers and partners, to build and grow businesses on the Windows Azure platform. Microsoft announced that upgrade from Community Technology Preview (CTP) accounts of the Windows® Azure™ platform (i.e., Windows Azure, SQL Azure and/or Windows Azure platform AppFabric) to paid commercial subscriptions is available. At the moment just for a list of countries, list in which Romania is not included. At least until February 1, 2010 the CTP accounts can be used. After this date the storage accounts will become read-only.      

The good news is that finally we don’t have a CTP version anymore. The bad news is that it is no longer free. And it costs. How much? Just watch the pricing list here. And before knowing the total…take a pen and a paper or try TCO and ROI calculator. In our case for ciripescu.ro, one worker role and one web role we’ll start from a minimum of $50 per month.      

But what services we get? In three words: Availability, Durability, Scalability. That means:      

  • Data from the storage (I’m not speaking about database, I’m talking about storage…) will always be available. Data is replicated across 3 nodes, at least. No request to the storage should fail. Also the uptime for my workers and web roles is around 99.99%. And this because at least 3 machines serves a worker or web role.
  • Data from storage is never lost. As I said data is replicated over 3 nodes. At least one node Is available and durable to serve the requested data.
  • I can scale my application as easy as clicking on a button and using a pay-as-you-go service. Et voila, I can add a new role to my application. With a click I have a new machine working for me, which means no hardware, no administrative tasks, etc.

Of course, these are not all the benefits. More can be found here.      

The purpose of the current post is to make a summary of what is behind ciripescu.ro a simple micro-blogging application developed around the concept of twitter. This platform can be extended and customized for any domain.      

So our intention is to develop a simple one-way communication and content delivery network using Windows Azure. Let’s define some entities:      

  • User –  of course we can do nothing without usernames, passwords and  emails J Additional info can be whatever you want, just like an avatar, full name etc.
  • Cirip – defines an entry of maximum 255 characters, visible to everybody. Users content is delivered through this entity
  • Private Message – a private message between two users
  • Invitation – represents an invitation a user can send to invite more other users to use the application. 

I won’t enter in more details about what the application does, you can watch it in action (at least until the end of January L) at www.ciripescu.ro. For the next sections I will concentrate on how the application does all its functionalities. Configurations and deployment aren’t so interesting since they are exposed to any hello world azure application video or tutorial.      

High level view

First let’s take a look at the architecture:      

High level design of Ciripescu.ro

  

The webrole is what people traditionally view as a web application. VS project contains everything a normal ASP.NET web application would: aspx pages, user controls, web services, code behind (including business layer or data access layer), etc. In fact it can be converted to a normal web application with two clicks from the context menu. It may be even an ASP MVC application. All HTTP requests are served by Windows Azure to one of these Web Roles. I’m saying “one of them”, because scaling up such an application involves running for Web Roles, and possibly more Worker Roles. If one web role has “enough” requests to process then a new incoming request will be handled by one of the other web roles.  With just one click you can have more web roles and more worker roles. Just one click….and a bigger pay bill at the end of the month. Yes, but…if you have your own datacenter? More web roles and worker roles means…electricity, hardware and human force.  And hardware is part from the companie’s actives…      

The worker role can be best compare to a windows service. It’s an application that runs in the cloud and performs tasks that are not suitable for a web application. Anything that takes too long to process and would unreasonably delay the loading of a page should clearly execute asynchronously and out of the normal IIS pipeline. For instance ciripescu.ro sends all its emails using such a worker Role. Why? Well, as most of the other social networking sites, ciripescu.ro has the ability to import contacts from a yahoo messenger account for instance and then lets the user select which of them he wishes to invited. This means that with one click a user could make ciripescu.ro send out a few hundred emails. Such a task could take between a few minutes, up to an hour. Obviously the user can’t wait that long for the page to load. Another example: let’s say I have to send a SMS message from my web application and sending SMS through a SMS gateway takes more than 10 seconds. Et voila, I already have the worker; it should do the job, while the user is free to “walk” to the rest of the application. So we can have as web roles any web application(website, ASP.NET MVC application) and as worker roles  any class library project(C# or VB). I don’t see a reason for which we wouldn’t have multiple worker or web roles written in different languages. Now it would be interesting to see a mix of worker and web roles each one developed using different platforms, PHP, Java, .NET that works together in the same application in the cloud. What would mean all that? That Azure is flexibile and not dedicated only for .NET ? It’s not tested yet that a ASP.NET MVC web role can use a Java worker role, but it’s a challenge for the future time. Anyway Azure is a programatically flexibile. You can easily integrate the existing web applications with new ones. Also integration with “on-premises” applications can be easily done via Bus Services. More about Bus services and Windows Azure flexibility can be found on Windows Azure training kit or just watching videos from PDC  2009. 

More available in part 2, here.

Behind ciripescu.ro, a Windows Azure cloud application (part 2)

Filed under: Technologies — Tudor Cret @ 2:14 pm
Tags: , , , , , , ,

 

Let’s go deeper(1) – web role and storage

As the high level design diagram suggests, the communication between web roles and worker roles is done through the various features of the Storage account, such as Queues. Queues are normally used to send messages/tasks between roles, as they implement all the safety features needed for asynchronous communication between multiple applications, running of different servers. This includes locking and recovery in case one of the roles crashes while still processing a message. Roles are automatically restarted when they crash, and items popped from the queue are made visible again. In fact this is the beauty of Windows Azure, from the perspective of the software developer: you write the code as if you only run one role and the cloud will make it work automatically with any number of roles, to infinitely scale it up. The beast is that Azure Tables are somehow limited, because there are no sorting or grouping options. But of course, there is SQL Azure (the classical SQL relational database – cloud version). Yes but sometime even at a relation database you make some denormalization for speed purposes. So it would have been nice to have at least sort option for Azure Tables.

While message passing between roles is mainly done using Azure Queues, other information are still passed by the traditional way: shared access to resources such as a database. Azure Store offers two different types of non-relational cloud based storage: Blobs (that store entire files) and Azure Tables (that store entities).

Ciripescu.ro uses Azure Tables to store all its business entities, instead of a classic SQL Database. We are still talking of business objects, a business layer and a data access layer, as you would in any other application. The only difference is that the data access layer stores the objects in a non-relational storage, designed to be using in a cloud environment. What makes Azure Tables so special is that it allows you to have very large tables that are still searchable very fast. In Ciripescu’s case, the table containing messages sent between users (Cirips) could have millions of items. Querying a sql table with a few million items can take several seconds. If we think of Twitter we would realize that a few million is more like a joke. How do you search among a table with a billion entities? The answer is simple: split that table on a cloud of servers, by using a relevant partition key and either only search that one partition where the object is, or search all them, but in parallel. This is exactly what Azure Tables does: for each object you define a partition key that will allow Windows Azure to transparently split the table, and a row key that allows fast searching of data in a single partition.

User entity

A business object would have to define those two fields and map relevant properties to them. Take note of the User class from ciripescu.ro, that inherits from TableStorageEntity (that defines the said properties) and maps the username as PatitionKey and String.Empty as row key, through its constructor. The TableStorageEntity class is defines in the Windows Azure SDK.  

In order to query an entity from Azure Tables, one must first create a DataServiceContext. This is a class that must inherit TableStorageDataServiceContext, which in turn inherits from the LINQ class DataServiceContext. Here, the conceptual object model looks like the one from NHibernate, but it’s not the same thing.  

User entity objects retrieval

Everything else is simple LINQ: the programmer creates a DataServiceQuery object for each Table and makes LINQ queries on it:  

var users = from u in Users select u;  

foreach (User u in users) { ….}   

This is how a data access layer class would query items from the storage:  

Quering items from the storage

Inserting, updating and deleting objects is done in the same way LINQ programmers are used  to. Let’s not forget that the data access clases from Azure SDK are build on LINQ classes:  

CRUD operations on User entity

The other entities in the application are built using the same object model.  

Now let us go to the main advantage of a social networking platform made on cloud based technology: the table that holds all the messages (Cirip). Social networking websites can get very popular. Think of tweeter: millions of tweets per day, and they have to not only be stored somewhere but also queried. How do you query a table with 1 billion entries? Azure Tables is the answer. Lets take a look at our Cirip table. Let’s assume two users, Tudor and Pitagora, had the following message exchange:  

Tudor: This is the first post on ciripescu   

Pitagora: hello world  

Pitagora: Windows Azure rocks  

Pitagora: What a good day for a Microsoft presentation  

Tudor: A barbeque would go better 🙂  

Tudor: I want a vacation  

PartitionKey RowKey Content
pitagora 9 What a good day for a Microsoft presentation
pitagora 12 Windows Azure rocks!
pitagora 20 Hello world
tudor 7 I want a vacation
tudor 8 A barbeque would go better 🙂
tudor 25 This is the first post on ciripescu

The PartitionKey and RowKey are two mandatory fields for any entity stores in Azure Tables. The programmer has to chose what he wants to store there carefully because all the power of the cloud depends on these two fields. Windows Azure splits tables in different partitions and stores them on different nodes, using the PartitionKey field. For redundancy and speed, each partition has 3 copies in the cloud. All partitions and copies of them are managed transparently by the cloud. The user doesn’t have to know of them are there. His only control over this process is the choice of PartitionKey. Entities with the same PartitionKey will belong to the same partition and be stored on the same node. The fastest possible query is the one that only searches in a single partition, so this choice depends from application to application. One has to ask: what is the most frequent query my application will do? In the case of ciripescu.ro, which is a microblogging application, our answer layed of the blog profile of each user. We decided that displaying all messages of a single user has to be the fastest query, so we chose the sender’s username as PartitionKey. In the example above messages from pitagora, can be stored on a different node then those from tudor. The order in which they are listed in the table also suggests that. Windows Azure sorted those messages by PartitionKey and RowKey.  

The second field, RowKey, is similar to the primary key in a SQL Table. Entities are index by it, and the fastest query done in a partition is one where the PartitionKey is the search criteria. In the context of Windows Azure, the Rowkey has another very important function: sorting. You have to remember that Azure Tables is not a relation database. It’s just of storage of entities. That means that you can’t run complex queries that involve counting, grouping and sorting. The entities are sorted ascendant by RowKey, and there is no way to change that. So in the case of the Cirip table, we want the latest Cirip to be displayed first. We chose DateTime.Max – DateTime.Now as a RowKey. The reason why I didn’t use real values in the example is because these are 11 digit numbers. The only thing I kept in the example is their order: the lastest message will always have the smallest RowKey.  

Using a non-relational database has some disadvantages too. Besides the lack of sorting and grouping capabilities sometimes you feel that you need relational entities. Because you will write more code at DAL layer. You can’t build :  

Select c from Cirip c, User u, Urmarire urm Where c.PartitionKey = u.PartitionKey and  

u.RowKey = f.RowKey and urm.PartitionKey = ‘current user’  

Anyway why Azure Tables and not SQL Azure? Because in our case is faster when searching. Also at the beginnings of Windows Azure there was not an SQL Azure. There was some kind of storage but not-relational. After some months and based on the feedback from developers Microsoft announced SQL Azure(summer 2009) which officially was released in the fall of 2009.

Worker roles and queues exposed here.

Behind ciripescu.ro, a Windows Azure cloud application (part 3)

Filed under: Technologies — Tudor Cret @ 2:13 pm
Tags: , , , , , , ,

Let’s go deeper(2) – worker role and queues

As mentioned above a worker role can be compared with a windows service. A little thing that sits quiet and it waits for its job. You tell it what to do, it knows how, you give it some input or not and it does its job. Also a worker can be seen as a background unit ready to execute a task. It depends on each one imagination and needs on how worker roles are used. Currently ciripescu.ro uses a worker role for email sending. It represents a “smart” email sender. Why smart? First of all we import a list of yahoo contacts to which an invitation has to be send. A person has 200 or 300 contact emails in his/her list. You can’t block the user until you send those emails. Also if you have online 10 users then you’ll have to send 2000-3000 emails in the same time. But what are doing if you have 500 users in the same time?  Also another problem appears. Yahoo for example banns you, if you send a large amount of emails per minute from the same email addres, even if you sign that emails. It sees that emails as spam. To avoid all these in the worker role we’ve built a scheduler for emails sending, of course giving priority to urgent mails such as ‘Welcome email’  or ‘Password reset. We’ve used queues to “tell” the worker the recepient(s) and the body of that email. More about Queues and communication can be found in Windows Azure training kit and Windows Azure SDK.

Azure Queues are a wonderful mechanism. They allow the communication between process threads or roles and they are based on a producer-consumer system. I can say that somehow they resolve the concurency problems of a distributed system. Ciripescu.ro uses one queue for sending urgent emails and another one for the rest of the emails, not urgent – that are sheduled upon a timetable. In the figure below you can see defined queues and how a new item (invitation message) is added to the queue, in web role:

Adding a new item to Azure Queue

Now the worker will consume the item recently added and it will deliver an invitation mail to ciripescu.ro upon a well known delivery timetable, like in the figure below:

Consuming a queue item

 

First we get the queue used for sending invitations emails (service.GetQueue(…)), we check if the queue exists/has items(queue.DoesQueueExist()) and then we consume the item by sending and email and deleting the item from queue(queue.DeleteMessage(..)).

In the beginnings of Windows Azure(winter 2009) a worker once started it would entered in  sleep state if it has nothing to process, or we needed to have an up and running worker all the time in order to process emails. For this an infinite loop was created (‘while(true)’), and the loop was triggered when the worker starts. In the last release of Azure this is resolved and all code inside the infinite loop should be passed to Run()  method defined by the role.

Azure costs…a lot of money from a developer point of view. And because at the beginnings of Azure the number of workers was limited to one, we’ve found a trick to limit the numbers of workers –  now is good to save some money.  What if when a worker starts you start more threads, each thread processing what a worker would normally do? Of course that you lose in performance, but it is a solution….

Outro or instead of conclusions

 Microsoft announced that upgrade from Community Technology Preview (CTP) accounts of the Windows® Azure™ platform (i.e., Windows Azure, SQL Azure and/or Windows Azure platform AppFabric) to paid commercial subscriptions is available, starting with January 4th. And IT COSTS.  The problems about the costs are relative. How much can an Azure application costs you can be calculated here using TCO and ROI.  Probably it’s a long discussion about which companies are going to pay for Azure, who is going to develop applications using Azure (since testing in the real cloud costs…etc) and more pages are required for this analysis. But for sure Azure is dedicated to enterprise applications,  to large companies which don’t want to invest more money in their own hardware and datacenters, which doesn’t want to invest more in sys admins. It’s sure that those may watch Azure as a storm in their activity. It’s sure that in the next years the biggest applications of the world will be in the cloud. Not only Micro$osft cloud…the others too. It’s sure that for small companies it will be hard to develop Azure applications.  We want to see Microsoft that comes and says: “No….it’s not sure…because…at least for the pricing…”. On the other hand Microsoft invested a lot in data centers and they must recover their money. Also you don’t need the cloud to host a simple web application with a small database in backend. There are enough companies to do this almost for nothing.

 Also it’s sure that www.ciripescu.ro will live as long as Azure will be in CTP. It’s sure that we liked to work with azure and to develop this application. I say “WE” because we are two of us who plays with Azure for almost one year. And here I want to thanks to my  colleague and friend Tudor Carean for his work at this project. 

In the end I would point what we liked most at windows azure, but also what we don’t like so much, when building http://www.ciripescu.ro:

  • Azure costs….a lot. For Ciripescu.ro it costs too much to pay after the CTP account will be disabled. It’s better to develop a non-cloud version and to host it on-premises.
  • We don’t like Azure Tables storage model(used at ciripescu.ro), because sometimes it is too rigid.  It is a good model, but it could have been better.
  • When deploying the application to the cloud it takes some time. The whole deployment mechanism is a black box process. You are not sure how much time to estimate with deployment.
  • Azure is  a new paradigm (at least for us),  it introduces the concept of cloud and cloud computing.
  • We like the idea of “worker role” and “web role” and the possibility of building hybrid applications .
  • Azure adoption was easy, it happened for just one time to reinstall windows because of Azure CTP kits 🙂
  • We like the idea of scalability and how it is implemented – “Azure is one click scalable”.
  • Availability and durability are also strong points of Azure.

Probably there are more others too…

Source code of the entire application is available on codeplex at http://ciripescu.codeplex.com/ .  For additional informations please contact me using contact details from the about page or send an email at contact [at] bitstarsolutions.com.

Create a free website or blog at WordPress.com.