Let's rokk! [Tudor Cret's blog]

April 18, 2011

Azure Diagnostics

Filed under: Windows Azure — Tudor Cret @ 2:49 pm
Tags: ,

Diagnostics and monitoring for services and applications that don’t run in the cloud doesn’t require a special attention since you have physical  access to the production server, more or less, it depends on how you’ve decided to host them. But in the cloud there are several challenges with diagnostics like:

  • Many instances
  • They move around
  • Massive amount of data
  • Can’t remote desktop in
  • No remote tools (yet)

So Microsoft have implemented a monitoring agent (MonAgentHost.exe) which runs on each instance in the cloud. The agent is started automatically by default. The listener is wired up in the app/web.config, like any other TraceListener. More you need to define a storage account connection string for this listener. In a nutshell it works in 5 steps:


  1. 1.Role instance starts and by default the monitoring agent is started too.
  2. 2.Diagnostic Monitor Starts
  3. 3.Monitor must be configured at start time or remotely any time using service configuration file.
  4. 4.Monitor starts to buffer data locally. The user might set a quota, too.
  5. 5.User initiates transfer to storage. The transfer can be scheduled  or on demand. I recommend a scheduled transfer. In this way diagnostics storage is up to date with live locally data.
  6. image


Below is the table with items that can be monitored in the cloud, if they are activated by default or not and the storage destination type.

Data Source



Trace Logs


Azure Table

Diagnostic Infrastructure Logs


Azure Table

IIS Logs



Performance Counters


Azure Table

Windows Event Logs


Azure Table

IIS Failed Request Logs



Crash Dumps



Arbitrary Files



Implementing Azure Diagnostics:

First Diagnostic Agent is loaded as an Azure module the ServiceDefinition.csdef:


Then module expects a connection string:


Attention, a production connection string must be https! Like this:

<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;
AccountName=YOURACCOUNT;AccountKey=YOURKEY" />

The common pattern to configure diagnostics suppose:

  • Get Config
    • From default
    • Current running
  • Make a change to the config
    • If changed within the instance it will affect only that instance. Don’t forget to start the agent immediately
    • If changed from outside for all roles then
      • change the central file
      • agent notices a change and reloads
      • affects all instances of the role
  • Start the Diagnostics agent with the new configuration

The code:

Method 1:

  • I’ve set up a transfer to the storage every one minute
  • Also I’ve added “System” and “Application” transfer from Event Viewer
Code Snippet
  1. //—————————————-method 1—————————-//
  2.                 //get config
  3.                 CloudStorageAccount storageAcc = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"));
  4.                 RoleInstanceDiagnosticManager ridm = storageAcc.CreateRoleInstanceDiagnosticManager(RoleEnvironment.DeploymentId,
  5.                     RoleEnvironment.CurrentRoleInstance.Role.Name, RoleEnvironment.CurrentRoleInstance.Id);
  6.                 DiagnosticMonitorConfiguration dmc = ridm.GetCurrentConfiguration();
  8.                 //change config
  9.                 // Transfer logs to storage every minute
  10.                 dmc.Logs.ScheduledTransferPeriod = tsTenSeconds;
  11.                 dmc.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = tsTenSeconds;
  12.                 dmc.Directories.ScheduledTransferPeriod = tsTenSeconds;
  13.                 dmc.WindowsEventLog.ScheduledTransferPeriod = tsTenSeconds;
  15.                 dmc.WindowsEventLog.DataSources.Add("System!*");
  16.                 dmc.WindowsEventLog.DataSources.Add("Application!*");
  17.                 dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Information;
  19.                 ridm.SetCurrentConfiguration(dmc);
  20.                 //—————————————-end method 1—————————-//

Method 2:

  • I’ve loaded the default configuration and I’ve set up a scheduled transfer
Code Snippet
  1. //—————————————-method 2—————————-//
  2.                 // Start up the diagnostic manager with the given configuration
  3.                 DiagnosticMonitorConfiguration dmc = DiagnosticMonitor.GetDefaultInitialConfiguration();
  4.                 dmc.Logs.ScheduledTransferPeriod = tsTenSeconds;
  5.                 // Transfer verbose, critical, etc. logs
  6.                 dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
  7.                 DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", dmc);
  8.                 //—————————————-end method 2—————————-//

Method 3:

  • I’ve created a custom listener that writes log entries in a dedicated Azure table
Code Snippet
  1. //—————————————-method 3—————————-//
  2.                 CloudStorageAccount storageAcc = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"));
  3.                 CloudTableClient tableStorage = storageAcc.CreateCloudTableClient();
  4.                 tableStorage.CreateTableIfNotExist(TableStorageTraceListener.DIAGNOSTICS_TABLE);
  5.                 AzureDiagnostics.TableStorageTraceListener listener =
  6.                  new AzureDiagnostics.TableStorageTraceListener("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString")
  7.                  {
  8.                      Name = "TableStorageTraceListener"
  9.                  };
  10.                 System.Diagnostics.Trace.Listeners.Add(listener);
  11.                 System.Diagnostics.Trace.AutoFlush = true;
  12.                 Trace.Listeners.Add(new DiagnosticMonitorTraceListener());
  13.                 //—————————————-end method 3—————————-//

Full code available here.

Visualizing the data

You might use Azure Storage Explorer to explore diagnostics storage account or you can use Cerebrata’s Azure Diagnostics Manager which is a dedicated tool for diagnostics management. It will display a more friendly UI than the Storage Explorer:



Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Blog at WordPress.com.

%d bloggers like this: