Windows Performance Counters

When a system is broken down into multiple processes - each with its own queue - it allows identifying which process is the bottleneck by examining how many messages (on average) are in each queue. However, it is not possible to know how long messages are waiting in each queue - which is the primary indicator of a bottleneck - without knowing the rate of messages coming into each queue, and the rate at which messages are being processed from each queue.

The Windows Performance Counters package provides additional performance counters to better understand how long it takes for messages to flow through queues, not just the number of messages in the queue.

Since all performance counters in Windows are exposed via Windows Management Instrumentation (WMI), it is very straightforward to pull this information into the existing monitoring infrastructure.

While the standard performance counters in the NServiceBus 6 package are marked obsolete, for compatibility reasons they are still enabled by default.

Counters

All counters are defined in the NuGet package dependency NServiceBus.Metrics. The dependency is automatically pulled in.

For more information about the metrics defined consult the Metrics documentation page.

Configuration

The counters can be enabled using the the following code:

var performanceCounters = endpointConfiguration.EnableWindowsPerformanceCounters();

In the NServiceBus host the counters are enabled by default.

Setting up an SLA value can be done using the following code:

var performanceCounters = endpointConfiguration.EnableWindowsPerformanceCounters();
performanceCounters.EnableSLAPerformanceCounters(TimeSpan.FromMinutes(3));

Installing counters

NServiceBus.Metrics.PerformanceCounters.MsBuild

This packages installs into the MSBuild pipeline and generates all performance counter installation scripts at compile time. It does this by interrogating types (in the target assembly) and attributes (from the NServiceBus.Metrics.PerformanceCounter and NServiceBus.Metrics NuGet packages) to infer what scripts to create. It is required for any project where performance counter installation scripts are needed. This package has a dependency on the NServiceBus.Metrics.PerformanceCounter NuGet package.

Script creation

Performance counter installation scripts are created at compile time by the NServiceBus.Metrics.PerformanceCounters.MsBuild NuGet package.

Scripts will be created in the directory format of [CurrentProjectDebugDir]\NServiceBus.Metrics.PerformanceCounters\[ScriptVariant].

For example: A project named ClassLibrary built in Debug mode will have the following directories created:

• ClassLibrary\bin\Debug\NServiceBus.Metrics.PerformanceCounters\CSharp
• ClassLibrary\bin\Debug\NServiceBus.Metrics.PerformanceCounters\PowerShell

Scripts will also be included in the list of project output files. This means those files produced will be copied to the output directory of any project that references it.

Scripts creation can be configured by applying [PerformanceCounterSettings] to the target assembly.

To produce all scripts

[assembly: PerformanceCounterSettings(CSharp = true, Powershell = true)]

To produce only C# scripts

[assembly: PerformanceCounterSettings(CSharp = true, Powershell = false)]

To produce only PowerShell scripts

[assembly: PerformanceCounterSettings(CSharp = false, Powershell = true)]

Promotion

As stated above, scripts are created in the target project output directory. Generally this directory will be excluded from source control. To add created scripts to source control they can be "promoted".

Some token replacement using MSBuild variables is supported.

• $(SolutionDir): The directory of the solution.
• $(ProjectDir): The directory of the project

All tokens are drive + path and include the trailing backslash \.

[assembly: PerformanceCounterSettings(ScriptPromotionPath = "$(SolutionDir)PromotedScripts")]

The path calculation is performed relative to the current project directory. For example, a value of PromotedScripts (with no tokens) would evaluate as $(ProjectDir)PromotedScripts.

Script usage

The above task takes preconfigured values based on configured Metrics-generated PowerShell and/or .cs files containing code creating performance counters. After running that task to create performance counters, the following calls have to be made with elevated permissions:

CounterCreator.Create();

./CreateNSBPerfCounters.ps1

To list the installed counters use

Get-Counter -ListSet NServiceBus | Select-Object -ExpandProperty Counter

Performance Monitor Users local security group

When running installers, the service account will be automatically added to the local Performance Monitor Users group if executed with elevated privileges.

System.InvalidOperationException

If the endpoint instance throws one of the following exceptions at startup, then the performance counters need to be reinstalled

• InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
• InvalidOperationException: NServiceBus performance counter for 'Critical Time' is not set up correctly.

Corrupted Counters

Corrupted performance counters can cause the endpoint to either hang completely during startup or fail with the following error:

NServiceBus performance counter for '{counterName}' is not set up correctly

Should this happen, try rebuilding the performance counter library using the following steps:

1. Open an elevated command prompt
2. Execute the following command to rebuild the performance counter library: lodctr /r

More information

• KB2554336: How to manually rebuild Performance Counters for Windows Server 2008 64bit or Windows Server 2008 R2 systems
• KB300956: How to manually rebuild Performance Counter Library values
• LODCTR at TechNet