Performance Counters

Project Hosting | Nuget: NServiceBus.Metrics.PerformanceCounters (Version: 1.1)
Target NServiceBus Version: 6.x

When a system is broken down into multiple processes, each with its own queue, it allows identifying which process is the bottleneck by examining how many messages (on average) are in each queue. The only issue is that without knowing the rate of messages coming into each queue, and the rate at which messages are being processed from each queue, it is not possible to know how long messages are waiting in each queue, which is the primary indicator of a bottleneck.

Despite the many performance counters Microsoft provides for MSMQ (including messages in queues, machine-wide incoming and outgoing messages per second, and the total messages in all queues), there is no built-in performance counter for the time it takes a message to get through each queue.

NServiceBus includes several performance counters. They are installed in the NServiceBus category.

Since all performance counters in Windows are exposed via Windows Management Instrumentation (WMI), it is very straightforward to pull this information into the existing monitoring infrastructure.

Counters

All counters are defined in the NuGet package dependency NServiceBus.Metrics. The dependency is automatically pulled in.

For more information about the metrics defined consult the Metrics documentation page.

Configuration

This counters can be enabled using the the following code:

Edit
var performanceCounters = endpointConfiguration.EnableWindowsPerformanceCounters();

In the NServiceBus Host this counters are enabled by default.

Setting up SLA value can be done using following code:

Edit
var performanceCounters = endpointConfiguration.EnableWindowsPerformanceCounters();
performanceCounters.EnableSLAPerformanceCounters(TimeSpan.FromMinutes(3));

Update counters

The counters are periodically updated every two seconds by default. To override the update interval:

Edit
performanceCounters.UpdateCounterEvery(TimeSpan.FromSeconds(2));

Installing Counters

NServiceBus.Metrics.PerformanceCounters.MsBuild

This packages installs into the MSBuild pipeline and generates all performance counter installation scripts at compile time. It does this by interrogating types (in the target assembly) and attributes (from the NServiceBus.Metrics.PerformanceCounter as well as NServiceBus.Metrics NuGet package) to infer what scripts to create. It is required for any project where those performance counter installation scripts are required. This package has a dependency on the NServiceBus.Metrics.PerformanceCounter NuGet package

Performance Counters Versions 1.1 and above: is dependent on NServiceBus.Metrics in calculating counters values.

Script Creation

Performance counter installation scripts are created at compile time by the NServiceBus.Metrics.PerformanceCounters.MsBuild NuGet package.

Scripts will be created in the directory format of [CurrentProjectDebugDir]\NServiceBus.Metrics.PerformanceCounters\[ScriptVariant].

For example for a project named ClassLibrary build in Debug mode the following directories will be created.

  • ClassLibrary\bin\Debug\NServiceBus.Metrics.PerformanceCounters\CSharp
  • ClassLibrary\bin\Debug\NServiceBus.Metrics.PerformanceCounters\Powershell

Scripts will also be included in the list of project output files. So this means those files produced will be copied to the output directory of any project that references it.

Scripts creation can configured via the use of [PerformanceCounterSettings] applied to the target assembly.

To Produce All scripts

Edit
[assembly: PerformanceCounterSettings(CSharp = true, Powershell = true)]

To Produce only CSharp scripts

Edit
[assembly: PerformanceCounterSettings(CSharp = true, Powershell = false)]

To Produce only Powershell scripts

Edit
[assembly: PerformanceCounterSettings(CSharp = false, Powershell = true)]

Promotion

As stated above, scripts are created in the target project output directory. Generally this directory will be excluded from source control. To add created scripts to source control they can be "promoted".

The target directory will be deleted and recreated as part of each build. So ensure to choose a path that is for script promotion only.

Some token replacement using MSBuild variables is supported.

  • $(SolutionDir): The directory of the solution.
  • $(ProjectDir): The directory of the project

All tokens are drive + path and include the trailing backslash \.

Edit
[assembly: PerformanceCounterSettings(ScriptPromotionPath = "$(SolutionDir)PromotedScripts")]

The path calculation is performed relative to the current project directory. So, for example, a value of PromotedScripts (with no tokens) would evaluate as $(ProjectDir)PromotedScripts.

Script Usage

The above task takes preconfigured values and based on configured Metrics generated PowerShell and/or cs files containing code creating performance counters. After running that task to create performance counters the following calls have to be made with elevated permissions:

CounterCreator.Create();
./CreateNSBPerfCounters.ps1

To list the installed counters use

Get-Counter -ListSet NServiceBus | Select-Object -ExpandProperty Counter
Metrics are defined in NServiceBus.Metrics NuGet package and will be dynamically turned into performance counters. When the NServiceBus.Metrics dependency is updated new counters might be available in the installation script. Make sure the scripts are executed with elevated permissions on required machines when the scripts have been updated.

Performance Monitor Users local security group

When running installers the service account will be automatically added to the local Performance Monitor Users group if executed with elevated privileges.

System.InvalidOperationException

If the endpoint instance throws one of the following exceptions at startup, then the performance counters need to be reinstalled

  • InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
  • InvalidOperationException: NServiceBus performance counter for 'Critical Time' is not set up correctly.

Corrupted Counters

Corrupted performance counters can cause the endpoint to either hang completely during startup or fail with the following error:

NServiceBus performance counter for '{counterName}' is not set up correctly

Should this happen try rebuilding the performance counter library using the following steps:

  1. Open an elevated command prompt
  2. Execute the following command to rebuild the performance counter library: lodctr /r

More information

Samples

Related Articles

  • Auditing Messages
    Configure where to send messages and it provides built-in message auditing for every endpoint.
  • Management using PowerShell
    Install the infrastructure for NServiceBus on servers using PowerShell.
  • Recoverability
    Explains how exception are handled, and actions retries, during message processing.

Last modified