ServiceControl Hardware Considerations

ServiceControl as an application can be used to process the entire message load of a system. This article provides general guidelines, recommendations, and performance benchmarks to help determine the resources to provide for a production environment. To identify the hardware specifications for any system environment, a combination of testing with the system and the information provided below will need to be used.

General recommendations

Install ServiceControl on a dedicated server in production.
Hosting ServiceControl and ServiceControl.Audit is preferred on seperate servers.
A minimum of 12 GB of RAM per instance (excluding RAM for OS and other services).
2 GHz quad core CPU or better
Database path located on disks suitable for low latency write operations (fiber, solid state drives, raid 10), with a recommended IOPS of at least 7500.

Use a storage benchmark tool to measure disk performance, such as Windows System Assessment Tool (winsat disk -drive g), CrystalDiskMark, or DiskSpd.

Hosting in the cloud

The only way to host ServiceControl in the cloud is to use a virtual machine.

Server performance monitoring

Due to changes in the system it supports, the requirements for a server hosting ServiceControl can change over time. It is highly recommended that monitoring of the CPU, RAM, disk I/O, and network I/O for the server running ServiceControl be included.

Real disk, CPU, RAM, and network performance can be monitored with the Windows Resource Monitor and/or Windows Performance counters.

Storage

It is recommended to:

Store ServiceControl data on a dedicated disk. This makes low-level resource monitoring easy and ensures different applications are not competing for storage IOPS.
Store multiple ServiceControl databases on seperate physical disks to prevent multiple instances to compete for the same disk resources.
Disable disk write caching (read caching is fine) to prevent data corruption if the (virtual) server or disk controler fails. This is a general best practice for databases.

Do not use an ephemeral AWS or Azure disk for ServiceControl data because these disks will be erased when the virtual machine reboots.

Suggestions to improve performance

More RAM

The embedded RavenDB will utilize additional RAM to improve indexing performance. During load ServiceControl can easily peak to 12GB

Message size / MaxBodySizeToStore

In general, the smaller the message, the quicker ServiceControl will be able to process audit records. Consider using smaller messages. For larger message payloads, consider using the data bus feature.

In addition, for audit messages, lower the ServiceControl/MaxBodySizeToStore setting to skip storage of larger audit messages. This setting will only reduce load if a non-binary serialization is used.

When using ServiceInsight, the message body will not be viewable for messages that exceed the ServiceControl/MaxBodySizeToStore limit.

Use a dedicated disk for the database

Use a dedicated disk for the ServiceControl database path.

Additionally, it is possible to store the embedded database index files on a separate disk. Use the Raven/IndexStoragePath setting to change the index storage location.

Azure disk limitations

Using multiple 7500 IOPS disks in striped mode in Azure may not improve performance due to increased latency; consider scaling out ServiceControl to multiple instances.

Scale out

If it is not possible to scale up ServiceControl to handle system volume, partition audit processing between multiple instances of ServiceControl. See Multiple ServiceControl Instances for more details.

Turn off full-text search

Even though it can be extremely valuable to search for specific messages based on their content, updating the full-text index requires a considerable amount of CPU as well as disk space. The ability to turn off the full-text search is available in the ServiceControl Management Utility.