Shipping the ServiceControl audit log to a SIEM with OpenTelemetry

When role-based access control is enabled, ServiceControl writes every authorization decision — allowed and denied — to a dedicated audit log formatted as Elastic Common Schema (ECS) JSON. This sample shows how to ship that audit trail over OTLP (OpenTelemetry Protocol) to an OpenTelemetry Collector, which can then deliver it to a SIEM (Security Information and Event Management system — a log aggregator such as Elastic Security, Splunk, or Microsoft Sentinel) for alerting and compliance retention.

Prerequisites

• Docker (or Podman) with the compose plugin

• A .env file next to docker-compose.yml containing a valid license:

  PARTICULARSOFTWARE_LICENSE=<license file contents>
  

The stack

The docker-compose.yml starts:

Configuring ServiceControl

Two things happen in the ServiceControl service definition: role-based access control is switched on, and the OTLP logging provider is added alongside the default NLog provider via the LoggingProviders setting. The standard OTEL_EXPORTER_OTLP_ENDPOINT environment variable points the exporter at the collector.

servicecontrol:
  image: particular/servicecontrol:latest   # role-based access control + the audit log require 6.18.0 or later
  command: ["--setup-and-run"]
  env_file: [.env]                       # provides PARTICULARSOFTWARE_LICENSE
  environment:
    TRANSPORTTYPE: RabbitMQ.QuorumConventionalRouting
    CONNECTIONSTRING: "host=rabbitmq"
    RAVENDB_CONNECTIONSTRING: "http://ravendb:8080"

    # --- Authentication (OIDC) + role-based access control ---
    SERVICECONTROL_AUTHENTICATION_ENABLED: "true"
    SERVICECONTROL_AUTHENTICATION_ROLEBASEDAUTHORIZATIONENABLED: "true"
    SERVICECONTROL_AUTHENTICATION_AUTHORITY: "http://keycloak:8080/realms/particular"
    SERVICECONTROL_AUTHENTICATION_AUDIENCE: "servicecontrol-api"
    # Keycloak puts realm roles in a nested claim; a dotted path reaches them
    SERVICECONTROL_AUTHENTICATION_ROLESCLAIM: "realm_access.roles"
    SERVICECONTROL_AUTHENTICATION_SERVICEPULSE_CLIENTID: "servicepulse"
    SERVICECONTROL_AUTHENTICATION_SERVICEPULSE_AUTHORITY: "http://localhost:8088/realms/particular"
    SERVICECONTROL_AUTHENTICATION_SERVICEPULSE_APISCOPES: '["servicecontrol"]'
    # Sample runs the identity provider over plain HTTP; never do this in production
    SERVICECONTROL_AUTHENTICATION_REQUIREHTTPSMETADATA: "false"

    # --- Audit log over OTLP ---
    # Keep NLog for the operational log/console and add the OTLP logging
    # provider. The ECS audit documents become the OTLP log record body.
    SERVICECONTROL_LOGGINGPROVIDERS: "NLog,Otlp"
    OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4317"
  ports:
    - "33333:33333"
  depends_on:
    rabbitmq: { condition: service_healthy }
    ravendb: { condition: service_healthy }
    keycloak: { condition: service_healthy }
    otel-collector: { condition: service_started }

With LoggingProviders=NLog,Otlp, the operational log keeps flowing to the console/log files through NLog, while every log record — including the audit trail — is additionally exported as OTLP log records. The audit trail is emitted on the logger categories ServiceControl.Audit (authorization decisions and message operations) and ServiceControl.Audit.Messages (per-message entries of bulk operations), with the pre-rendered ECS JSON document as the OTLP log record body. Allowed decisions are logged at Information level; denied decisions at Warning level, so alerting on denials does not require parsing the payload.

The ServiceControl.Audit.Messages category can be high volume on large bulk retries or archives. It can be filtered independently of the operation-level trail through standard Microsoft.Extensions.Logging configuration — see filtering the per-message audit stream.

Configuring the collector

The collector runs as a regular container next to ServiceControl:

otel-collector:
  # Pinned deliberately: the collector is pre-1.0 and its configuration format
  # still changes between releases, so a floating tag could break otel-collector.yaml.
  image: otel/opentelemetry-collector-contrib:0.156.0
  command: ["--config=/etc/otelcol-contrib/config.yaml"]
  volumes:
    - ./otel-collector/otel-collector.yaml:/etc/otelcol-contrib/config.yaml:ro
  depends_on:
    elasticsearch: { condition: service_healthy }

It receives all ServiceControl logs, keeps only the audit categories, parses the ECS document, and hands it to one or more exporters:

receivers:
  # ServiceControl pushes its logs here (OTEL_EXPORTER_OTLP_ENDPOINT)
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch: {}

  # ServiceControl exports ALL its logs when the Otlp provider is enabled.
  # The audit trail is identified by its logger category ("ServiceControl.Audit"
  # and "ServiceControl.Audit.Messages"), which arrives as the OTLP
  # instrumentation scope name. Drop everything else so only the audit
  # trail reaches the SIEM pipeline.
  filter/audit-only:
    logs:
      log_record:
        - not IsMatch(instrumentation_scope.name, "^ServiceControl\\.Audit")

  # The ECS document is the log record body (a JSON string). Parsing it into
  # a structured body lets backends index the individual ECS fields - this is
  # what the Elasticsearch exporter's bodymap mapping mode expects. The
  # elastic.mapping.mode scope attribute selects that mode per record
  # (since collector 0.156 the exporter's mapping::mode config is ignored).
  transform/parse-ecs:
    log_statements:
      - context: log
        statements:
          - set(body, ParseJSON(body)) where IsMatch(body, "^\\{")
      - context: scope
        statements:
          - set(attributes["elastic.mapping.mode"], "bodymap")

exporters:
  # Prints every audit entry to the collector's stdout so the sample is
  # observable with "docker compose logs otel-collector".
  debug:
    verbosity: detailed

  # Elasticsearch / Elastic Security. In "bodymap" mapping mode (selected via
  # the elastic.mapping.mode scope attribute above) the parsed ECS document is
  # indexed as-is, exactly as ServiceControl emitted it. Elasticsearch's
  # built-in logs-*-* index template already maps ECS fields; no custom
  # mapping is needed. Production clusters use https and an api_key.
  elasticsearch:
    endpoint: http://elasticsearch:9200
    logs_index: logs-servicecontrol.audit-default

  # --- Other SIEM delivery options (add to the pipeline as needed) ---
  #
  # Splunk (HTTP Event Collector):
  #
  # splunk_hec:
  #   endpoint: https://splunk:8088/services/collector
  #   token: ${env:SPLUNK_HEC_TOKEN}
  #   index: servicecontrol_audit
  #
  # Grafana Loki natively ingests OTLP (Loki >= 3.0), no dedicated exporter needed:
  #
  # otlphttp/loki:
  #   endpoint: https://loki:3100/otlp
  #
  # Plain files (rotated), e.g. for compliance archival:
  #
  # file:
  #   path: /var/log/otel/servicecontrol-audit.json
  #   rotation:
  #     max_megabytes: 30
  #     max_backups: 14

service:
  pipelines:
    logs/audit:
      receivers: [otlp]
      processors: [filter/audit-only, transform/parse-ecs, batch]
      exporters: [debug, elasticsearch]

Four details worth calling out:

• Filtering by category. The Microsoft.Extensions.Logging category name arrives as the OTLP instrumentation scope name, so the filter processor can isolate the audit trail (ServiceControl.Audit*) from operational logs without inspecting payloads.
• Parsing the body. ServiceControl deliberately exports each audit entry as a self-contained ECS JSON string in the record body. The transform processor's ParseJSON turns it into a structured body so backends index the individual fields.
• Mapping mode. The elastic.mapping.mode scope attribute selects the Elasticsearch exporter's bodymap mode, which indexes the parsed body verbatim — the Elasticsearch document is then exactly the ECS document ServiceControl emitted. (Since collector 0.156 the exporter's mapping::mode config option is deprecated and ignored; the scope attribute or the X-Elastic-Mapping-Mode client metadata key replaces it.)
• Fan-out. A single collector pipeline can deliver the same stream to multiple destinations — a SIEM for alerting plus rotated files for long-term retention.

Running the sample

docker compose up -d

Once healthy, generate audit events. Either log into ServicePulse at http://localhost:9090 (e.g. as reader / reader) and browse around, or use the API directly:

# An allowed request: 'reader' may view failed messages
TOKEN=$(curl -s http://localhost:8088/realms/particular/protocol/openid-connect/token \
  -d grant_type=password -d client_id=servicepulse \
  -d username=reader -d password=reader \
  -d scope="openid profile servicecontrol" | jq -r .access_token)

curl -s -H "Authorization: Bearer $TOKEN" http://localhost:33333/api/errors > /dev/null

# A denied request: 'reader' may not retry messages
curl -s -X POST -H "Authorization: Bearer $TOKEN" http://localhost:33333/api/errors/retry/all

Then watch the audit entries arrive at the collector:

docker compose logs otel-collector | grep "Body:"

Each OTLP record carries the ECS document as its body, together with the resource attributes service.name, service.version, and service.instance.id identifying which ServiceControl instance emitted it.

Querying the audit trail in Elasticsearch and Kibana

The collector delivers the same entries to Elasticsearch, into the logs-servicecontrol.audit-default data stream. Elasticsearch's built-in logs-*-* index template recognizes the ECS field names and maps them correctly (event.category, event.outcome, and user.name become keyword fields; @timestamp becomes a date) — no custom mapping or index template needs to be installed. Query the denied decisions directly:

curl -s "http://localhost:9200/logs-servicecontrol.audit-default/_search?q=event.category:iam%20AND%20event.outcome:failure" | jq .hits.total.value

Or use Kibana at http://localhost:5601: create a data view for logs-servicecontrol.audit-* with @timestamp as the time field (Stack management → Data views), then in Discover run the KQL query:

event.category:iam and event.outcome:failure

Every denied authorization decision appears, with the full ECS document expandable per hit. Because user.name is a keyword field it is aggregatable, so visualizations and alert rules such as "failures per user over time" work immediately.

The ECS audit document

ECS is Elastic's open specification for naming log fields consistently so that tooling — dashboards, correlation rules, detection content — works across data sources. It is the de-facto ingestion schema for SIEMs beyond Elastic's own: most systems either ingest ECS natively or ship converters for it. In 2023 Elastic donated ECS to OpenTelemetry, and the two schemas are converging namespace by namespace, though the security-relevant event.* classification fields remain ECS-only for now.

An allowed decision looks like:

{
  "@timestamp": "2026-07-07T16:57:13.7857082+00:00",
  "event": {
    "kind": "event",
    "category": ["iam"],
    "type": ["allowed"],
    "action": "error:messages:view",
    "outcome": "success"
  },
  "user": {
    "id": "38589380-f1ca-43da-9585-b9e94b2a49ad",
    "name": "reader"
  },
  "servicecontrol": {
    "permission": "error:messages:view",
    "reason": "User holds 'error:messages:view' via role(s) [reader]"
  }
}

A denied decision differs in event.type (["denied"]), event.outcome ("failure"), the log level (Warning), and the reason. The fields follow ECS's categorization model:

Because the documents are already ECS, they ingest into Elastic/Kibana with no custom mapping, and SIEM content such as "alert when event.category:iam AND event.outcome:failure spikes for one user.name" works out of the box — querying the audit trail demonstrates exactly this query against the included Elasticsearch container.

Delivering to a SIEM

Is there something that can output the logs to the destination of choice? Yes — that is exactly the OpenTelemetry Collector's job. The otel/opentelemetry-collector-contrib distribution ships exporters for the common destinations; the sample delivers to Elasticsearch and its collector config contains ready-to-uncomment blocks for the others:

Alternatives to the collector exist and speak OTLP too: Elastic Agent embeds an OpenTelemetry Collector distribution (EDOT), Vector has an OpenTelemetry source, and Fluent Bit ships OTLP input/output plugins. The collector remains the most direct fit here because the ECS documents need no re-mapping — any OTLP-capable pipeline can carry them.

Deployments that cannot use OTLP can still collect the audit trail from its file/console destination: when running as a Windows service the audit trail is written to audit.json in the instance log directory, and in containers it goes to standard output — see the authorization documentation for details.