SQL Persistence in multi-tenant system

Component: Sql Persistence
NuGet Package NServiceBus.Persistence.Sql (4.x)
Target NServiceBus Version: 7.x

This sample demonstrates how to configure SQL Persistence to store tenant-specific data in separate catalogs, for each tenant. The tenant-specific information includes saga state, and business entities that are accessed using NServiceBus-managed session. In addition, the Outbox is used to guarantee consistency between the saga state and the business entity. Outbox data is also stored in the tenant-specific database.

Because this sample uses the Learning Transport, which provides delayed delivery (timeouts) as well as publish/subscribe natively, there is no need for a common database to store data for those capabilities. A message transport like MSMQ, which does not provide native delayed delivery or publish/subscribe, would require a common database to store timeouts and subscriptions shared by all tenants.

The sample assumes that the tenant information is passed as a custom message header tenant_id.

Prerequisites

An instance of SQL Server Express is installed and accessible as .\SqlExpress.

At startup each endpoint will create its required SQL assets including databases, tables and schemas.

The databases created by this sample are:

  • SqlMultiTenantA
  • SqlMultiTenantB

Running the project

  1. Start the Sender project (right-click on the project, select the Debug > Start new instance option).
  2. The text Press <enter> to send a message should be displayed in the Sender's console window.
  3. Start the Receiver project (right-click on the project, select the Debug > Start new instance option).
  4. The Sender should display subscription confirmation Subscribe from Receiver on message type OrderSubmitted.
  5. Press A or B on the Sender console to send a new message either to one of the tenants.

Verifying that the sample works correctly

  1. The Receiver displays information that an order was submitted.
  2. The Sender displays information that the order was accepted.
  3. Finally, after a couple of seconds, the Receiver displays confirmation that the timeout message has been received.
  4. Open SQL Server Management Studio and go to the tenant databases. Verify that there are rows in saga state table (dbo.OrderLifecycleSagaData) and in the orders table (dbo.Orders) for each message sent.
If used with a message transport that does not support native timeouts, timeout data is stored in a shared database so make sure to not include any sensitive information. Keep such information in saga data and only use timeouts as notifications.

Code walk-through

This sample contains three projects:

  • Shared - A class library containing common code including messages definitions.
  • Sender - A console application responsible for sending the initial OrderSubmitted message and processing the follow-up OrderAccepted message.
  • Receiver - A console application responsible for processing the OrderSubmitted message, sending OrderAccepted message and randomly generating exceptions.

Sender project

The Sender does not store any data. It mimics the front-end system where orders are submitted by the users and passed via the bus to the back-end.

Receiver project

The Receiver mimics a back-end system. It is configured to use SQL persistence in multi-tenant mode.

Creating the schema

The default SQL Persistence installers create all schema objects in a single catalog. In multi-tenant scenarios schema objects need to be created manually. The ScriptRunner class provides APIs required to run schema creation scripts.

This code snippet makes sure that business entity and saga tables are created in the tenant databases.

await ScriptRunner.Install(dialect, tablePrefix, () => new SqlConnection(Connections.TenantA), scriptDirectory,
    shouldInstallOutbox: true,
    shouldInstallSagas: true,
    shouldInstallSubscriptions: false,
    shouldInstallTimeouts: false);

await ScriptRunner.Install(dialect, tablePrefix, () => new SqlConnection(Connections.TenantB), scriptDirectory,
    shouldInstallOutbox: true,
    shouldInstallSagas: true,
    shouldInstallSubscriptions: false,
    shouldInstallTimeouts: false);

Because the Outbox tables are stored in multiple per-tenant databases, SQL Persistence is not able to automatically clean Outbox entries. This setting must be confirmed by disabling Outbox cleanup:

var outboxSettings = endpointConfiguration.EnableOutbox();
outboxSettings.DisableCleanup();

The Outbox tables on each tenant database must be cleaned by an outside process like SQL Agent.

Connecting to the tenant database

To allow for database isolation between the tenants the connection to the database needs to be created based on the message being processed. This requires cooperation of two components:

The connection factory retrieves the value of the tenant_id header and builds a connection string based on the header value.

persistence.MultiTenantConnectionBuilder(tenantIdHeaderName: "tenant_id",
    buildConnectionFromTenantData: tenantId =>
    {
        var connectionString = Connections.GetForTenant(tenantId);
        return new SqlConnection(connectionString);
    });

When SQL Persistence needs to open a connection, the connection factory is called using the value extracted from the message. As an alternative, other connection factory options exist that allow consulting multiple headers to extract tenant information.

Propagating the tenant information downstream

In order to propagate the tenant information to the outgoing messages (including timeouts) this sample uses the same approach as the tenant information propagation sample: a pair of behaviors, one in the incoming pipeline and the other in the outgoing pipeline.

Related Articles

  • Outbox
    Reliable messaging without distributed transactions.
  • SQL Persistence
    A persister that targets relational databases, including SQL Server, Oracle. MySQL, and PostgreSQL.

Last modified