RavenDB Persistence in multi-tenant systems

Component: RavenDB Persistence
NuGet Package NServiceBus (7.x)
RavenDB's implementation of distributed transactions contains a bug that could cause an endpoint, in certain (rare) conditions, to lose data. If RavenDB is configured to enlist in distributed transactions, read DTC not supported for RavenDB Persistence.

This sample demonstrates how to configure RavenDB Persistence to store tenant-specific data in separate databases. The tenant-specific information includes the saga state, the business documents that are accessed using RavenDB-managed session, and the outbox records.

This sample uses Outbox to guarantee consistency between the saga state and the business entity.

The sample assumes that the tenant information is passed as a custom message header tenant_id.

Prerequisites

The databases created by this sample are:

  • MultiTenantSamples
  • MultiTenantSamples-A
  • MultiTenantSamples-B

Running the project

  1. Start the Sender project (right-click on the project, select the Debug > Start new instance option).
  2. The text Press <enter> to send a message should be displayed in the Sender's console window.
  3. Start the Receiver project (right-click on the project, select the Debug > Start new instance option).
  4. The Sender should display subscription confirmation Subscribe from Receiver on message type OrderSubmitted.
  5. Press A or B on the Sender console to send a new message either to one of the tenants.

Verifying that the sample works correctly

  1. The Receiver displays information that an order was submitted.
  2. The Sender displays information that the order was accepted.
  3. Finally, after a couple of seconds, the Receiver displays confirmation that the timeout message has been received.
  4. Open SQL Server Management Studio and go to the tenant databases. Verify that there are rows in saga state table (dbo.OrderLifecycleSagaData) and in the orders table (dbo.Orders) for each message sent.
If used with a message transport that does not support native timeouts, timeout data is stored in a shared database so make sure to not include any sensitive information. Keep such information in saga data and only use timeouts as notifications.

Code walk-through

This sample contains three projects:

  • Shared - A class library containing common code including messages definitions.
  • Sender - A console application responsible for sending the initial OrderSubmitted message and processing the follow-up OrderAccepted message.
  • Receiver - A console application responsible for processing the OrderSubmitted message, sending OrderAccepted message and randomly generating exceptions.

Sender project

The Sender does not store any data. It mimics the front-end system where orders are submitted by the users and passed via the bus to the back-end.

Receiver project

The Receiver mimics a back-end system. It is configured to use SQL persistence in multi-tenant mode.

Outbox cleanup

The built-in Outbox cleanup does not work in a multi-tenant environment because it’s executed in the context of the shared database, while Outbox documents are stored in the tenants’ database. For that reason it needs to be disabled.

endpointConfiguration.SetFrequencyToRunDeduplicationDataCleanup(Timeout.InfiniteTimeSpan);

The simplest way to ensure that the dispatched Outbox documents are removed is to use the RavenDB Document expiration bundle. The following code ensures that tenants’ databases are created with this bundle enabled:

var existingDbs = globalAdmin.GetDatabaseNames(100);
var id = $"MultiTenantSamples-{tenant}";
if (existingDbs.Contains(id))
{
    return;
}

globalAdmin.CreateDatabase(new DatabaseDocument
{
    Id = id,
    Settings =
    {
        { "Raven/ActiveBundles", "DocumentExpiration" },
        { "Raven/DataDir", $@"~\Databases\{id}" }
    }
});

To make sure the expiration bundle removes old Outbox documents they need to be marked for expiry. This task is performed by a document store listener.

class MarkForExpiryListener : IDocumentStoreListener
{
    public bool BeforeStore(string key, object entityInstance, RavenJObject metadata, RavenJObject original)
    {
        if (entityInstance.GetType().Name != "OutboxRecord")
        {
            return false;
        }

        var dispatched = entityInstance.GetPropertyValue<bool>("Dispatched");
        if (dispatched)
        {
            var dispatchedAt = entityInstance.GetPropertyValue<DateTime>("DispatchedAt");
            var expiry = dispatchedAt.AddDays(10);
            metadata["Raven-Expiration-Date"] = new RavenJValue(expiry);
        }
        return false;
    }

    public void AfterStore(string key, object entityInstance, RavenJObject metadata)
    {
    }
}

This component is executed each time the RavenDB session is about to send data modifications to the server. It checks if the updated document is an Outbox record and if it has been marked as dispatched. In that case it marks it for expiry after 10 days.

Connecting to the tenant database

To allow for database isolation between the tenants the connection to the database needs to be created based on the message being processed. RavenDB persistence offers an extension point which allows for a customized database name to be used when opening a session.

persistence.SetMessageToDatabaseMappingConvention(headers =>
{
    return headers.TryGetValue("tenant_id", out var tenantId)
        ? $"MultiTenantSamples-{tenantId}"
        : "MultiTenantSamples";
});

The code above ensures that when the tenant_id header is present, the session will point to the tenant database.

Propagating the tenant information downstream

In order to propagate the tenant information to the outgoing messages (including timeouts) this sample uses the same approach as the tenant information propagation sample: a pair of behaviors, one in the incoming pipeline and the other in the outgoing pipeline.

Related Articles


Last modified