Discarding Old Messages

Component: NServiceBus
NuGet NServiceBus (4.x)
Standard support for version 4.x of NServiceBus has expired. For more information see our Support Policy.

A message sent through the Particular Service Platform may have Time-To-Be-Received (TTBR) set, according to the users’ decision. TTBR indicates to the platform that a delayed message will be discarded, instead of processed, if not handled within a specified period of time. A discarded message might no longer have any business value, and discarding it frees up storage, CPU and memory resources.

This is important mainly in environments with high volumes of messages where there is little business value in processing a delayed message since it will already be replaced by a newer, more relevant version.

Only messages that have not been handled will have the TTBR set. A failed message moved to the error queue or a successfully handled message, including its possible audit message, is considered handled. By removing TTBR from handled messages it is ensured that no message will be lost after it has been processed. The TTBR from the original message can always be inspected by looking at the NServiceBus.TimeToBeReceived header.

If a message cannot be received by the target process in the given time frame, including all time in queues and in transit, it may be desirable to discard it by using TimeToBeReceived.

To discard a message when a specific time interval has elapsed:

Using an Attribute

// Discard after one minute
[TimeToBeReceived("00:01:00")]
public class MyMessage
{
}

Using a custom convention

configure.DefiningTimeToBeReceivedAs(
    type =>
    {
        if (type == typeof(MyMessage))
        {
            return TimeSpan.FromHours(1);
        }
        return TimeSpan.MaxValue;
    });

Clock synchronization issues

When sending a message with a certain TimeToBeReceived value it could happen that the receiver drops the message if clocks are too much out of sync. For example, if TimeToBeReceived is 1 minute and the receivers clock is 1 minute ahead compared to the sender the message could potentially never be delivered thus processed.

Because clocks usually are at most a few minutes out of sync this issue only applies to relatively small TimeToBeReceived values.

For this reason it is wise to add the maximum amount of allowed clock offset, called clock drift, to the TTBR value. For example, when using TimeToBeReceived value of 90 seconds, one should allow for 300 seconds of maximum clock drift so the TTBR value becomes 90 + 300 = 390 seconds.

Discarding old messages at startup

In certain situations it may be required that old messages are not processed after restarting the endpoint. Usually this functionality is used in development and test environments, but may be also appropriate when messages contain information that gets outdated, e.g. change notifications, readings from sensors in IoT apps, etc.

Discarding old messages at startup should be used only in special scenarios when such functionality is desired. As a general rule, it's not recommended to be used in production as it may lead to subtle message loss situations.

To discard old messages at startup:

configure.PurgeOnStartup(true);

Caveats

TimeToBeReceived relies on underlying functionality in the transport infrastructure to discard expired messages. This feature's usefulness is highly affected by the actual implementation in the different transports.

MSMQ transport

MSMQ continuously checks the TimeToBeReceived of all queued messages. As soon as the message has expired, it is removed from the queue, and disk space reclaimed.

MSMQ however only allows a single TimeToBeReceived for all messages in a transaction. If multiple messages enlist in a single transaction, then TimeToBeReceived from the first message will be used for all messages, leading to a potential message loss scenario. To prevent message loss, TimeToBeReceived is not supported for endpoints with transaction mode TransportTransactionMode.AtomicSendsWithReceive or Transaction Scope (Distributed Transaction).

RabbitMQ transport

RabbitMQ continuously checks the TimeToBeReceived, but only for the first message in each queue. Expired messages are not removed from the queue, and their disk space is not reclaimed, until they reach the front of the queue. Using TimeToBeReceived as a disk-saving measure on RabbitMQ is not ideal for queues with long-lived messages, like audit and forward, unless it is ensured that all messages use the same TimeToBeReceived. The TimeToBeReceived of other messages in front of a message in a queue will affect when the message is actually removed.

Azure transports

The Azure transports only evaluate the TimeToBeReceived for a message when the message is received from the queue. Expired messages are not removed from the queue and their disk space will not be reclaimed until they reach the front of the queue and a consumer tries to read them. Using TimeToBeReceived as a disk saving measure on the Azure transports is not a great choice for queues with long-lived messages like audit and forward.

SQL transport

The SQL Server transport evaluates the TimeToBeReceived for a message when the message is received from the queue and immediately discards expired messages. Disk space occupied by expired messages is not reclaimed until they reach the front of the queue.

Related Articles

  • Auditing Messages
    Configure where to send messages and it provides built-in message auditing for every endpoint.

Last modified