Introduction

Reliability in an event-driven architecture requires more than publishing messages and hoping everything works. I need to think about at-least-once delivery, idempotency, local transactions with Outbox, observable states, ordering by key and compensation. When I leave these points aside, the system becomes a laboratory of duplicates, poorly managed retries and inconsistent states between a local operation and the next asynchronous processing step.

This article shows a practical path to build a flow with PostgreSQL and Azure Service Bus Emulator, keeping the complexity in a reproducible lab. The central idea is simple: I record the local effect and the intent to publish in a single transaction, then process events with inbox and saga, without pretending that exactly-once is a real promise in distributed systems.

Prerequisites

I recommend Docker Desktop or Docker Engine, Compose, .NET 10 and a local environment with enough memory to run PostgreSQL, SQL Server for the emulator and the API. The complete example is available in the blog’s samples repository: BlogSamples/Messaging/EventDriven/.

Infrastructure and structure required for a robust implementation

A robust event implementation is not just an HTTP endpoint and a worker. It needs a small local platform with four well-defined blocks:

  1. an API to accept the business command and persist the local effect;
  2. a PostgreSQL database as the source of truth for the order state, the saga and the integration messages;
  3. a local broker, in this case the Azure Service Bus Emulator, for transport, routing, sessions and DLQ;
  4. workers to publish Outbox messages and process messages idempotently.

The minimal structure of the example must include:

  • Endpoints/ to create and query orders;
  • Contracts/ with versioned envelopes and DTOs;
  • Persistence/ with DbContext, migrations and models for order, outbox, inbox and saga;
  • Saga/ with the orchestrator and the state transitions;
  • ServiceBus/ with publisher, consumers and adapters for the emulator;
  • docker-compose.yml and README.md to bring the stack up locally.

In addition, the solution needs some operational components that many examples leave out:

  • separate tables for order, stock, payment, saga, outbox and inbox;
  • correlation IDs and causation IDs in every event;
  • controlled retries, DLQ and explicit handling of transient failures;
  • a local idempotency policy for each consumer;
  • enough observability to answer “where did the saga get stuck?”.

Without these blocks, the flow may look correct in a simple scenario and fail in production when the network, the broker or the system state start to vary.

Why an event-driven architecture turns into chaos

Chaos starts when I rely on two separate writes and expect the second one to happen. In distributed systems, a message can be sent, the database can fail on commit and the application can try again. Without handling, I end up with a local state different from what was published. The Outbox pattern solves this problem by making publication part of the local transaction. The Inbox pattern solves consumer duplication, which is one of the most common problems in at-least-once delivery.

This kind of problem shows up when I need to integrate different domains, such as order, stock, payment and notification, and each of them has its own processing cycle. The common mistake is thinking that the event is just an operational detail. In practice, it must be treated as part of the business contract. If I do not think about retry, compensation, ordering, observability and idempotency from the start, the application begins to accumulate unnoticed states and with every failure I gain more noise than value.

I also see teams trying to solve everything with polling, aggressive retries and scattered logs. That usually makes things worse. An event-driven architecture is not a magic mechanism to make everything distributed and resilient; it is a way to organize complexity when the operation needs to be decoupled and tolerate temporary failures. If I do not have a flow with explicit states, the user experience becomes a mix of in-flight transactions and invisible failures.

The example architecture with PostgreSQL and Azure Service Bus

The local example uses PostgreSQL as the application database and an Azure Service Bus Emulator topic with filtered subscriptions. The API creates an order, records an event in the Outbox and an orchestrator consumes the flow. The flow is simple, but it already covers the decisions that most often cause problems: per-order ordering, compensation, retry and DLQ.

Comparison between the happy path and the compensation path of a saga

The decision to use PostgreSQL as the application’s system of record comes from the fact that I need a strong local transaction to record the business effect and, at the same time, persist the intent to publish. Azure Service Bus acts as the asynchronous delivery and orchestration layer between independent steps. This design works well when I need an observable timeline and a single point where the order state can be queried. In other words, the database takes care of state; the broker takes care of propagation.

I find this combination useful when the flow involves multiple systems, each with variable response time, and I need to prevent a failure in one subsystem from bringing down the whole operation. The point is not to make everything distributed for the sake of it; the point is to separate what must be locally consistent from what can be processed asynchronously and compensated.

⚠️ Warning: if your problem is still simple, with a single transaction and no need for decoupling, I would not start with this architecture. The cost of operating Outbox, Inbox and Saga is real and must be justified by a business or integration problem.

📂 Source Code: The complete example is available in the blog’s samples repository: BlogSamples/Messaging/EventDriven/

At-least-once delivery and duplicate messages

At-least-once delivery means that a message can be received more than once. This is not a bug; it is the cost of operating with retries and temporary failures. What I need to do is treat duplication as a normal case. The Inbox is the right tool for this, because it stores the message identifier and the consumer, preventing improper reprocessing.

When I say “at least once”, I mean that the system can receive the same message more than once and, therefore, every side effect must be idempotent. That applies to database writes, status updates, sending e-mails, creating audit records and any action that cannot be repeated carelessly. The broker can confirm delivery and then the local process can fail before recording the state. In that case, on the next attempt, I will receive the same message again. Without a deduplication mechanism, I can create two reservations, two authorizations or two payment records for the same original intent.

This is one of the reasons why I never trust “exactly-once delivery” as an implementation promise. What exists, in general, is a combination of at-least-once on the transport, idempotency on the consumer and a compensation layer for cases where the operation cannot move forward.

Consumer idempotency with the Inbox pattern

The consumer must be idempotent by nature. If it receives a duplicate, it must recognize it and return success without repeating side effects. The inbox state is the record that the message has already been processed, regardless of broker retries or replays.

The simplest way to implement this is to persist, in the same transactional context as the business effect, the message identifier and the consumer name. When the same message arrives again, I check it and cut the flow before any additional action. In real systems, this is usually done with an Inbox table with a composite constraint on (consumer, message_id), because that gives me a strong guarantee that two concurrent executions will not perform the same effect twice.

This matters because the broker does not replace the application’s responsibility to be safe. Service Bus can have duplicate detection in some configurations, but that does not remove the need for local deduplication control. The Inbox pattern is the layer that makes the consumer recognize “I have already handled this” and not repeat the action.

Transactional Outbox in PostgreSQL and the dual write problem

Dual write is the classic consistency problem between database and broker. The Outbox solves this scenario by persisting the intent to publish in the same transaction as the local effect. Then a separate publisher reads the outbox, sends the message and marks the result with a controlled update. This creates a window of uncertainty, but it does not make the application lie about what was confirmed.

The practical implementation usually goes like this: I start a transaction in PostgreSQL, persist the business state, insert the message into the Outbox and only then commit. If the commit succeeds, the message is guaranteed as a publication intent. Next, a second process — almost always a worker — reads the pending messages and publishes them to the broker. The crucial point is that I do not update the status to “published” until publication has actually happened. This design protects me from having a local effect without the corresponding notification.

This pattern is much more reliable than trying to publish first and then record the local state. If I do that, I run the risk of losing the message or creating a business state without a corresponding event. The Outbox is one of the few patterns that truly reduces the consistency problem between database and messaging without requiring a complex distributed protocol.

ℹ️ Info: I detail the dual write problem and the full implementation of the Transactional Outbox Pattern, including the asynchronous publisher and the consumer’s idempotency obligations, in the second part of this series: Transactional Outbox Pattern: Solving the Dual Write.

Eventual consistency without hiding state from the user

Eventual consistency does not mean “leaving the user in the dark”. I need to expose the current order state, the saga step and the timestamps. An order may be in processing, but the experience must say exactly where it is and what is happening. Observable state is part of the design, not an optional detail.

When I really need this architecture

I do not implement Outbox, Inbox and Saga because it “looks modern”. I implement them because there is a real operational problem: a business operation needs to go through multiple steps, some of them asynchronous, and I need to maintain trust even when one part fails. If I am dealing with a simple create-and-read flow, with a single transaction and no compensation, I probably do not need this.

The decision usually becomes clearer when I see signals like these:

  • one step depends on another system that may be slow or fail;
  • I need to avoid having the whole operation rolled back because of a partial failure;
  • I need to keep traceability of the flow across domains;
  • message duplication is a real possibility and not a rare case;
  • the user needs to know which phase of the operation they are in.
SituationRecommendation
simple CRUD, a single local transaction and no external integrationkeep the flow synchronous and simple
an order needs to reserve stock, charge a card and notify the customerconsider Outbox + Saga + Inbox
several services react to events and the flow must be compensableuse saga and explicit states
the problem is only performance and not distributed consistencydo not start with events

The practical rule is: if I need a distributed operation that can fail at an intermediate point and still maintain a reliable business experience, I should consider this architecture.

How I would implement this in practice

The implementation starts with a design decision, not with a tool. I first define the business steps and their states. Then I separate two things: what must be locally consistent and what can be processed asynchronously.

  1. I start with a local transaction in PostgreSQL to save the business effect and record a publication intent in the Outbox.
  2. I add an Inbox table to ensure that the same event does not produce the same effect twice.
  3. I create a versioned envelope with message_id, correlation_id, causation_id, event_type, schema_version and a typed payload.
  4. I set up a worker responsible for reading pending Outbox messages and publishing them to the Service Bus topic.
  5. I implemented consumers with PeekLock, AutoCompleteMessages=false and completion only after the local effect has been saved.
  6. I keep an explicit saga state to decide whether to advance, compensate or stop the flow.
  7. I record logs and metrics with a correlation ID so I can analyze the whole flow when something goes wrong.

This roadmap is simple to explain, but it requires discipline. Most problems in event-driven architectures come from the team treating the event as a technical detail rather than a business contract.

Saga by choreography or orchestration: which to choose

An orchestrated saga is easier to trace when I have a known business flow and an explicit orchestrator. Choreography fits better when each piece reacts locally to events and the flow is broadly distributed. For this example, I chose orchestration because it makes state and compensation easier to explain and test.

In practice, I choose orchestration when the flow is essentially a state machine: order created, stock reserved, payment processed, compensation in case of failure. In this scenario, I need an entity that tracks the global state of the business process and decides the next step. Choreography makes more sense when I have many services reacting to events and the complexity is distributed among them. The advantage of choreography is less coupling; the disadvantage is that understanding the complete execution becomes harder and traceability tends to get worse.

A saga is not a solution for every problem. It is the correct way to model a sequence of steps that must be compensated when one piece fails. If I simply have a linear process and will not need a partial rollback, I probably do not need a Saga.

Per-order ordering with Azure Service Bus sessions

Service Bus sessions preserve ordering by business key, in this case by order. This is not equivalent to a global ordering across all orders; what I gain is local ordering per session, which is essential in flows that must be processed sequentially without blocking other orders.

Retry, PeekLock and Dead Letter Queue

PeekLock lets me decouple processing from completion. If something fails, I abandon the message for retry. If the error is persistent or invalid, I send it to the DLQ with a useful reason. This is much safer than simply losing the message or completing it too early.

I like to think of three categories of failure when I implement this. The first is a transient failure: database unavailable, network timeout, connection exhaustion. In that case, I abandon the message and let the broker deliver it again. The second is a contract or validation failure: the payload is malformed, an unexpected field arrived, the schema changed incompatibly. Here I do not want to reprocess indefinitely. I prefer to record the cause and send it to the DLQ with a clear reason. The third is a non-recoverable business failure, such as an event that arrives when the order is in an inconsistent state. In that case, the saga must decide whether it describes a contract violation or simply a tolerable inconsistency to be observed.

This model makes my system predictable. I do not hide the failure under infinite retries, and I also do not lose the information when a message is definitively invalid.

Event contracts and schema evolution

Events need clear contracts. I cannot rely only on the payload without versioning. The example uses versioned envelopes with correlation IDs, causation IDs and event type. This helps evolve the schema without turning the saga into a chaos of incompatibilities.

Observability with correlation and causation IDs

Observability improves a lot when every message carries correlation and causation IDs. I can trace the whole path of the order, correlate logs, metrics and traces and locate exactly where the saga deviated from the normal flow.

When not to use an event-driven architecture

I do not use events when the problem is simple, with strong immediate consistency and no need for decoupling. When a simple transaction already solves it, an event-driven architecture can introduce too much cost and complexity. The practical rule is: if I do not need asynchronous flow, compensation and distributed observability, I probably should not start with events.

Practical Example: order, stock and payment

The complete example is available at BlogSamples/Messaging/EventDriven/. It implements a simple flow in which an order creates an outgoing event, stock responds and then payment is processed. In case of rejection, the saga emits compensation and cancels the order.

Happy path and sad path

On the happy path, the order starts as Criado (created), emits an event to reserve stock, receives confirmation, moves on to payment and ends with the order approved. On the sad path, some step fails: stock unavailable or payment declined. In that case, the saga does not try to move forward; it changes the state to a compensation phase and publishes a cancellation or reversal event.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
public sealed class PedidoSagaState
{
    public Guid PedidoId { get; set; }
    public SagaStatus Status { get; set; }
    public SagaEtapa Etapa { get; set; }
    public List<string> Processados { get; set; } = [];
    public List<SagaOutboxMessage> Outbox { get; set; } = [];
}

public static class PedidoSagaOrquestrador
{
    public static PedidoSagaState HandlePedidoCriado(PedidoSagaState saga)
    {
        saga.Status = SagaStatus.EmAndamento;
        saga.Etapa = SagaEtapa.AguardandoReserva;
        saga.Outbox.Add(new SagaOutboxMessage
        {
            TipoEvento = "ReservarEstoque",
            CorrelationId = saga.PedidoId.ToString()
        });

        return saga;
    }

    public static PedidoSagaState HandleReservaConfirmada(PedidoSagaState saga)
    {
        saga.Etapa = SagaEtapa.AguardandoPagamento;
        saga.Processados.Add("estoque-reservado");
        return saga;
    }

    public static PedidoSagaState HandlePagamentoRecusado(PedidoSagaState saga)
    {
        saga.Status = SagaStatus.Compensando;
        saga.Etapa = SagaEtapa.Cancelado;
        saga.Outbox.Add(new SagaOutboxMessage
        {
            TipoEvento = "CancelarPedido",
            CorrelationId = saga.PedidoId.ToString()
        });

        return saga;
    }
}

🧠 Note: the saga state is the source of truth for deciding whether I advance, compensate or continue the flow.

Tips and Good Practices

  • Keep the saga state explicit and observable.
  • Treat duplication as the rule, not the exception.
  • Retry and DLQ semantics must be documented.
  • Version contracts and evolve them in a controlled way.
  • Use sessions only when ordering by business key matters.
  • Do not use events for everything. If the operation is local and synchronous, the complexity of messaging may not be worth the cost.
  • Define clearly what is compensable and what is not. Not every failure has a real rollback.
  • Make the consumer idempotent before thinking about high throughput.
  • Monitor correlation IDs and causation IDs from the start, because that is what will make debugging feasible when the event chain grows.

Objective Summary

  • Outbox — makes publication part of the local transaction and reduces the dual write problem.
  • Inbox — absorbs duplication and makes the consumer idempotent.
  • Saga — organizes compensation and business states in distributed flows.
  • Azure Service Bus — offers topics, subscriptions, sessions and DLQ for asynchronous flows.
  • PostgreSQL — provides a transactional database to persist the application state and the outbox.

Read Also

References