Sagas

When building distributed systems, it is crucial to ensure that the system remains consistent even in the presence of failures. One way to achieve this is by using the Saga pattern.

What is a Saga?

A Saga is a design pattern for handling transactions that span multiple services. It breaks the process into a sequence of local operations, each with a corresponding compensating action.

If a failure occurs partway through, these compensations are triggered to undo completed steps, ensuring your system stays consistent even when things go wrong.

How does Restate help?

Restate makes it easy to implement resilient sagas in your code:

• Durable Execution: Restate guarantees that your code runs to completion. If a transient failure occurs, Restate automatically retries from the point of failure and ensures that all compensations run.
• Resilience built-in: No need to manually track state or retry logic. Restate handles all persistence and compensation orchestration for you.
• Code-first approach: Define sagas using regular code, no DSLs. Track compensations in a list, and execute them on non-transient failures.

Example

Here is a typical travel booking workflow, where you book a flight, then rent a car, and finally book a hotel. If any step fails for a non-transient reason (e.g. driver license not accepted, hotel full), we want to roll back the previous steps to keep the system consistent.

Restate lets us implement this purely in code without any DSLs or extra infrastructure.

• Wrap your business logic in a try-block, and throw a terminal error for cases where you want to compensate and finish.
• For each step you do in your try-block, add a compensation to a list.
• In the catch block, in case of a terminal error, you run the compensations in reverse order, and rethrow the error.

Note that for Golang we use defer to run the compensations at the end.

TypeScript

const bookingWorkflow = restate.service({
  name: "BookingWorkflow",
  handlers: {
    run: async (ctx: restate.Context, req: BookingRequest) => {
      const { customerId, flight, car, hotel } = req;
      // create a list of undo actions
      const compensations = [];

      try {
        // For each action, we register a compensation that will be executed on failures
        compensations.push(() => ctx.run("Cancel flight", () => flightClient.cancel(customerId)));
        await ctx.run("Book flight", () => flightClient.book(customerId, flight));

        compensations.push(() => ctx.run("Cancel car", () => carRentalClient.cancel(customerId)));
        await ctx.run("Book car", () => carRentalClient.book(customerId, car));

        compensations.push(() => ctx.run("Cancel hotel", () => hotelClient.cancel(customerId)));
        await ctx.run("Book hotel", () => hotelClient.book(customerId, hotel));
      } catch (e) {
        // Terminal errors are not retried by Restate, so undo previous actions and fail the workflow
        if (e instanceof restate.TerminalError) {
          // Restate guarantees that all compensations are executed
          for (const compensation of compensations.reverse()) {
            await compensation();
          }
        }
        throw e;
      }
    },
  },
});

restate.endpoint().bind(bookingWorkflow).listen(9080);

Java

@Service
public class BookingWorkflow {

  public record BookingRequest(
      String customerId, FlightRequest flight, CarRequest car, HotelRequest hotel) {}

  @Handler
  public void run(Context ctx, BookingRequest req) throws TerminalException {
    // Create a list of undo actions
    List<Runnable> compensations = new ArrayList<>();

    try {
      // For each action, we register a compensation that will be executed on failures
      compensations.add(() -> ctx.run("Cancel flight", () -> FlightClient.cancel(req.customerId)));
      ctx.run("Book flight", () -> FlightClient.book(req.customerId, req.flight()));

      compensations.add(() -> ctx.run("Cancel car", () -> CarRentalClient.cancel(req.customerId)));
      ctx.run("Book car", () -> CarRentalClient.book(req.customerId, req.car()));

      compensations.add(() -> ctx.run("Cancel hotel", () -> HotelClient.cancel(req.customerId)));
      ctx.run("Book hotel", () -> HotelClient.book(req.customerId, req.hotel()));
    }
    // Terminal exceptions are not retried by Restate. We undo previous actions and fail the
    // workflow.
    catch (TerminalException e) {
      // Restate guarantees that all compensations are executed
      for (Runnable compensation : compensations) {
        compensation.run();
      }
      throw e;
    }
  }

  public static void main(String[] args) {
    RestateHttpServer.listen(Endpoint.bind(new BookingWorkflow()));
  }
}

Kotlin

@Service
class BookingWorkflow {

  @Handler
  suspend fun run(ctx: Context, req: BookingRequest) {

    // Create a list of undo actions
    val compensations = mutableListOf<suspend () -> Unit>()

    try {
      // For each action, we register a compensation that will be executed on failures
      compensations.add { ctx.runBlock("Cancel flight") { cancelFlight(req.customerId) } }
      ctx.runBlock("Book flight") { bookFlight(req.customerId, req.flight) }

      compensations.add { ctx.runBlock("Cancel car") { cancelCar(req.customerId) } }
      ctx.runBlock("Book car") { bookCar(req.customerId, req.car) }

      compensations.add { ctx.runBlock("Cancel hotel") { cancelHotel(req.customerId) } }
      ctx.runBlock("Book hotel") { bookHotel(req.customerId, req.hotel) }
    }
    // Terminal exceptions are not retried by Restate. We undo previous actions and fail the
    // workflow.
    catch (e: TerminalException) {
      // Restate guarantees that all compensations are executed
      compensations.reversed().forEach { it() }
      throw e
    }
  }
}

fun main() {
  RestateHttpServer.listen(endpoint { bind(BookingWorkflow()) })
}

Python

booking_workflow = restate.Service("BookingWorkflow")


@booking_workflow.handler()
async def run(ctx: restate.Context, req: BookingRequest):
    # Create a list of undo actions
    compensations = []

    try:
        # For each action, we register a compensation that will be executed on failures
        compensations.append(
            lambda: ctx.run("Cancel flight", flight_client.cancel, args=(req.customer_id,))
        )
        await ctx.run("Book flight", flight_client.book, args=(req.customer_id, req.flight))

        compensations.append(
            lambda: ctx.run("Cancel car", car_rental_client.cancel, args=(req.customer_id,))
        )
        await ctx.run("Book car", car_rental_client.book, args=(req.customer_id, req.car))

        compensations.append(
            lambda: ctx.run("Cancel hotel", hotel_client.cancel, args=(req.customer_id,))
        )
        await ctx.run("Book hotel", hotel_client.book, args=(req.customer_id, req.hotel))

    # Terminal errors are not retried by Restate, so undo previous actions and fail the workflow
    except TerminalError as e:
        # Restate guarantees that all compensations are executed
        for compensation in reversed(compensations):
            await compensation()
        raise e


app = restate.app([booking_workflow])

if __name__ == "__main__":
    import hypercorn
    import asyncio

    conf = hypercorn.Config()
    conf.bind = ["0.0.0.0:9080"]
    asyncio.run(hypercorn.asyncio.serve(app, conf))

Go

type BookingWorkflow struct{}

func (BookingWorkflow) Run(ctx restate.Context, req BookingRequest) (err error) {

  // Create a list of undo actions
  var compensations []func() (restate.Void, error)

  // Run compensations at the end if err != nil
  defer func() {
    if err != nil {
      for _, compensation := range compensations {
        if _, compErr := compensation(); compErr != nil {
          err = compErr
        }
      }
    }
  }()

  compensations = append(compensations, func() (restate.Void, error) {
    return restate.Run(ctx,
      func(ctx restate.RunContext) (restate.Void, error) {
        return CancelFlight(req.CustomerId)
      },
      restate.WithName("Cancel flight"),
    )
  })
  if _, err = restate.Run(ctx,
    func(ctx restate.RunContext) (restate.Void, error) {
      return BookFlight(req.CustomerId, req.Flight)
    },
    restate.WithName("Book flight"),
  ); err != nil {
    return err
  }

  compensations = append(compensations, func() (restate.Void, error) {
    return restate.Run(ctx,
      func(ctx restate.RunContext) (restate.Void, error) {
        return CancelCar(req.CustomerId)
      },
      restate.WithName("Cancel car"),
    )
  })
  if _, err = restate.Run(ctx,
    func(ctx restate.RunContext) (restate.Void, error) {
      return BookCar(req.CustomerId, req.Car)
    },
    restate.WithName("Book car"),
  ); err != nil {
    return err
  }

  compensations = append(compensations, func() (restate.Void, error) {
    return restate.Run(ctx,
      func(ctx restate.RunContext) (restate.Void, error) {
        return CancelHotel(req.CustomerId)
      },
      restate.WithName("Cancel hotel"),
    )
  })
  if _, err = restate.Run(ctx,
    func(ctx restate.RunContext) (restate.Void, error) {
      return BookHotel(req.CustomerId, req.Hotel)
    },
    restate.WithName("Book hotel"),
  ); err != nil {
    return err
  }

  return nil
}

func main() {
  if err := server.NewRestate().
    Bind(restate.Reflect(BookingWorkflow{})).
    Start(context.Background(), ":9080"); err != nil {
    log.Fatal(err)
  }
}

Example not available in your language?

This pattern is implementable with any of our SDKs. We are still working on translating all patterns to all SDK languages. If you need help with a specific language, please reach out to us via Discord or Slack.

When to use Sagas

Restate automatically retries all transient failures, like network hiccups or temporary service outages. But not all failures are temporary.

For these failures, sagas are essential:

1. Business logic requirements:

   • Some failures are not transient but a business decision (e.g. “Hotel is full” or “Driver license not accepted”), retrying won't help.
   • In this case, you can throw a terminal error to stop the execution and trigger the compensations.

2. User/system-initiated cancellations:

   • If a user cancels a long-running invocation (say via UI or CLI), this triggers a terminal error.
   • Restate will not retry.
   • Again, a saga can kick in to undo previous successful operations so the system doesn't end up in an inconsistent state (e.g., booking a hotel but not a car).

Running the example

1

Download the example

ts

restate example typescript-patterns-use-cases && cd typescript-patterns-use-cases

java

restate example java-patterns-use-cases && cd java-patterns-use-cases

kotlin

restate example kotlin-patterns-use-cases && cd kotlin-patterns-use-cases

python

restate example python-patterns-use-cases && cd python-patterns-use-cases

go

restate example go-patterns-use-cases && cd go-patterns-use-cases

2

Start the Restate Server

restate-server

3

Start the Service

ts

npx tsx watch ./src/sagas/booking_workflow.ts

java

./gradlew -PmainClass=my.example.sagas.BookingWorkflow run

kotlin

./gradlew -PmainClass=my.example.sagas.BookingWorkflowKt run

python

python sagas/app.py

go

go run ./src/sagas

4

Register the services

restate deployments register localhost:9080

5

Send a request

ts

curl localhost:8080/BookingWorkflow/run --json '{
        "flight": {
            "flightId": "12345",
            "passengerName": "John Doe"
        },
            "car": {
            "pickupLocation": "Airport",
            "rentalDate": "2024-12-16"
        },
            "hotel": {
            "arrivalDate": "2024-12-16",
            "departureDate": "2024-12-20"
        }
    }'

java

curl localhost:8080/BookingWorkflow/run --json '{
        "flight": {
            "flightId": "12345",
            "passengerName": "John Doe"
        },
            "car": {
            "pickupLocation": "Airport",
            "rentalDate": "2024-12-16"
        },
            "hotel": {
            "arrivalDate": "2024-12-16",
            "departureDate": "2024-12-20"
        }
    }'

kotlin

curl localhost:8080/BookingWorkflow/run --json '{
        "flight": {
            "flightId": "12345",
            "passengerName": "John Doe"
        },
            "car": {
            "pickupLocation": "Airport",
            "rentalDate": "2024-12-16"
        },
            "hotel": {
            "arrivalDate": "2024-12-16",
            "departureDate": "2024-12-20"
        }
    }'

python

curl localhost:8080/BookingWorkflow/run --json '{
        "flight": {
            "flightId": "12345",
            "passengerName": "John Doe"
        },
            "car": {
            "pickupLocation": "Airport",
            "rentalDate": "2024-12-16"
        },
            "hotel": {
            "arrivalDate": "2024-12-16",
            "departureDate": "2024-12-20"
        }
    }'

go

curl localhost:8080/BookingWorkflow/Run --json '{
        "flight": {
            "flightId": "12345",
            "passengerName": "John Doe"
        },
            "car": {
            "pickupLocation": "Airport",
            "rentalDate": "2024-12-16"
        },
            "hotel": {
            "arrivalDate": "2024-12-16",
            "departureDate": "2024-12-20"
        }
    }'

4

Check the UI or service logs

See in the Restate UI (localhost:9070) how all steps were executed, and how the compensations were triggered because the hotel was full.

Advanced: Idempotency and compensations

Since sagas in Restate are implemented in user code, compensations are flexible and powerful, as long as they're idempotent: you can reset service state, call other services to undo prior actions, use ctx.run to delete rows or reverse database operations.

The example above uses the customer ID to guarantee idempotency, so that on retries it will not create duplicate bookings or rentals. The example assumes that the API provider deduplicates the requests based on this ID.

Based on the API you are using, generating the idempotency key and registering the compensation can be done in different ways:

1. Two-phase APIs: First you reserve, then confirm or cancel. Register the compensation after reservation, when you have the resource ID. Reservations that are not confirmed, get automatically cancelled by the API after a timeout.

TypeScript

const bookingId = await ctx.run(() =>
  flightClient.reserve(customerId, flight)
);
compensations.push(() =>
  ctx.run(() => flightClient.cancel(bookingId))
);

// ... do other work, like reserving a car, etc. ...

await ctx.run(() => flightClient.confirm(bookingId));

Python

booking_id = await ctx.run("reserve", flight_client.reserve, args=(customer_id, flight))
compensations.append(
    lambda: ctx.run("cancel", flight_client.cancel, args=(booking_id,))
)

#  ... do other work, like reserving a car, etc. ...

await ctx.run("confirm", flight_client.confirm, args=(booking_id,))

Java

String bookingId = ctx.run(String.class, () -> FlightClient.reserve(flight));
compensations.add(() -> ctx.run(() -> FlightClient.cancel(bookingId)));

// ... do other work, like reserving a car, etc. ...

ctx.run(() -> FlightClient.confirm(bookingId));

Kotlin

// For each action, we register a compensation that will be executed on failures
val bookingId = ctx.runBlock { reserveFlight(customerId, flight) }
compensations.add { ctx.runBlock { cancelFlight(bookingId) } }

// ... do other work, like reserving a car, etc. ...

compensations.add { ctx.runBlock { confirmFlight(bookingId) } }

Go

bookingId, err := restate.Run(ctx, func(ctx restate.RunContext) (string, error) {
  return ReserveFlight(req.CustomerId, req.Flight)
})
if err != nil {
  return err
}

compensations = append(compensations, func() (restate.Void, error) {
  return restate.Run(ctx, func(ctx restate.RunContext) (restate.Void, error) {
    return CancelFlight(bookingId)
  })
})

// ... do other work, like reserving a car, etc. ...

if _, err = restate.Run(ctx, func(ctx restate.RunContext) (restate.Void, error) {
  return ConfirmFlight(bookingId)
}); err != nil {
  return err
}

2. One-shot APIs with idempotency key: First, you generate an idempotency key and persist it in Restate. Then, you register the compensation (e.g. refund), and finally do the action (e.g. charge). We need to register the compensation before doing the action, because there is a chance that the action succeeded but that we never got the confirmation.

TypeScript

const paymentId = ctx.rand.uuidv4();
compensations.push(() =>
  ctx.run(() => paymentClient.refund(paymentId))
);
await ctx.run(() => paymentClient.charge(paymentInfo, paymentId));

Python

payment_id = await ctx.run("payment id", lambda: str(uuid.uuid4()))
compensations.append(
    lambda: ctx.run("refund", payment.refund, args=(payment_id,))
)
await ctx.run("charge", payment.charge, args=(payment_info, payment_id))

Java

String paymentId = ctx.random().nextUUID().toString();
compensations.add(() -> ctx.run(() -> PaymentClient.refund(paymentId)));
ctx.run(() -> PaymentClient.charge(paymentInfo, paymentId));

Kotlin

val paymentId = ctx.random().nextUUID().toString()
compensations.add { ctx.runBlock { refund(paymentId) } }
ctx.runBlock { charge(paymentInfo, paymentId) }

Go

paymentID := restate.Rand(ctx).UUID().String()

// Register the refund as a compensation, using the idempotency key
compensations = append(compensations, func() (restate.Void, error) {
  return restate.Run(ctx, func(ctx restate.RunContext) (restate.Void, error) {
    return Refund(paymentID)
  })
})

// Do the payment using the idempotency key
if _, err = restate.Run(ctx, func(ctx restate.RunContext) (restate.Void, error) {
  return Charge(paymentID, req.PaymentInfo)
}); err != nil {
  return err
}

Related resources

• Error Handling guide
• Terminal errors: Java / TS / Go / Python / Rust
• Cancellation of invocations
• Blog post: Graceful cancellations: How to keep your application and workflow state consistent 💪
• Microservice orchestration use case page