matklad on cancellation terminology and crash-only shutdown

Logged at IST: 2026-09-01 21:29 IST

What it is: matklad's short terminology note separating synchronous cancellation, asynchronous cancellation, and graceful shutdown, shared by TigerBeetle for its crash-only design angle.

Gist: The post's useful move is to stop using “cancellation” and “shutdown” as one blob. Synchronous cancellation is a control-flow operation: canceling means the task has already finished by the next line, like stack unwinding through exceptions, returned errors, RAII, finally, with/try resources, or defer.

Asynchronous cancellation is different: it is a communication protocol. One party asks another party to stop, but the work may still be running until it acknowledges and joins. matklad's examples are CPU thread-pool jobs and io_uring operations, where buffers and OS resources must remain owned until outstanding work is actually done.

Graceful shutdown sits higher up as an application-level service pattern: stop accepting new connections, keep serving existing ones, and let a load balancer route new work elsewhere. The TigerBeetle section then makes the distributed-systems point. Grid.cancel is asynchronous cancellation, StateMachine.reset is synchronous cancellation, and a client shutdown comment calls something “graceful shutdown” even though matklad now thinks that name is wrong. TigerBeetle itself is crash-only: if the system must survive SIGKILL and power loss anyway, then intentionally exercising crash paths can simplify implementation and improve coverage. Tail-latency tolerance also handles gray failures, because a very slow node and a crashed node can look equivalent from the outside.

Newsletter angle: Sharp distributed-systems terminology note: don't call every cleanup path “graceful shutdown”; distinguish stack unwinding, async resource ownership, and app-level connection draining.