APIs, Queues, Back Pressure, Load Shedding and Idempotency

People misuse queues all the time. The most egregious case being to fix issues with slow apps, and consequently, with overload.


You'll need to pick what has to give whenever stuff goes bad. You'll have to pick between blocking on input (back-pressure), or dropping data on the floor (load-shedding).


The real problem is that everyone involved used queues as an optimization mechanism.


But someone should have picked what had to give: do you stop people from inputting stuff in the system, or do you shed load. Those are inescapable choices, where inaction leads to system failure.


a proper idempotent API with end-to-end principles in mind will make it so these instances of back-pressure and load shedding should rarely be a problem for your callers, because they can safely retry requests and know if they worked.

-- Fred Hebert

from "Queues Don't Fix Overload"

Quoted on Thu Nov 20th, 2014