By Adrian Philipp
[[Talk]] by Chris Keathley
elements in the queue = arrival rate * avg processing time
Arrival rate and processing time need be be under control. The only way to do this is to drop requests.
- Load shedding: Upstream or downstream service drops requests
- Autoscaling: It's not a solution. Adding more capacity helps. However the limit is just moved. Only using CPU for scaling is not enough.
- Circuit breakers: Only a partial solution as they stop all traffic to the downstream service, even though some traffic could still be handled.
Dynamically find out the limits of sending and receiving systems and use load shedding.
Signals used to set limits:
- Successful vs failed requests
- Additive Increase Multiplicative Decrease: Success: limit + 1, Backoff: limit * 0,95
These notes are unpolished collections of thoughts, unfinished ideas, and things I want to remember later. In the spirit of learning in public, I'm sharing them here. Have fun exploring, if you want!