Skip to content

Building adaptive systems

[[Talk]] by Chris Keathley

Little's law

elements in the queue = arrival rate * avg processing time

For example:

requests = rps * ms

Overload mitigation

Arrival rate and processing time need be be under control. The only way to do this is to drop requests.

  • Load shedding: Upstream or downstream service drops requests
  • Autoscaling: It's not a solution. Adding more capacity helps. However the limit is just moved. Only using CPU for scaling is not enough.
  • Circuit breakers: Only a partial solution as they stop all traffic to the downstream service, even though some traffic could still be handled.

Adaptive concurrency

Dynamically find out the limits of sending and receiving systems and use load shedding.

Signals used to set limits:

  • Latency
  • Successful vs failed requests
  • Additive Increase Multiplicative Decrease: Success: limit + 1, Backoff: limit * 0,95

Source

  • Youtube talk
  • Regulartor Elixir library

#published

Links to this note

These notes are unpolished collections of thoughts, unfinished ideas, and things I want to remember later. In the spirit of learning in public, I'm sharing them here. Have fun exploring, if you want!
© 2020 by Adrian Philipp