What Is Backpressure? Protecting Systems from Overload

What happens when a producer generates data faster than a consumer can handle? Memory overflow? System crash? Backpressure controls data flow based on consumer capacity.

The Problem

Producer: 10,000 msg/s → Consumer: 1,000 msg/s → Buffer grows → OOM! 💥

Backpressure Strategies

1. Drop — Discard excess messages (acceptable for logs, sensors)
2. Buffer — Queue with size limits (producer waits when full)
3. Throttle — Slow down the producer
4. Pull-based — Consumer requests only what it can handle

Node.js Stream Backpressure

readable.on('data', (chunk) => {
  if (!writable.write(chunk)) {
    readable.pause();
    writable.once('drain', () => readable.resume());
  }
});

Watermark Strategy

High Watermark (80%): Slow producer
Low Watermark (20%):  Resume producer

Real-World Examples

| System | Mechanism | |--------|-----------| | TCP | Receive window | | Node.js | pause/resume, drain | | Kafka | Consumer lag, poll config | | RxJS | buffer, throttle, sample |

Best Practices

1. Never use unbounded buffers | 2. Watermarks for flow control
2. Monitor buffer fill rate | 4. Prefer pull-based | 5. Alert at 80% capacity

Conclusion

Backpressure is critical for data flow reliability. Without managing the speed difference between producer and consumer, system failure is inevitable.

Learn backpressure and reactive patterns on LabLudus.