1. Author

    Good question! Yes, if you would implement a way to send a signal to the slow consumer, it could react to that and try to fix the situation, e.g. by incrementing the amount of worker threads. But it depends on the case because, if the issue is in the network, scaling up wouldn’t solve it.

  2. In Computer Science, we can implement Pipes and Filters. It is Producer-Consumer. Filter1 is called a Producer and Filter2 is called a Consumer. Both filters are connected by a pipe called a Queue. Can we auto scale up a slow consumer into more threads based on its queue getting quite full, and then auto scale down when the queue does not have enough data to employ multiple threads anymore?

  3. what’s frightening is that programmers actually look at this and start writing code for re-requesting missing records. I can image such a scenario in a few cases, but in general the DB should not send missing blocks, this would mean that the client has to be able to detect missing blocks. Why is the server sending this?

    1. Author

      Well, as stated in the post you should choose wisely whether you are dropping or blocking. In general, these dropping use cases are great when it’s more important to have a recent piece of information than having them all (e.g. video streaming, stock values, the last state of an entity, etc.). If programmers find themselves writing code to requesting missing blocks, this allow-drops reactive style is definitely not the right solution for that problem.


This site uses Akismet to reduce spam. Learn how your comment data is processed.