A Herd Of Rabbits Part 2: RabbitMQ Data Pipelines
abbitMQ is a powerful message borker allowing engineers to implement complex messaging topologies with relative ease. At the day job we used RabbitMQ as the backbone of our real time data infrastructure. In the previous post we setup a simple PostgreSQL trigger to send change capture messages to a RabbitMQ exchange. Conceptually, this is where we left off:
In this early stage, we basically have a fire-hose that we can selectively tap into. But we have no way to control the flow of data.
To recap a bit before we get too deep, we had a simple and manual way of handling real time operations. Effectively, we just baked all of the logic in the specific application code path.Read More