I’m curious in your experience how many clients can run pg_replicate at once?
With MySQL I saw the interesting use-case of the black hole storage engine to scale out replication logs but ultimately the only usage I’m aware of was for scaling other mysql read replicas.
The idea of scaling an application by tailing logs from a database sounds very interesting to me, and I’m curious if you’ve explored this at all. There’s of course things like Kafka (and then things like Debezium), but it’s hard to beat direct!
> I’m curious in your experience how many clients can run pg_replicate at once?
I'd expect more clients to put more pressure on the resource usage on the db. But it's not clear whether the relationship between number of clients and resource usage is linear, quadratic or something else since I haven't done benchmarking yet.
> The idea of scaling an application by tailing logs from a database sounds very interesting to me, and I’m curious if you’ve explored this at all. There’s of course things like Kafka (and then things like Debezium), but it’s hard to beat direct!
It doesn't exist yet, but I was thinking of creating a sink which exposes Postgres WAL via websockets. This way the number of clients might scale much better.
With MySQL I saw the interesting use-case of the black hole storage engine to scale out replication logs but ultimately the only usage I’m aware of was for scaling other mysql read replicas.
The idea of scaling an application by tailing logs from a database sounds very interesting to me, and I’m curious if you’ve explored this at all. There’s of course things like Kafka (and then things like Debezium), but it’s hard to beat direct!