Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

can you just buffer some writes in Vector and eliminate Kafka?

I setup some Vector to buffer ElasticSearch writes years ago, also for logs, it ran so well without any problems that I almost fogot about it.



Or vice versa: make ClickHouse ingest batches directly from Kafka. Messages are already buffered in Kafka, I don't get why Vector is necessary here.


tbh the only thing they needed was a correct schema that didn’t constantly spawn new parts and async_insert enabled.


https://clickhouse.com/docs/knowledgebase/kafka-to-clickhous...

For anyone if curious.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: