Haven't used it personally, but I would suggest looking into Apache Hudi | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		loginx 7 months ago \| parent \| context \| favorite \| on: Fivetran to acquire Census Haven't used it personally, but I would suggest looking into Apache Hudi

zoogeny 7 months ago [–]

Good to know about but looks more like an open source snowflake (e.g. data lake). Fivetran and Census are the in/out process layers that bookend the data lake. Although, Hudi does look like it has some of that functionality baked in.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact