Datafusion, and Ballista by definition, also provides a Dataframe API that let's you construct queries programmatically. It also has preliminary support for UDFs.
We also have community members implementing Spark native executors using Datafusion, which showed significant speed improvements in the initial PoC.
We also have community members implementing Spark native executors using Datafusion, which showed significant speed improvements in the initial PoC.