LakeSail

LakeSail is an open source Rust framework designed to unify stream processing, batch processing, and compute-intensive (artificial intelligence) workloads. It provides a developer-friendly, interoperable and observable platform for big data and artificial intelligence, promising an easy transition from Apache Spark. LakeSail provides an alternative implementation of the Spark SQL and Spark DataFrame APIs, promising significant performance improvements and cost savings.

LakeSail can be used as an alternative implementation of the Spark SQL and Spark DataFrame APIs. Install via pip (pysail[spark]). It is possible to start a Sail server and connect from PySpark with minimal code changes. Kubernetes deployment is also supported.

Unify streaming, batch and AI workloads
Alternative implementation of Spark SQL and DataFrame API
Powered by Rust for great performance
Commercial support options available