Introduction
Run the Spark code you already have, in the AWS account you already own, without operating the cluster.
LakeSail is for teams with PySpark jobs, SQL, and notebooks that need somewhere to run: on their own infrastructure, on their own data, without a platform team to keep it all up.
Spark and Python, as you write them
LakeSail runs PySpark and Spark SQL as they are; only where the code runs changes. Python runs inside the engine, and a compute profile carries the packages and environment your code expects, so every job, session, and notebook starts with the same setup.
To find out about your own code, check it against Sail before you set anything up.
What you can do
Put batch work on a schedule. A job is versioned SQL or Python with a cron schedule, retries, run history, and alerts when something breaks. Save SQL once as a query and reference it from any number of jobs.
Keep the orchestrator you have. If Airflow, Dagster, or Prefect already owns your DAG, leave it there and make LakeSail a task in it. A personal access token gives the orchestrator the API with your permissions: trigger a run, hold it until its upstream is ready, poll its status, cancel it on teardown, and get a signed webhook when it finishes.
Work interactively from your own tools. A session is a live Spark Connect endpoint. Point PySpark, Scala, or any other Spark Connect client at it and work against your tables directly.
Or skip the client: a notebook runs in the console, backed by a session on the same engine and the same data as your jobs.
Size compute per workload. Each job, session, or notebook gets the compute it needs while it runs and gives it back when it stops, so the cluster is never sized for the biggest job. See how compute runs and scales.
Leave your data where it is. Your S3, your catalog, your table formats. LakeSail reads and writes the tables you already have and copies nothing out.
Run it as a team. Members, teams, and roles control who can see and run what, at the organization level or per team, with SSO and MFA on top. Credentials live in managed secrets.
Under the hood
LakeSail hosts the control plane. Compute and data stay in your AWS account, running on Sail, LakeSail's open-source engine.
LakeSail never sees your data. Queries run and results land inside your account.
Concepts has the full model: accounts, networks, clusters, catalogs, and how workloads map onto them.
Start here
- Quickstart: from signup to a running job in your AWS account. Budget 10 to 15 minutes hands-on, plus 15 to 25 minutes while AWS provisions.
- Try Sail locally: run the engine on your laptop first. No AWS needed.