LakeSail PlatformPREVIEW

Appearance

The managed Apache Flink service on the LakeSail platform is deprecated.

LakeSail is building Sail, an open-source computation framework in Rust to seamlessly integrate stream-processing, batch-processing, and compute-intensive (AI) workloads. The LakeSail platform will offer the managed solution for Sail. Existing PySpark and Flink SQL workloads can be migrated with ease. Please stay tuned and contact us if you are interested!

Concepts ​

When you use LakeSail, you own an organization and register Kubernetes clusters in the organization. You create workspaces in the organization where each of the workspaces maps to a namespace [1] in a Kubernetes cluster. You create users and service accounts in the organization and grant them access to workspaces. You create Flink applications and Flink sessions in the workspaces.

Organization Resources ​

Organization
An organization is an isolated tenant in LakeSail. An organization can be a company or a team signed up to use LakeSail.
Workspace
A workspace is a logical unit of isolated resources within an organization. A workspace maps to one namespace [1:1] in a Kubernetes cluster.
Cluster
The organization administrator registers Kubernetes clusters [2] to LakeSail, by specifying the network addresses and credentials of the clusters. The Kubernetes cluster can be a one that you manage yourself, either on-premise or operated by a cloud provider of your choice, or a one provisioned and operated by LakeSail.
User
A user is an identity assumed by a human to interact with the LakeSail API and web console.
Service Account
A service account is an identity assumed by applications and services to interact with the LakeSail API.
Flink Application
In LakeSail, a Flink application is a Flink job along with its dependencies (code artifacts, compute and storage resource configuration, etc.). A Flink application can have only one Flink job, which determines the lifecycle of the underlying Flink cluster. The Flink cluster is deployed to Kubernetes in Application Mode [3] [4].
Flink Session
In LakeSail, a Flink session is a group of Flink jobs that have shared dependencies (code artifacts, compute and storage resource configuration, etc.). A Flink session can have multiple Flink jobs, which share the same Flink cluster. The job lifecycle is independent of the Flink cluster lifecycle. The Flink cluster is deployed to Kubernetes in Session Mode [4:1].

  1. Namespace is a Kubernetes concept. You can refer to the Kubernetes documentation for more details. ↩︎ ↩︎

  2. A Kubernetes cluster is not to be confused with a Flink cluster, which is running inside a namespace in a Kubernetes cluster. ↩︎

  3. Although Flink Application Mode allows concurrent execution of multiple Flink jobs, LakeSail only allows one job per Flink application. This is inline with the restriction in Flink Kubernetes Operator. ↩︎

  4. You can refer to the Flink documentation for more details about different Flink deployment modes. ↩︎ ↩︎