Getting Started

In this guide, you will learn how to launch a LakeSail server locally in a Docker container. You will connect the LakeSail server to a local Kubernetes cluster. You will learn how to access the LakeSail web console and the REST API.

Prerequisites

This guide requires that you have the following software in your environment. Please refer to the documentation of each software for installation instructions.

1. Docker
2. minikube
3. Helm

Preparing a Local Kubernetes Cluster

In the following steps, we will use minikube to create a local Kubernetes cluster and install the required dependencies.

1. Run the following command to start the minikube cluster.

   minikube start --apiserver-names=minikube

   INFO

   Please make sure to include the --apiserver-names option. Otherwise, there will be a TLS certificate error when the LakeSail server tries to connect to the Kubernetes API server via the hostname of the minikube container.

2. Run the following command to install cert-manager in the minikube cluster.

   kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.13.3/cert-manager.yaml

3. Run the following command to install the Flink Kubernetes Operator in the minikube cluster.

   helm repo add flink-operator https://downloads.apache.org/flink/flink-kubernetes-operator-1.7.0/
   helm install flink-kubernetes-operator flink-operator/flink-kubernetes-operator

4. Run kubectl get pods and you should see the Flink Kubernetes Operator pods running.

INFO

The steps above are adapted from the Flink Kubernetes Operator documentation. You can visit the documentation for more details.

Launching the LakeSail Server

Run the following command to launch the LakeSail server locally. The command starts a Docker container named lakesail-server in the background.

docker run \
  --name lakesail-server -d \
  -p "18080:8080" \
  --network minikube \
  -e "LAKESAIL_SERVER_LISTENER_PORT=8080" \
  -e "LAKESAIL_SERVER_EXTERNAL_PORT=18080" \
  -e "LAKESAIL_SERVER_EXTERNAL_WEB_PORT=18080" \
  -e "LAKESAIL_LOCAL_KUBE_CONFIG_PATH=${HOME}/.kube/config" \
  -e "LAKESAIL_LOCAL_KUBE_CONFIG_OVERRIDE_SERVER=https://minikube:8443" \
  -v "${HOME}/.kube/config:${HOME}/.kube/config:ro" \
  -v "${HOME}/.minikube:${HOME}/.minikube:ro" \
  ghcr.io/lakehq/lakesail:0.2.0 start

INFO

You can run docker run --rm ghcr.io/lakehq/lakesail:0.2.0 to view the help message showing all the available subcommands for the LakeSail server executable.

INFO

The Docker container maps the host port 18080 to the container port 8080 which the LakeSail server listens on.

You can change the host port by modifying the -p option along with the environment variables LAKESAIL_SERVER_EXTERNAL_PORT and LAKESAIL_SERVER_EXTERNAL_WEB_PORT. In this guide, the two external ports (one for the REST API and one for the web console) share the same value.

INFO

The Docker container mounts the Kubernetes configuration file and minikube credentials from the host to the container as read-only volumes.

If your minikube configuration is at a different location, modify the paths in the -v options accordingly.

INFO

The Docker container should be launched in the same Docker network as the minikube cluster.

If your minikube cluster is in a Docker network other than minikube, modify the --network option accordingly.

If your miniube cluster is accessible from a different address inside the Docker network, you can modify the LAKESAIL_LOCAL_KUBE_CONFIG_OVERRIDE_SERVER environment variable accordingly.

Signing in to the LakeSail Web Console

Visit http://localhost:18080/ in your browser. Click Sign In on the homepage which will direct you to the login page. Enter the following information and click Sign In.

• Organization: local
• Username: user
• Password: password

You should see the homepage of the LakeSail web console. You should find a workspace with name "Default Workspace" and ID DEFAULT in the My Workspaces list. This built-in workspace is linked to your minikube cluster.

Creating an API Key

You can use the LakeSail REST API to manage organization resources and Flink resources. To access the API, you need to create an API key associated with a service account. The following steps create an API key using the web console.

1. Click the 👤 button on the right of the navigation bar and choose My Organization from the drop-down menu.
2. On the left side of the page, click Service Accounts to list all the service accounts. You will find a built-in service account named service-account. Click Details to view the details of the service account.
3. Click Create Key. In the resulting page, either leave the Never Expires checkbox checked or enter an expiration date. Click Submit. You will see a new API key created for the service account. Make a note of the API key since it will be displayed only once.

INFO

The local LakeSail server you launched does not have persistent storage. If you stop and restart the server, you will need to create a new API key.

Testing the LakeSail REST API

In your terminal, export the API key as an environment variable.

export LAKESAIL_API_KEY="..."

INFO

Please make sure to include all the characters in the API key, including leading and trailing - or _ characters if present.

Then run the following command to invoke the LakeSail REST API.

curl -i -X GET \
  "http://localhost:18080/api/flink/v1/workspaces/DEFAULT/applications" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer ${LAKESAIL_API_KEY}"

The endpoint in the request above returns a list of Flink applications in the built-in workspace with ID DEFAULT. Since you have not submitted any Flink applications yet, you should see a 200 OK status with the following JSON response.

{"items":[]}

You can find more information about the LakeSail REST API in the LakeSail API documentation as well as the API Reference.

Cleaning Up

Run the following commands to stop and delete the LakeSail server container.

docker stop lakesail-server
docker rm lakesail-server

You can stop and delete the minikube cluster by running the following commands.

minikube stop
minikube delete

Next Steps

You have now successfully launched a LakeSail server locally. Please continue to the Tutorials to learn how to use LakeSail to run Flink workloads.

Also, feel free to explore the rest of the documentation to learn more about LakeSail.