Connect a catalog

A catalog connects LakeSail to table metadata. It does not copy your data. This guide walks through connecting each supported provider.

Default catalog

Each cloud network includes a default AWS Glue catalog. It starts empty and does not import tables from other catalogs. Workloads use it automatically unless you configure their catalogs. Follow this guide to connect a different catalog. Use Settings → Cloud Catalogs to manage the default catalog's infrastructure.

Prerequisites

Depends on the provider:

• AWS Glue: an Active cloud account. For a Glue Data Catalog in another AWS account, that account must grant the LakeSail workload role access. See Connect an AWS account if you don't already have one.
• Iceberg REST, Unity Catalog, OneLake: the catalog's URL and a managed secret containing its auth token.
• Hive Metastore: one or more metastore endpoints and any required Kerberos settings.
• Memory: none.

You don't need a cluster to create a catalog. You'll need one to run anything against the catalog.

Data access

A catalog provides table metadata. It does not copy the underlying data or grant access to it. When a workload runs:

• The cluster network must be able to reach the catalog endpoint and its data locations.
• The workload must have permission to read or write those locations.
• For S3-backed tables on LakeSail-managed compute, the underlying data must be in the network workspace bucket. This applies to jobs, sessions, and notebooks, including saved queries run by jobs.

Catalog credentials grant access to metadata, not S3 data. The default Glue catalog uses the workspace bucket as its default table location. Data stored outside S3 uses that storage provider's credentials. If the catalog or storage endpoint is private, make it reachable from the cluster network before running a workload.

If your S3-backed tables already exist elsewhere, make their data available to LakeSail before running them.

Open the catalog modal

1. Open Catalogs in the sidebar.
2. Click Create catalog.
3. Fill in the shared fields:
   • Catalog Name: e.g. production-catalog.
   • Teams: select at least one team that can use the catalog.
   • Provider: one of the options below. The rest of the form changes based on your choice.

Common settings

Catalogs also have these optional settings:

• Default Database: used for unqualified table names when this is the workload's default catalog.
• Global Temporary Database: used for global temporary views when this is the workload's default catalog.
• Listing Cache: controls the scope, size, and TTL for database, table, and view listings. Not shown for memory catalogs.
• Runtime Environment Variables: passed to workloads that attach the catalog. Values can be literals, managed secret references, or JSON templates containing secret references.

Provider: AWS Glue

Use this when your tables are registered in an AWS Glue Data Catalog the workload can access.

• AWS Catalog ID (optional): the AWS account ID for the Glue Data Catalog. Leave empty to use the connected account.
• AWS Region (optional): e.g. us-east-1. Leave empty to use the workload's AWS region.
• Custom Endpoint URL (optional): only if you route Glue through a VPC endpoint or a non-standard URL.

No credentials are required. LakeSail uses the IAM role from your cloud account.

Provider: Iceberg REST

Use this for any Iceberg REST Catalog, such as Tabular, LakeFS, Polaris, a self-hosted Nessie, or the REST-compatible endpoint of a warehouse.

• Catalog URI: e.g. https://catalog.example.com. The REST endpoint.
• Warehouse (optional): s3://my-bucket/warehouse. Sets the default warehouse location.
• Prefix (optional): path prefix appended to the URI.
• OAuth Access Token or Bearer Access Token: select the managed secret containing whichever token your catalog uses.

Provider: Unity Catalog

For Databricks Unity Catalog.

• Server URI: your Unity metastore URL.
• Default Catalog (optional): e.g. main.
• Personal Access Token: select the managed secret containing the PAT.

Provider: OneLake (Microsoft Fabric)

• OneLake URL: the OneLake item location in <workspace>/<item-name>.<item-type> format, for example OneLake_LakeSail/LakeSail.Lakehouse.
• Bearer Token: select the managed secret containing the Fabric workspace token.

Provider: Hive Metastore

Use this for a Hive Metastore reachable from the cluster network.

• HMS Endpoints: one or more host:port or thrift://host:port endpoints, separated by commas.
• Thrift Transport: buffered or framed.
• Authentication: none or kerberos. Kerberos also requires the service principal and minimum SASL QOP.
• Connect Timeout: how long Sail waits for each endpoint.

Provider: Memory

An in-memory catalog, useful for ephemeral workloads and testing. Supply only:

• Initial Database: e.g. default.
• Database Comment (optional).

After creating

The catalog appears in the Catalogs list. Add it to a compute profile to make it available to jobs, sessions, and notebooks. A job or notebook can customize the catalogs inherited from its profile, and a saved query can bind to a catalog directly.

Tables inside the catalog appear automatically; LakeSail does not require you to register them individually.

API reference

• Catalogs: create, describe, update, delete catalogs and their team assignments.