Iceberg REST Catalog
The Iceberg REST catalog provider in Sail allows you to connect to an external catalog that exposes the Iceberg REST Catalog API.
Options
An Iceberg REST catalog can be configured using the following options.
type(required): The catalog provider. Set this option toiceberg-rest.name(required): The catalog name.uri(required): The base URI of the Iceberg REST catalog server.warehouse(optional): The catalog warehouse location.prefix(optional): The URL-encoded prefix for catalog API endpoints.namespace_separator(optional): The client-side fallback for the Iceberg RESTnamespace-separatorcatalog property. Use the separator that the REST server expects, such as::or/. Sail also accepts URL-encoded values returned by REST servers, such as%3A%3Afor::. If this option is not set or is empty, Sail uses the Iceberg REST default unit separator.access_delegation(optional): The value of theX-Iceberg-Access-Delegationheader sent when loading a table. The default isvended-credentials. Set this option tononeor leave it empty to omit the header, which is needed for servers that reject the header.oauth_access_token(optional): The OAuth 2.0 access token.bearer_access_token(optional): The bearer token for authentication. This option takes precedence overoauth_access_token.bearer_access_token_file(optional): The path to a file that holds the bearer token. This option takes precedence overbearer_access_tokenandoauth_access_token. Sail reads the token from this file for every request, so it picks up a rotated token, such as a kubelet-projected service account token, without restarting the server. If a request is rejected with401 Unauthorized, Sail reloads the file and retries the request once. The file must contain the raw token without theBearerprefix for the HTTPAuthorizationheader. Surrounding whitespace in the file is trimmed. Empty or unreadable files produce an error rather than falling back to other options.username(optional): The user name for HTTP Basic authentication, which some REST catalog servers, such as Apache Gravitino, accept instead of a token. Must be set together withpassword.password(optional): The password for HTTP Basic authentication. Sail sendsAuthorization: Basic <base64(username:password)>. Basic authentication cannot be combined withoauth_access_token,bearer_access_token, orbearer_access_token_file.sigv4_enabled(optional): Whether to sign every request with AWS Signature Version 4 (SigV4). Use this option for Iceberg REST endpoints that verify SigV4, such as Amazon S3 Tables and RustFS. The default isfalse.signing_region(optional): The AWS region used for signing, such asus-east-1. If this option is not set, Sail resolves the region from the AWS default region provider chain, such as theAWS_REGIONenvironment variable or the AWS profile.signing_name(optional): The service name used for signing. The default isexecute-api. Uses3tablesfor Amazon S3 Tables ands3for RustFS.
See Common Options for caching configuration.
Server Configuration
Sail calls GET /v1/config before catalog operations. The final Iceberg REST catalog configuration is merged in this order:
- Server
defaults. - Sail client configuration.
- Server
overrides.
Server overrides take precedence over matching values configured in Sail. For example, if the REST server returns namespace-separator in overrides, Sail uses that value to encode multipart namespaces even when namespace_separator is configured locally. Configuring namespace_separator in Sail does not configure the REST server. The server must already decode the same separator, usually by advertising it from /v1/config.
Examples
This example configures an Iceberg REST catalog without authentication.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com"}]'This example uses an OAuth access token.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com", warehouse="s3://data/warehouse", oauth_access_token="..."}]'This example uses a bearer token.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com", warehouse="s3://data/warehouse", bearer_access_token="..."}]'This example reads a bearer token from a file, such as a kubelet-projected service account token.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com", warehouse="s3://data/warehouse", bearer_access_token_file="/var/run/secrets/tokens/catalog-token"}]'This example uses HTTP Basic authentication.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com", warehouse="s3://data/warehouse", username="...", password="..."}]'This example connects to the Amazon S3 Tables Iceberg REST endpoint with SigV4 authentication. The warehouse is the ARN of the table bucket. The AWS credentials are resolved from the default credential provider chain.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="s3tables", uri="https://s3tables.us-east-1.amazonaws.com/iceberg", warehouse="arn:aws:s3tables:us-east-1:111122223333:bucket/my-table-bucket", sigv4_enabled=true, signing_region="us-east-1", signing_name="s3tables"}]'This example connects to the RustFS Iceberg REST endpoint with SigV4 authentication and static AWS credentials.
export AWS_REGION="us-east-1"
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="rustfs", uri="http://localhost:9000/iceberg", warehouse="warehouse", access_delegation="none", sigv4_enabled=true, signing_name="s3"}]'This example configures a client-side namespace separator fallback.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com", namespace_separator="::"}]'This example omits the access delegation header.
export SAIL_CATALOG__LIST='[{type="iceberg-rest", name="sail", uri="https://catalog.example.com", access_delegation="none"}]'Table DDL
See the Iceberg DDL support matrix for supported operations.
- Registering an existing table requires the server's register-table endpoint and access to the table's metadata file.
- Format upgrades require server support for the target version. Column defaults require Iceberg format version 3 and typed literals.
DROP TABLE ... PURGErequests deletion of the table's data. Support and deletion behavior depend on the catalog server.
