External data targets and their grants to compute profiles.
GET
/infra/data-targetsList external data targets
Returns external data targets across the organization's cloud accounts.
Parameters
limitinteger
limit
The maximum number of items to list.
Type
integer
Format
"int32"Minimum
0Maximum
100Default
100markerstring
marker
The pagination token returned from the previous list operation.
Type
string
Responses
Responses
A paginated list of external data targets.
application/json
JSON "items": [ { "externalDataTarget": { "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ "read" ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" }, "grantedWorkloadConfigCount": 0, "capabilities": { "delete": { "allowed": true, "blocked": { "label": "string", "message": "string" } } } } ], "included": { "externalConnections": [ { "id": "A1B2C3D4E5", "name": "Production Cluster", "provider": "string", "providerConfig": { "accountId": "123456789012", "region": "us-east-1", "roleArn": "arn:aws:iam::123456789012:role/LakeSailConnection", "externalIdSecretArn": "arn:aws:secretsmanager:us-east-1:123456789012:secret:lakesail-ext-id", "stackArn": "arn:aws:cloudformation:us-east-1:123456789012:stack/lakesail-connection/abc123" }, "status": "active", "statusMessage": "string", "lastUsedAt": "string", "templateSyncedAt": "string", "templateSyncMessage": "string", "templateUpdateAvailable": true, "subscriptionId": "string", "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" } ] }, "marker": "string"
{
}
GET
/infra/connections/{connection}/data-targetsList external data targets by connection
Returns the external data targets registered beneath a cloud account.
Parameters
limitinteger
limit
The maximum number of items to list.
Type
integer
Format
"int32"Minimum
0Maximum
100Default
100markerstring
marker
The pagination token returned from the previous list operation.
Type
string
Responses
Responses
A paginated list of external data targets.
application/json
JSON "items": [ { "externalDataTarget": { "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ "read" ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" }, "grantedWorkloadConfigCount": 0, "capabilities": { "delete": { "allowed": true, "blocked": { "label": "string", "message": "string" } } } } ], "included": { "externalConnections": [ { "id": "A1B2C3D4E5", "name": "Production Cluster", "provider": "string", "providerConfig": { "accountId": "123456789012", "region": "us-east-1", "roleArn": "arn:aws:iam::123456789012:role/LakeSailConnection", "externalIdSecretArn": "arn:aws:secretsmanager:us-east-1:123456789012:secret:lakesail-ext-id", "stackArn": "arn:aws:cloudformation:us-east-1:123456789012:stack/lakesail-connection/abc123" }, "status": "active", "statusMessage": "string", "lastUsedAt": "string", "templateSyncedAt": "string", "templateSyncMessage": "string", "templateUpdateAvailable": true, "subscriptionId": "string", "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" } ] }, "marker": "string"
{
}
POST
/infra/connections/{connection}/data-targetsRegister an external data target
Registers an external data target beneath a cloud account without changing the external resource.
Request Body
application/json
JSON "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ "read" ]
{
}
Responses
Responses
External data target registered.
application/json
JSON "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5", "capabilities": { "delete": { "allowed": true, "blocked": { "label": "string", "message": "string" } } }, "providerDetails": { "type": "aws", "connectionPolicyStatement": [ { "additionalProperties": "string" } ] }
{
}
GET
/infra/connections/{connection}/data-targets/{target}Describe an external data target
Returns an external data target and its optional connection-wide customer policy statements.
Responses
Responses
External data target details.
application/json
JSON "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5", "capabilities": { "delete": { "allowed": true, "blocked": { "label": "string", "message": "string" } } }, "providerDetails": { "type": "aws", "connectionPolicyStatement": [ { "additionalProperties": "string" } ] }
{
}
DELETE
/infra/connections/{connection}/data-targets/{target}Remove an external data target
Soft-deletes an external data target after verifying that no compute profiles still use it. The external resource and its bucket policy are not changed.
Responses
Responses
The requested operation was done successfully.
GET
/infra/connections/{connection}/data-targets/{target}/workload-configsList compute profiles granted access to the external data target
Returns the compute profiles with non-deleted grants to the external data target.
Parameters
limitinteger
limit
The maximum number of items to list.
Type
integer
Format
"int32"Minimum
0Maximum
100Default
100markerstring
marker
The pagination token returned from the previous list operation.
Type
string
Responses
Responses
A paginated list of compute profiles granted access to the external data target.
application/json
JSON "items": [ { "id": "A1B2C3D4E5", "name": "Production Cluster", "workloadType": "string", "clusterId": "A1B2C3D4E5", "config": { "sail": { "executionMode": "string", "maxRetries": 3, "compute": { "driver": { "instanceType": "c8g.2xlarge", "ebsConfig": { "volumeType": "gp3", "volumeSize": 100, "iops": 3000, "throughput": 125, "deleteOnTermination": true } }, "worker": { "instanceType": "c8g.4xlarge", "ebsConfig": { "volumeType": "gp3", "volumeSize": 100, "iops": 3000, "throughput": 125, "deleteOnTermination": true }, "maxNodes": 4, "capacityType": "on-demand" } }, "lakesailImageUri": "string", "libraries": [ { "pypi": { "package": "string", "repo": "string" }, "requirements": { "source": "string", "uri": "string" }, "whl": "string" } ], "extraEnv": { "additionalProperties": { "type": "string", "value": "string" } } } }, "endpointLimits": { "spark_connect": 0 }, "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5", "catalogs": [ ], "defaultCatalogId": "A1B2C3D4E5" } ], "marker": "string"
{
}
GET
/workloads/configs/{workloadConfig}/data-targetsList grants
Returns the grants for a compute profile.
Parameters
limitinteger
limit
The maximum number of items to list.
Type
integer
Format
"int32"Minimum
0Maximum
100Default
100markerstring
marker
The pagination token returned from the previous list operation.
Type
string
Responses
Responses
Grants for the compute profile.
application/json
JSON "included": { "externalDataTargets": [ { "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ "read" ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" } ], "operations": [ { "id": "A1B2C3D4E5", "organizationId": "A1B2C3D4E5", "memberId": "A1B2C3D4E5", "operation": "string", "status": "string", "result": { "items": [ { "provider": "string", "instanceType": "string", "vCpus": 0, "memoryMiB": 0, "architecture": "string", "gpuCount": 0, "instanceFamily": "string", "maxPods": 0, "group": "string", "allocatableMilliCpu": 0, "pricePerHourUsd": 0 } ], "metadata": { "provider": "string", "systemPodsPerNode": 0, "systemPodsPerSystemNode": 0, "systemPodsFixed": 0, "systemPodsPerNodeMargin": 0, "systemPodsFixedMargin": 0, "systemMilliCPUPerNode": 0, "systemMilliCPUFixed": 0, "karpenterMilliCPUPerReplica": 0, "karpenterReplicas": 0 } }, "error": { "code": "string", "message": "string", "errorType": "string", "details": { "retryable": true, "retryAfter": 0, "fieldErrors": { "additionalProperties": "string" }, "requestId": "string", "causedBy": "string" } }, "progress": { "stage": "string", "message": "string", "resources": [ { "urn": "string", "type": "string", "name": "string", "operation": "string", "status": "string", "duration": 0 } ], "summary": { "total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "unchanged": 0 } }, "resourceType": "string", "resourceId": "A1B2C3D4E5", "createdAt": "string", "updatedAt": "string", "startedAt": "string", "completedAt": "string" } ] }, "items": [ { "id": "A1B2C3D4E5", "externalDataTargetId": "A1B2C3D4E5", "access": [ "read" ], "status": "active", "statusMessage": "string", "capabilities": { "retry": { "allowed": true, "blocked": { "label": "string", "message": "string" } }, "test": { "allowed": true, "blocked": { "label": "string", "message": "string" } }, "remove": { "allowed": true, "blocked": { "label": "string", "message": "string" } } }, "latestTestOperationId": "A1B2C3D4E5" } ], "marker": "string", "capabilities": { "grant": { "allowed": true, "blocked": { "label": "string", "message": "string" } } }
{
}
GET
/workloads/configs/{workloadConfig}/data-targets/{target}Describe a grant
Returns one grant and its configuration.
Responses
Responses
Grant configuration details.
application/json
JSON "item": { "id": "A1B2C3D4E5", "externalDataTargetId": "A1B2C3D4E5", "access": [ "read" ], "status": "active", "statusMessage": "string", "capabilities": { "retry": { "allowed": true, "blocked": { "label": "string", "message": "string" } }, "test": { "allowed": true, "blocked": { "label": "string", "message": "string" } }, "remove": { "allowed": true, "blocked": { "label": "string", "message": "string" } } }, "latestTestOperationId": "A1B2C3D4E5", "providerDetails": { "type": "aws", "profileRoleArn": "arn:aws:iam::123456789012:role/lakesail-conn1234-profile5678-wc", "profileRegion": "us-east-1", "customerPolicyStatement": [ { "additionalProperties": "string" } ] } }, "included": { "externalDataTargets": [ { "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ "read" ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" } ], "operations": [ { "id": "A1B2C3D4E5", "organizationId": "A1B2C3D4E5", "memberId": "A1B2C3D4E5", "operation": "string", "status": "string", "result": { "items": [ { "provider": "string", "instanceType": "string", "vCpus": 0, "memoryMiB": 0, "architecture": "string", "gpuCount": 0, "instanceFamily": "string", "maxPods": 0, "group": "string", "allocatableMilliCpu": 0, "pricePerHourUsd": 0 } ], "metadata": { "provider": "string", "systemPodsPerNode": 0, "systemPodsPerSystemNode": 0, "systemPodsFixed": 0, "systemPodsPerNodeMargin": 0, "systemPodsFixedMargin": 0, "systemMilliCPUPerNode": 0, "systemMilliCPUFixed": 0, "karpenterMilliCPUPerReplica": 0, "karpenterReplicas": 0 } }, "error": { "code": "string", "message": "string", "errorType": "string", "details": { "retryable": true, "retryAfter": 0, "fieldErrors": { "additionalProperties": "string" }, "requestId": "string", "causedBy": "string" } }, "progress": { "stage": "string", "message": "string", "resources": [ { "urn": "string", "type": "string", "name": "string", "operation": "string", "status": "string", "duration": 0 } ], "summary": { "total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "unchanged": 0 } }, "resourceType": "string", "resourceId": "A1B2C3D4E5", "createdAt": "string", "updatedAt": "string", "startedAt": "string", "completedAt": "string" } ] }
{
}
PUT
/workloads/configs/{workloadConfig}/data-targets/{target}Grant a compute profile access to an external data target
Grants a compute profile access to a registered external data target and starts its IAM configuration. Target registration authorizes profiles on the same cloud account to use the target. The compute profile can use only the target's approved access levels. Once the grant is active, workloads on this profile use its profile IAM role. Customer-managed grants to the cluster's Job Runner Role do not carry over to that role. Returns 409 while workloads using this compute profile are running.
Request Body
application/json
JSON "access": [ "read" ]
{
}
Responses
Responses
Grant configuration details.
application/json
JSON "item": { "id": "A1B2C3D4E5", "externalDataTargetId": "A1B2C3D4E5", "access": [ "read" ], "status": "active", "statusMessage": "string", "capabilities": { "retry": { "allowed": true, "blocked": { "label": "string", "message": "string" } }, "test": { "allowed": true, "blocked": { "label": "string", "message": "string" } }, "remove": { "allowed": true, "blocked": { "label": "string", "message": "string" } } }, "latestTestOperationId": "A1B2C3D4E5", "providerDetails": { "type": "aws", "profileRoleArn": "arn:aws:iam::123456789012:role/lakesail-conn1234-profile5678-wc", "profileRegion": "us-east-1", "customerPolicyStatement": [ { "additionalProperties": "string" } ] } }, "included": { "externalDataTargets": [ { "id": "A1B2C3D4E5", "connectionId": "A1B2C3D4E5", "name": "Production Cluster", "provider": "AWS", "service": "S3", "targetConfig": { "type": "aws_s3", "accountId": "123456789012", "region": "us-east-1", "bucketName": "customer-data", "objectPrefix": "warehouse/production/" }, "access": [ "read" ], "createdAt": "string", "createdBy": "A1B2C3D4E5", "updatedAt": "string", "updatedBy": "A1B2C3D4E5" } ], "operations": [ { "id": "A1B2C3D4E5", "organizationId": "A1B2C3D4E5", "memberId": "A1B2C3D4E5", "operation": "string", "status": "string", "result": { "items": [ { "provider": "string", "instanceType": "string", "vCpus": 0, "memoryMiB": 0, "architecture": "string", "gpuCount": 0, "instanceFamily": "string", "maxPods": 0, "group": "string", "allocatableMilliCpu": 0, "pricePerHourUsd": 0 } ], "metadata": { "provider": "string", "systemPodsPerNode": 0, "systemPodsPerSystemNode": 0, "systemPodsFixed": 0, "systemPodsPerNodeMargin": 0, "systemPodsFixedMargin": 0, "systemMilliCPUPerNode": 0, "systemMilliCPUFixed": 0, "karpenterMilliCPUPerReplica": 0, "karpenterReplicas": 0 } }, "error": { "code": "string", "message": "string", "errorType": "string", "details": { "retryable": true, "retryAfter": 0, "fieldErrors": { "additionalProperties": "string" }, "requestId": "string", "causedBy": "string" } }, "progress": { "stage": "string", "message": "string", "resources": [ { "urn": "string", "type": "string", "name": "string", "operation": "string", "status": "string", "duration": 0 } ], "summary": { "total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "unchanged": 0 } }, "resourceType": "string", "resourceId": "A1B2C3D4E5", "createdAt": "string", "updatedAt": "string", "startedAt": "string", "completedAt": "string" } ] }
{
}
DELETE
/workloads/configs/{workloadConfig}/data-targets/{target}Remove a grant from a compute profile
Applies the remaining grants before deleting the grant. Target registration authorizes profiles on the same cloud account to use the target. Edit access to the compute profile controls this action. Returns 409 while workloads using this compute profile are running.
Responses
Responses
Grant removal operation initiated.
application/json
JSON "id": "A1B2C3D4E5", "organizationId": "A1B2C3D4E5", "memberId": "A1B2C3D4E5", "operation": "string", "status": "string", "result": { "items": [ { "provider": "string", "instanceType": "string", "vCpus": 0, "memoryMiB": 0, "architecture": "string", "gpuCount": 0, "instanceFamily": "string", "maxPods": 0, "group": "string", "allocatableMilliCpu": 0, "pricePerHourUsd": 0 } ], "metadata": { "provider": "string", "systemPodsPerNode": 0, "systemPodsPerSystemNode": 0, "systemPodsFixed": 0, "systemPodsPerNodeMargin": 0, "systemPodsFixedMargin": 0, "systemMilliCPUPerNode": 0, "systemMilliCPUFixed": 0, "karpenterMilliCPUPerReplica": 0, "karpenterReplicas": 0 } }, "error": { "code": "string", "message": "string", "errorType": "string", "details": { "retryable": true, "retryAfter": 0, "fieldErrors": { "additionalProperties": "string" }, "requestId": "string", "causedBy": "string" } }, "progress": { "stage": "string", "message": "string", "resources": [ { "urn": "string", "type": "string", "name": "string", "operation": "string", "status": "string", "duration": 0 } ], "summary": { "total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "unchanged": 0 } }, "resourceType": "string", "resourceId": "A1B2C3D4E5", "createdAt": "string", "updatedAt": "string", "startedAt": "string", "completedAt": "string"
{
}
POST
/workloads/configs/{workloadConfig}/data-targets/{target}/applyRetry grant configuration
Retries a failed IAM configuration using the access already stored on the grant. Target registration authorizes profiles on the same cloud account to use the target. Edit access to the compute profile controls this action.
Responses
Responses
External data target configuration operation initiated.
application/json
JSON "id": "A1B2C3D4E5", "organizationId": "A1B2C3D4E5", "memberId": "A1B2C3D4E5", "operation": "string", "status": "string", "result": { "items": [ { "provider": "string", "instanceType": "string", "vCpus": 0, "memoryMiB": 0, "architecture": "string", "gpuCount": 0, "instanceFamily": "string", "maxPods": 0, "group": "string", "allocatableMilliCpu": 0, "pricePerHourUsd": 0 } ], "metadata": { "provider": "string", "systemPodsPerNode": 0, "systemPodsPerSystemNode": 0, "systemPodsFixed": 0, "systemPodsPerNodeMargin": 0, "systemPodsFixedMargin": 0, "systemMilliCPUPerNode": 0, "systemMilliCPUFixed": 0, "karpenterMilliCPUPerReplica": 0, "karpenterReplicas": 0 } }, "error": { "code": "string", "message": "string", "errorType": "string", "details": { "retryable": true, "retryAfter": 0, "fieldErrors": { "additionalProperties": "string" }, "requestId": "string", "causedBy": "string" } }, "progress": { "stage": "string", "message": "string", "resources": [ { "urn": "string", "type": "string", "name": "string", "operation": "string", "status": "string", "duration": 0 } ], "summary": { "total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "unchanged": 0 } }, "resourceType": "string", "resourceId": "A1B2C3D4E5", "createdAt": "string", "updatedAt": "string", "startedAt": "string", "completedAt": "string"
{
}
POST
/workloads/configs/{workloadConfig}/data-targets/{target}/testTest access through a grant
Starts a workload-equivalent access test for an active target grant. Target registration authorizes profiles on the same cloud account to use the target. Profile editors and callers permitted to run workloads through the profile can start this test.
Responses
Responses
Access test operation initiated.
application/json
JSON "id": "A1B2C3D4E5", "organizationId": "A1B2C3D4E5", "memberId": "A1B2C3D4E5", "operation": "string", "status": "string", "result": { "items": [ { "provider": "string", "instanceType": "string", "vCpus": 0, "memoryMiB": 0, "architecture": "string", "gpuCount": 0, "instanceFamily": "string", "maxPods": 0, "group": "string", "allocatableMilliCpu": 0, "pricePerHourUsd": 0 } ], "metadata": { "provider": "string", "systemPodsPerNode": 0, "systemPodsPerSystemNode": 0, "systemPodsFixed": 0, "systemPodsPerNodeMargin": 0, "systemPodsFixedMargin": 0, "systemMilliCPUPerNode": 0, "systemMilliCPUFixed": 0, "karpenterMilliCPUPerReplica": 0, "karpenterReplicas": 0 } }, "error": { "code": "string", "message": "string", "errorType": "string", "details": { "retryable": true, "retryAfter": 0, "fieldErrors": { "additionalProperties": "string" }, "requestId": "string", "causedBy": "string" } }, "progress": { "stage": "string", "message": "string", "resources": [ { "urn": "string", "type": "string", "name": "string", "operation": "string", "status": "string", "duration": 0 } ], "summary": { "total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "unchanged": 0 } }, "resourceType": "string", "resourceId": "A1B2C3D4E5", "createdAt": "string", "updatedAt": "string", "startedAt": "string", "completedAt": "string"
{
}