Data Storage
Sail supports various storage solutions, including file systems and cloud storage services.
You can use the SparkSession.read and SparkSession.write API to load data from and write data to the storage.
You can also use the CREATE TABLE SQL statement to create a table that refers to data stored in the storage.
Here is a summary of the supported (✅) and unsupported (❌) storage features for reading and writing data. There are also features that are planned in our roadmap (🚧).
| Storage | Read Support | Write Support |
|---|---|---|
| File Systems | ✅ | ✅ |
| AWS S3 | ✅ | ✅ |
| HDFS | ✅ | ✅ |
| Hugging Face | ✅ | ❌ |
| Azure Data Lake Storage (ADLS) | 🚧 | 🚧 |
| Azure Blob Storage | 🚧 | 🚧 |
| Google Cloud Storage (GCS) | 🚧 | 🚧 |
| JDBC | ❌ | ❌ |