Notebooks
A notebook is a hosted, Marimo-based interactive Python notebook backed by a Sail session. Where a session is a runtime you connect your own client to, a notebook provides both the editor and the runtime.
When to use a notebook
- Exploratory Python + Spark in one place. You want to write PySpark and see results without wiring up a local client or managing tokens.
- Team-managed exploration. Team Admins and Editors can manage the notebook runtime while the creator retains access to the editor.
- Authoring before you productionize. Prototype a transform in a notebook, then lift the working logic into a job once it's stable.
When not to use one:
- Scheduled or batch work. Use a job. Notebooks are interactive and don't carry schedules, versioning, or run history.
- Connecting an external client (local PySpark or a service). Use a session; it exposes Spark Connect over gRPC for exactly that.
Prerequisites
- A compute profile (workload config) on a cluster whose Readiness is Ready. The profile defines the Sail pod the notebook runs on: instance type, libraries, and environment. Its environment variables also reach the notebook kernel; see Environment variables and secrets.
- A catalog if you want to query your tables (recommended). A notebook inherits the catalogs attached to its compute profile, and can override that set with its own. For S3-backed tables, the underlying data must be in the compute profile's network workspace bucket.
Notebook timing
A notebook uses a backing session with these clocks:
| Setting | Value | What happens |
|---|---|---|
| Idle timeout | 30 minutes by default; 2 minutes to 8 hours | An inactive notebook's backing session moves to Idle. Activity is the notebook being open in a browser, a request through the session proxy, or a running cell. |
| Idle close delay | 5 minutes by default | The idle session closes and the notebook stops. Opening the notebook during the delay keeps it. Notebook contents persist. |
| Maximum duration | 7 days per start | The notebook stops regardless of activity. Starting it again creates a new backing session and resets this clock. Organization settings can change this value. A platform node upgrade can end a notebook that has run longer than 48 hours. |
Create a notebook
- Open Notebooks in the sidebar and click Create Notebook.
- Fill in:
- Notebook name: e.g.
daily-sales-analysis. - Team: who can access the notebook according to their team roles. Only the creating member can open the notebook editor.
- Compute profile: the profile whose Sail pod backs the notebook. Pick an existing one or create a new one inline.
- Idle timeout (minutes): how long the notebook may remain inactive before its backing session becomes idle. See Notebook timing, or accept the default.
- Notebook name: e.g.
- Click Create notebook. The notebook starts in Stopped.
Start, open, and stop
A notebook has its own lifecycle, independent of the workload it runs:
| Status | Meaning |
|---|---|
| Stopped | No Sail or Marimo pods are running |
| Starting | The platform is provisioning the Sail and Marimo pods |
| Running | Fully active and ready to use |
| Stopping | Tearing the pods down |
| Error | Provisioning or teardown failed; check the status message |
- Start provisions the pods and moves the notebook to Running.
- Open launches the Marimo editor against the running notebook.
- Stop tears the pods down to reclaim compute. Your notebook contents persist; only the runtime goes away.
Notebook contents are saved to the network's workspace bucket under notebooks/, in your AWS account. That is what survives a stop, and what remains if the notebook is deleted.
The platform also stops a notebook on its own. The idle timeout, set at creation and editable afterwards, controls when the backing session becomes idle. When it elapses, the session enters its idle close delay and then stops the notebook.
A cell that is still running counts as activity, whether or not the notebook is open in a browser. A long-running cell keeps the notebook alive until it finishes, and the idle timeout then runs from that point, so you have the full timeout to come back for the result.
Stopping early still pays
A Running notebook holds compute until the idle timeout fires, so stopping it yourself when you step away reclaims compute sooner. Starting it again is quick on a warm cluster.
Logs
Logs on the notebook page opens two live streams: notebook, the Marimo pod that runs the editor and your cells, and engine, the Sail driver of the backing session, where engine-side failures such as out-of-memory kills appear. They are available from the moment the notebook starts, and no longer available once its resources are cleaned up after a stop.
Environment variables and secrets
The environment variables on the notebook's compute profile are available inside cells through os.environ, alongside the Sail engine. Use a managed secret for any credential your code needs, such as a database password or an API token, and reference it from the profile.
Never paste a credential into a cell. Notebook contents are saved as plain text, so a pasted value is stored with the notebook and shared with everyone who can read it. The same applies to printed values: a secret written to cell output is saved with the notebook.
Values are resolved when the notebook starts. After changing a variable or replacing a secret, stop and start the notebook to pick up the new value.
A few variable names are set by the notebook runtime itself, such as the Spark Connect address and the Python import path. A profile variable with one of those names is not delivered; the notebook refuses to start and its status message names the variable. Rename it in the compute profile.
Change the compute profile
The compute profile is editable only while the notebook is stopped. To move a notebook to bigger compute (or a different cluster), stop it, change the profile, and start it again.
Team access
The notebook's team controls access according to each member's team role:
- Team Admin and Editor can edit, start, and stop the notebook.
- Viewer can view its details.
- Only the creating member can open the notebook editor.
Organization roles can also grant management access, but they do not override the creator check for opening the editor.
API reference
- Notebooks:
CreateNotebook,StartNotebook,StopNotebook,OpenNotebook,UpdateNotebook,DeleteNotebook, plus team sharing. - API Reference: the workload config (compute profile) a notebook references.