Scheduling
Jobs run manually by default. A scheduled job can use the interval builder or a cron expression.
For the job definition itself, see Defining Jobs. For runs, see Runs & Debugging.
Prerequisites
- A saved job. See Defining Jobs. Schedule fields are part of the job and are set on the same edit form.
Set a schedule
On the job's create or edit form:
- Under Trigger, select Scheduled.
- Select a Schedule type:
- Interval: choose every N minutes, hours, weeks, or months. Hourly schedules also select the minute; weekly schedules select days and a time; monthly schedules select a day and time.
- Cron: enter a standard 5-field cron expression, such as
0 2 * * *for 2:00 AM every day.
- Select a Timezone. The form previews the next run in local time; its tooltip shows the same time in UTC.
- Choose the concurrency and missed-schedule policies.
- Save the job.
Multi-week intervals
Standard cron cannot represent every N weeks. If you enter more than one week, the schedule runs every week on the selected days.
The cron and timezone fields have length caps; see Limits & Quotas.
Save the job. If the job is Active, the schedule starts firing at the next matching tick.
Active vs. paused
A job has a status independent of its schedule:
- Active: the schedule fires and produces runs.
- Paused: the definition is intact but no scheduled runs fire. Manual runs still work.
Pause is the first option to consider when a job is misbehaving. It is reversible and leaves history intact. Toggle from the job's detail page.
Resuming a paused job
The schedule keeps coming due while a job is paused. None of those runs start, and the job's Scheduled Runs Overdue shows how many there are.
When you set the job back to Active, you choose what happens to them:
- Skip Them (the default): the runs are dropped and never made up. The schedule continues from the next tick.
- Recover Them: the runs start within about a minute, as the missed-schedule policy and the concurrency policy allow. Under
latestthat is a single run, however long the job was paused.
Kept runs follow the concurrency policy like any other:
allow: every one of them runs, up to Max concurrent runs at a time.skip: they start one at a time. The next scheduled run drops those not started by then.replace: they start one at a time, and each cancels any unfinished run before it. The next scheduled run drops those not started by then.
Use allow when none may be lost.
The job form asks whenever you resume a scheduled job. Over the API, send missedScheduleOnResume (skip or recover) with the update that sets status to active. Without it the runs are skipped. Changing the schedule in the same save drops them either way.
Concurrency policy
The concurrency policy decides what happens if a new tick fires while a previous run hasn't finished.
| Policy | Behavior | Use when |
|---|---|---|
skip | Skip the new tick if any run is still non-terminal | The pipeline is idempotent and overlaps waste compute |
allow | Run in parallel, up to Max concurrent runs. A run that is due while that many are still going waits for a free slot | Ticks are independent and each one matters |
replace | Cancel every unfinished run of the job, then start a new one | Only the latest tick matters (e.g. "refresh current state") |
skip is the safest default. Pick allow when every tick matters and overlap is safe. Pick replace when staleness matters more than completion.
With allow, Max concurrent runs caps the overlap. The minimum and default are 1, which runs one at a time with later runs waiting. To let runs overlap without a practical cap, enter a value higher than the job will ever need, for example 999999. Every overlapping run uses its own compute. Every unfinished run of the job counts towards the limit, including ones started by hand.
replace ends every unfinished run of the job, not only scheduled ones: a run started by hand, or one an orchestrator is holding, is cancelled by the next tick too. The cancelled runs end in Cancelled with the reason in their status message. See Cancel a run.
The policy applies to scheduled ticks only. Runs started by hand or through the API are not limited by it.
Missed-schedule policy
The missed-schedule policy decides what happens if the scheduler was unavailable when a tick should have fired (cluster down, platform incident).
latest: fire exactly once on recovery. Use for "keep me current" pipelines where missing ticks is fine as long as the next run catches up.all: fire once per missed tick, oldest first. Use for append-only pipelines where every tick matters (e.g. hourly ingest where each hour is a distinct partition).
latest is the default and fits most workloads. Pick all only when the workload is a function of the scheduled time rather than wall-clock.
Which time a run is for
A scheduled run can start long after the time it was scheduled for: it waited for a free slot, or it is made up after an outage or a pause. Several made-up runs then start at the same moment. Every run therefore carries its logical time, the time it is for, as opposed to when it ran:
- For a scheduled run, the time it was scheduled for. Under
all, each made-up run gets its own missed time. Underlatest, the one run gets the newest missed time. - For a run started by hand or through the API, the time it was created.
The platform always sets it, and you cannot choose it. It reaches you:
- In your job's code, as the environment variable
LAKESAIL_LOGICAL_TIME(RFC 3339, UTC, for example2026-09-19T10:00:00Z). - On the run, as Logical Time in the app for scheduled runs, and
logicalTimein the API and in webhook payloads.
A job that processes a time window should take the window from this value, not from the clock:
import os
from datetime import datetime, timedelta
window_end = datetime.fromisoformat(os.environ["LAKESAIL_LOGICAL_TIME"])
window_start = window_end - timedelta(hours=1)Dropped or overdue
A scheduled run that cannot start when it is due is either dropped or becomes overdue: it still starts, later.
| What happened | What becomes of the run |
|---|---|
The job uses skip or replace, and an earlier run had not finished | skip drops the run for good. replace cancels the earlier run and starts this one |
The job uses allow and was at Max concurrent runs | Overdue. It starts when a slot frees up |
| The scheduler was not running when the run was due | Overdue. It is made up when the scheduler is back |
| The job was paused | Overdue until the job is resumed. Whoever resumes it skips it or recovers it |
Overdue runs are all handled the same way, by the missed-schedule policy: latest turns them into one run, all starts every one of them, oldest first. So under skip, a run that takes longer than the schedule interval loses the ticks that fire meanwhile, even with missed-schedule all. Under allow with all, no tick is ever lost.
Overdue runs on the job
The job's page shows Scheduled Runs Overdue: how many there are, and since when, which is the time the oldest was due. The same numbers are on the job in the API as scheduledRunsOverdue and scheduledRunsOverdueSince. There is no limit on how many runs can be overdue. Under the Latest Only missed-schedule policy it counts runs, not missed ticks: all missed ticks become one run, so it shows 1 run however far behind the job is.
Overdue runs start as slots free up, within about a minute. A few can also be overdue briefly after an outage or a resume. A number that keeps growing means runs take longer than the schedule leaves room for. Raise Max concurrent runs, or use a larger compute profile so each run finishes sooner. To be told when it happens, add a notification rule for Scheduled Runs Queueing, see Notifications.
Trigger a manual run
From the job's detail page, click Run Job. A manual run queues immediately regardless of the schedule and does not change the next scheduled run.
Schedule hygiene
A few practices to avoid common problems:
- Avoid
*/1 * * * *(every minute) for anything non-trivial. Workload startup and scheduling overhead means ticks can pile up. Use sessions for sub-minute work. - Use the timezone the schedule should follow. An IANA timezone such as
America/New_Yorkapplies its daylight-saving rules. - Pick concurrency and missed-schedule deliberately. The defaults (
skip+latest) are safe but they mean missed work is lost. If that is not acceptable for the pipeline, change both.
API reference
- Jobs: schedule fields (
cronExpression,cronTimezone,concurrencyPolicy,maxConcurrentRuns,missedSchedulePolicy) live on the job entity. There is no separate schedule resource.