Skip to content

Scheduling

Jobs run manually by default. Adding a schedule turns a job into a recurring pipeline that fires on any cron expression.

For the job definition itself, see Defining jobs. For runs, see Runs & debugging.

Prerequisites

  • A saved job. See Defining jobs. Schedule fields are part of the job and are set on the same edit form.

Set a schedule

On the job's edit form:

  • Cron expression: a standard 5-field cron. Leave empty for manual-only. Examples:
    • 0 2 * * *: every day at 2:00 AM
    • */15 * * * *: every 15 minutes
    • 0 9 * * 1: every Monday at 9:00 AM
  • Timezone: IANA zone (e.g. America/New_York, Europe/London). Defaults to UTC. This matters for daily and weekly schedules that should align with business hours and handle DST.

The cron and timezone fields have length caps; see Limits & quotas.

Save the job. If the job is Active, the schedule starts firing at the next matching tick.

Active vs. paused

A job has a status independent of its schedule:

  • Active: the schedule fires and produces runs.
  • Paused: the definition is intact but no scheduled runs fire. Manual runs still work.

Pause is the first option to consider when a job is misbehaving. It is reversible and leaves history intact. Toggle from the job's detail page.

Concurrency policy

The concurrency policy decides what happens if a new tick fires while a previous run hasn't finished.

PolicyBehaviorUse when
skipSkip the new tick if any run is still non-terminalThe pipeline is idempotent and overlaps waste compute
allowRun in parallel, up to maxConcurrentRunsTicks are independent and can safely overlap
replaceCancel the running run, start a new oneOnly the latest tick matters (e.g. "refresh current state")

skip is the safest default. Pick allow when overlap has been confirmed safe. Pick replace when staleness matters more than completion.

When allow is chosen, set maxConcurrentRuns to cap concurrent execution.

Missed-schedule policy

The missed-schedule policy decides what happens if the scheduler was unavailable when a tick should have fired (cluster down, platform incident).

  • latest: fire exactly once on recovery, at the recovery time. Use for "keep me current" pipelines where missing ticks is fine as long as the next run catches up.
  • all: fire once per missed tick, with backdated dispatch times. Use for append-only pipelines where every tick matters (e.g. hourly ingest where each hour is a distinct partition).

latest is the default and fits most workloads. Pick all only when the workload is a function of the dispatch time rather than wall-clock.

Trigger a manual run

From the job's detail page, click Run now. A manual run queues immediately regardless of the schedule. Manual runs on a scheduled job do not affect the schedule. The next cron tick fires as usual.

Schedule hygiene

A few practices to avoid common problems:

  • Avoid */1 * * * * (every minute) for anything non-trivial. The control plane and Kubernetes scheduling overhead means ticks pile up. Use sessions for sub-minute work.
  • Align timezones with the humans who own the pipeline. A job owned by an NYC team should schedule in America/New_York. DST shifts otherwise cause two runs on one day and zero on another.
  • Pick concurrency and missed-schedule deliberately. The defaults (skip + latest) are safe but they mean missed work is lost. If that is not acceptable for the pipeline, change both.

API reference

  • Jobs: schedule fields (cronExpression, cronTimezone, concurrencyPolicy, maxConcurrentRuns, missedSchedulePolicy) live on the job entity. There is no separate schedule resource.

Can't find the answer here? Email us: support@lakesail.com