Cron hot-reload: CronService.reload + POST /api/cron/reload (5/13)

[alex-clickhouse]

What

CronService.reload() and POST /api/cron/reload. Re-reads the merged jobs, re-imports the gate plugins, unschedules the jobs that went away or were disabled, and rebuilds every job that is still enabled.

Every enabled job is rebuilt, not only the ones whose YAML changed. Rebuilding a trigger keeps the next fire time as it was for a crontab job, and for an interval job anchored to a successful run, so for those the work is redundant but free. That leaves one case where it costs something: an interval job that has never succeeded has nothing to anchor to, so each reload restarts its countdown. It's documented rather than worked around.

A gate spec that cannot be built now refuses the job instead of being skipped. An unknown type is the case that matters — it is what a deleted or unimportable gate plugin looks like from the loader — and it takes the job with it: a 400 that refuses the whole reload, or a skipped job with an error at startup. nerve config validate reports an unrecognized type as unverified rather than wrong, because it never loads plugins and so cannot tell a typo from a plugin's own type.

The reload is all-or-nothing. The whole change set is computed, and every trigger built, before the scheduler is touched. That covers the gate registry too: it is process-global and has to be replaced before jobs are rebuilt, because a CronJob builds its gates at construction time — so a reload that is then refused puts it back, rather than leaving a deleted plugin's gate unregistered and the jobs that used it unbuildable.

Reserved job ids (cleanup, wakeup_sweep, and the source: namespace) are enforced at load time.

Why

Config that can only be applied by restarting the daemon is config nobody edits. Restarting drops in-flight sessions, so the cost of a one-line schedule change was high enough to discourage making it.

Refusing a job over its gates is the part worth arguing. A gate is a precondition, so dropping one does not make a job safer — it makes it run more often. "Only when the inbox is busy" becomes "every time", while the config that said otherwise sits there looking correct. Skipping the spec keeps the job alive at the cost of silently inverting what it was for, and hot-reload turns that from a restart-time mistake into an unattended one: a workspace pull that deletes a gate plugin would widen every job using it, on a running daemon, within seconds of the pull. Refusing is the failure that shows up — the reload's 400 names the job, startup logs it, and the schedule that was already running is left alone.

The cost is real and accepted: a plugin that stops importing takes its jobs off the schedule until it is fixed, and one bad gate refuses a reload that may also carry unrelated edits. Deleting a plugin is therefore not a way to switch a gate off; it stops those jobs instead of ungating them, and the docs say so.

Rebuilding unconditionally rather than diffing is a separate trade. The alternative — compare each job against the running one and reschedule only what changed — protects the never-succeeded interval case above and lets the response say which jobs the file changed rather than which were rescheduled. It costs a definition of "changed" that has to be kept in step with CronJob, where a field left out is silently never rescheduled. The only consumer of that distinction is a log line, so updated in the response now means "already had a trigger and was rescheduled", and the docstring and docs/cron.md say so rather than implying a diff.

Reserving the id namespace at load time closes a quieter problem: a user job named cleanup would displace the Nerve-managed one, and the only symptom would be maintenance silently no longer running.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds cron hot-reload support to the Nerve daemon, allowing the running scheduler to be updated from on-disk cron config without a restart, while hard-reserving daemon-owned job IDs to prevent user cron definitions from displacing internal jobs or source runners.

Changes:

• Introduces CronService.reload() plus POST /api/cron/reload, applying scheduler changes by diff (add/remove/update/enable/disable) while preserving interval alignment for unchanged jobs.
• Enforces reserved cron job IDs (cleanup, wakeup_sweep, and the source: namespace) at load time and surfaces them in CLI reporting.
• Improves schedule parsing semantics (fractional intervals, and differentiating “not a crontab” vs “invalid crontab”) and adds extensive test coverage + docs for the new behavior.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/test_cron.py	Adds regression/behavior tests for fractional intervals and invalid crontab handling.
tests/test_cron_reload.py	Adds comprehensive reload tests (diff behavior, all-or-nothing safety, reserved IDs, API route behavior).
tests/test_cron_gate_plugins.py	Adds tests for gate-plugin hot-reload behavior (replace=True) and propagation into scheduled jobs.
nerve/sources/runner.py	Ensures SourceRunner.job_id is always derived under the reserved source: namespace.
nerve/gateway/routes/cron.py	Adds POST /api/cron/reload endpoint mapping config errors to HTTP 400.
nerve/cron/service.py	Implements reload/diff logic, reserved-id dropping, and new schedule error types.
nerve/cron/jobs.py	Adds reserved-id helpers and a strict mode for YAML/job validation used by reload.
nerve/cron/gate_plugins.py	Adds hot-reload support via replace=True (unregister + rescan) and warns on vanished gate types.
nerve/cli.py	Updates cron doctor/listing output to flag/exclude reserved IDs from enabled counts.
docs/cron.md	Documents reload behavior, reserved IDs, strict refusal cases, and interval syntax (including fractions).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.