Task · BBC-0004

Decide how to handle GitHub scheduled-workflow drift on the catalog crons

Status
To Do
Priority
medium
Updated
2026-08-29

Description

GitHub deprioritises schedule triggers under load, and both catalog workflows are drifting far enough that their documented cadence is no longer what endpoints actually receive. This was found while diagnosing a BumblebeeCatalogStale page; the alert thresholds have since been widened to absorb the drift, so this task is about whether the underlying publish cadence should be made reliable rather than merely tolerated.

Observed on 2026-08-29 via gh run list:

Why it matters: endpoints refresh their local copy before each scan, so publish cadence is the ceiling on catalog freshness fleet-wide. A skipped slot is not visible anywhere except the age metric – scans keep succeeding and keep reporting clean against an older catalog, which is the blind-detector failure mode the staleness alerts exist to catch. The alert thresholds (30h all-source, 18h OSV-only) currently absorb the observed drift, so drift getting worse is silent until it crosses those.

Options to weigh, no decision made yet:

  1. Accept the drift and treat the alert thresholds as the real contract. Zero work; documents the true guarantee rather than the nominal one.
  2. Add a second, offset schedule entry to each workflow so a skipped slot has a nearby retry. Cheap, stays on GitHub-hosted runners, but doubles nominal runs and GitHub may deprioritise both together.
  3. Trigger from something that is not GitHub’s scheduler (an external cron calling workflow_dispatch, or a self-hosted runner). Reliable, but adds an external dependency and a credential, and rknightion is a User account so it has no runner groups or org-level Actions policy – see the github-actions-oss rule.

Constraints that must hold whichever way this goes: the two workflows publish to the same catalog-latest release under separate concurrency groups, so they must never be dispatched concurrently. BUMBLEBEE_SHA and BUMBLEBEE_RELEASE live in both files and must stay in step. Do not touch CATALOG_SCHEMA_VERSION.

References:

Acceptance Criteria

Definition of Done

View the source file on GitHub