Task · BBC-0003

Migrate the repo task surface to just and retire Makefiles and ad-hoc scripts

Description

Migrate bumblebee-catalog’s task surface to just

1. Outcome

bumblebee-catalog gets a top-level justfile implementing the fleet’s seven mandatory recipes plus thin wrapper recipes for each of the six ci/*.py scripts. ci.yml calls just check instead of a bare python3 -m py_compile line. osv-catalog.yml and extra-catalogs.yml call just <recipe> <same args as today> instead of python3 ci/<script>.py <args> — behavior is byte-for-byte identical, only the invocation surface changes. AGENTS.md gets a “Task interface” section. backlog/config.yml’s definition_of_done names just recipes instead of a bare python3 -m py_compile line and separate actionlint/zizmor commands.

This repo has no Makefile and no shell scripts. git ls-files | grep -E '\.(sh|bash|zsh|ps1)$' returns nothing, and no Makefile/GNUmakefile exists anywhere in the tree. The “migration” here is narrower than most fleet repos: introduce the justfile, wrap the existing Python scripts, and switch three workflow files + two doc files to call it. Nothing is deleted except nothing (there is no dead file to remove).

Deliberate scope limit — read before touching osv-catalog.yml / extra-catalogs.yml: only the python3 ci/*.py ... invocation lines in these two workflows get converted to just <recipe> .... The surrounding orchestration (checkout, go build, curl release downloads + checksum verification, git clone/sparse-checkout, gh release create/upload, the floor-check gh release download) is left exactly as-is — untouched, not wrapped, not moved into the justfile. See §9 “Traps” for why.

2. The complete justfile

Create justfile at the repo root:

set shell := ["bash", "-euo", "pipefail", "-c"]

# show the task surface
default:
    @just --list

# nothing to install — ci/*.py is stdlib-only, no lockfile, no venv
setup:
    @echo "python3: {{ require('python3') }}"
    @echo "no dependencies to install (ci/*.py is stdlib-only)"

# format the justfile (no other formatter is configured in this repo)
[group('check')]
fmt:
    @just --fmt

# verify justfile formatting — fails if `just fmt` would change it
[no-exit-message]
[group('check')]
fmt-check:
    just --fmt --check

# syntax-check every CI script — the only static check this repo has today
[no-exit-message]
[group('check')]
lint:
    python3 -m py_compile ci/*.py

# lint and audit GitHub Actions workflows (needs actionlint + zizmor on PATH; not run by `check`)
[no-exit-message]
[group('check')]
lint-workflows:
    actionlint
    zizmor .github/workflows/

# no test suite in this repo — see AGENTS.md ("nothing that runs meaningfully on a laptop")
[group('check')]
test:
    @echo "no tests in this repo"

# the full local gate — exactly what ci.yml enforces
[group('check')]
check: fmt-check lint test

# CI superset: check plus workflow linting (actionlint.yml/zizmor.yml already run these as separate
# GitHub-native workflows; this recipe exists so a dev can reproduce the same gate locally)
[group('check')]
ci: check lint-workflows

# regenerate the DataDog malicious-package catalog — args pass straight through to the script
[group('gen')]
gen-datadog-catalog *args:
    python3 ci/datadog_catalog.py {{ args }}

# regenerate the GHSA (CWE-506) malware catalog — args pass straight through to the script
[group('gen')]
gen-ghsa-catalog *args:
    python3 ci/ghsa_catalog.py {{ args }}

# validate a generated catalog and write its *-meta.json — args pass straight through to the script
[group('gen')]
validate-catalog *args:
    python3 ci/validate.py {{ args }}

# build a fixture package that should trip a given catalog — args pass straight through to the script
[group('gen')]
make-fixture *args:
    python3 ci/make_fixture.py {{ args }}

# assert a bumblebee scan actually flagged the seeded fixture — args pass straight through to the script
[group('gen')]
assert-finding *args:
    python3 ci/assert_finding.py {{ args }}

# assert the full published catalog SET loads in the deployed bumblebee binary — args pass through
[group('gen')]
assert-catalog-set *args:
    python3 ci/assert_catalog_set.py {{ args }}

Notes for whoever implements this:

3. Makefile disposition

N/A — confirmed via find . -iname Makefile -o -iname GNUmakefile (excluding vendor/, node_modules/, third_party/, .venv/): no Makefile exists anywhere in this repo. No git rm needed for this section.

4. Script disposition

Script Lines Disposition Recipe Why
ci/assert_catalog_set.py 123 KEEP just assert-catalog-set <args> Real program: argparse, subprocess, temp-dir handling, downloads/spawns the bumblebee binary. Not a thin wrapper.
ci/assert_finding.py 44 KEEP just assert-finding <args> Parses NDJSON scan output and asserts structural content — logic, not sequencing.
ci/datadog_catalog.py 125 KEEP just gen-datadog-catalog <args> Real catalog-generation program (manifest parsing, ecosystem filtering, null-record dropping).
ci/ghsa_catalog.py 182 KEEP just gen-ghsa-catalog <args> Real catalog-generation program (advisory parsing, CWE-506 filtering, MAL- alias dedup).
ci/make_fixture.py 63 KEEP just make-fixture <args> Builds a seeded fixture package on disk — a program, not a sequencing wrapper.
ci/validate.py 117 KEEP just validate-catalog <args> Schema/entry-count validation + meta-file writer with real branching logic (--force, floor check).

No script in this repo is ABSORB-eligible — there are no thin cd-and-run-a-tool shell wrappers at all, tracked or otherwise. Every “task” in this repo’s workflows is either a bare shell one-liner already (go build ..., curl ...) or a call into one of the six real Python programs above. Every KEEP script above gets a just recipe as its entry point; nobody should invoke python3 ci/<script>.py directly again outside the recipe body itself.

5. CI changes

.github/workflows/ci.yml

Add a setup-just step and collapse the syntax-check step to just check:

      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      - uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
        with:
          just-version: '1.58.0'
      - name: Run the check gate
        run: just check

This replaces the existing Syntax-check CI scripts step (run: python3 -m py_compile ci/*.py). just check runs fmt-check (new — checks the justfile’s own formatting), lint (the same py_compile command, unchanged), and test (a no-op that prints why). Net effect: strictly more coverage than today, zero behavior change to the existing check.

Do not touch: the harden-runner step, the ci-success job, its needs: [validate], or the if: always() / contains(needs.*.result, ...) logic. ci-success is the branch-ruleset gate name.

.github/workflows/osv-catalog.yml

Insert setup-just right after the first checkout step (Checkout this repo (CI scripts)), before Checkout bumblebee (pinned):

      - name: Checkout this repo (CI scripts)
        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
        with:
          persist-credentials: false

      - uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
        with:
          just-version: '1.58.0'

      - name: Checkout bumblebee (pinned)
        ...

Then three exact line-for-line substitutions further down (everything else in each step — comments, set -euo pipefail, surrounding shell — stays untouched):

  1. Step “Validate + write meta” — replace the python3 ci/validate.py \ invocation:

            python3 ci/validate.py \
              --catalog "$OUTPUT" \
              --old "old/$OUTPUT" \
              --ossf-sha "$OSSF_SHA" \
              --bumblebee-sha "$BUMBLEBEE_SHA" \
              --target-schema "$CATALOG_SCHEMA_VERSION" \
              "${force[@]}"

    becomes

            just validate-catalog \
              --catalog "$OUTPUT" \
              --old "old/$OUTPUT" \
              --ossf-sha "$OSSF_SHA" \
              --bumblebee-sha "$BUMBLEBEE_SHA" \
              --target-schema "$CATALOG_SCHEMA_VERSION" \
              "${force[@]}"

    (the force=() array-building if above it is untouched — that’s step logic, not a script call)

  2. Step “Positive-match self-test (catalog actually matches)” — replace both invocations:

    • python3 ci/make_fixture.py --catalog "$OUTPUT" --out fixturejust make-fixture --catalog "$OUTPUT" --out fixture
    • python3 ci/assert_finding.py --ndjson scan.ndjson --seed-file fixture/seed.jsonjust assert-finding --ndjson scan.ndjson --seed-file fixture/seed.json
  3. Step “Assert the catalog SET loads in the deployed binary” — replace:

            python3 ci/assert_catalog_set.py "${args[@]}"

    with

            just assert-catalog-set "${args[@]}"

    (the args=(...) array construction and the gh release download peer-fetch above it are untouched)

Do not touch: Checkout bumblebee (pinned), Set up Go, Build bumblebee (generator only), Download the DEPLOYED release, Sparse-checkout OSSF malicious-packages, Download last-good (floor check), Publish catalog-latest release asset, the concurrency: block, the permissions: contents: write, or any env: value (especially BUMBLEBEE_SHA / BUMBLEBEE_RELEASE / CATALOG_SCHEMA_VERSION — see AGENTS.md’s “Hard constraints” section, unrelated to this migration).

.github/workflows/extra-catalogs.yml

Same setup-just insertion point (right after Checkout this repo (CI scripts), before Checkout bumblebee (pinned)), same SHA/version pin as above. Then:

  1. Step “Generate datadog-malicious.json” — replace:

            python3 ci/datadog_catalog.py \
              --samples dd/samples \
              --source "${DATADOG_REPO}@${DD_SHA}" \
              --out datadog-malicious.json

    with

            just gen-datadog-catalog \
              --samples dd/samples \
              --source "${DATADOG_REPO}@${DD_SHA}" \
              --out datadog-malicious.json
  2. Step “Generate ghsa-malicious.json” — replace:

            python3 ci/ghsa_catalog.py \
              --advisories ghsa/advisories/github-reviewed \
              --source "${GHSA_REPO}@${GHSA_SHA}" \
              --out ghsa-malicious.json

    with

            just gen-ghsa-catalog \
              --advisories ghsa/advisories/github-reviewed \
              --source "${GHSA_REPO}@${GHSA_SHA}" \
              --out ghsa-malicious.json
  3. Step “Validate + write meta” — replace both python3 ci/validate.py \ invocations with just validate-catalog \, keeping every arg line under each identical (one call for datadog-malicious.json/datadog-meta.json/--label datadog, one for ghsa-malicious.json/ghsa-meta.json/--label ghsa). The shared force=() block above both calls is untouched.

  4. Step “Positive-match self-test (each catalog actually matches)” — inside the for cat in datadog-malicious.json ghsa-malicious.json; do ... done loop, replace:

    • python3 ci/make_fixture.py --catalog "$cat" --out "fixture-$cat"just make-fixture --catalog "$cat" --out "fixture-$cat"
    • python3 ci/assert_finding.py --ndjson "scan-$cat.ndjson" --seed-file "fixture-$cat/seed.json"just assert-finding --ndjson "scan-$cat.ndjson" --seed-file "fixture-$cat/seed.json"

    (the loop itself, the release/bumblebee scan ... invocation inside it, and the PKGVER=$(...) capture stay exactly as-is — the loop’s control flow is not being touched, only the two script calls inside it)

  5. Step “Assert the catalog SET loads in the deployed binary” — replace:

            python3 ci/assert_catalog_set.py "${args[@]}"

    with

            just assert-catalog-set "${args[@]}"

Do not touch: Checkout bumblebee (pinned), Set up Go, Build bumblebee (generator only), Download the DEPLOYED release, Fetch DataDog dataset manifests, Clone GitHub Advisory Database, Download last-good catalogs (floor check), Publish catalog-latest release assets, the concurrency: block, permissions: contents: write, or any env: value.

Workflows explicitly out of scope for this task

actionlint.yml, codeql.yml, dependency-review.yml, keepalive.yml, scorecard.yml, zizmor.yml — all six are either GitHub-native reusable-workflow calls (uses: rknightion/.github/.github/workflows/...@<sha>) or a bare git commit --allow-empty && git push housekeeping step. §8 of the fleet standard forbids converting a uses: into run: just, and none of these six files contain any build/test/lint/format/generate/validate shell logic to collapse. Do not edit any of these six files.

6. Docs and agent-contract changes

AGENTS.md

Add a new ## Task interface section. Insert it directly after the ## Verifying a change section (before ## Commits), replacing nothing — this is new content:

## Task interface

This repo's task surface is a `justfile`. Discover it, don't guess it:

    just --list                        # human-readable
    just --dump --dump-format json     # machine-readable
    just --show <recipe>               # what a recipe actually runs

- `just check` is the full local gate and is exactly what `ci.yml` enforces. It must pass before you
  commit.
- Prefer `just <recipe>` over the underlying tool. If you are typing `python3 ci/validate.py`, you
  want `just validate-catalog` instead.
- Run `just` with stdin from /dev/null. This repo has no `[confirm]`-marked recipes today, but if one
  is added later, stop and ask before running it — never pass `--yes` or `JUST_YES=1`.
- If a task you need does not exist, add a recipe with a `#` doc comment and a `[group(...)]` rather
  than running a bare command.

Also update the existing “Verifying a change” section’s opening paragraph, which currently reads:

ci.yml runs python3 -m py_compile ci/*.py and nothing else. It proves the scripts parse. It does not run them, does not touch a catalog, and will not fail for most things a change can break.

Change ci.yml runs python3 -m py_compile ci/*.py and nothing elsetoci.ymlrunsjust check and nothing else. Leave the rest of that paragraph (the “weak claim” point and the three rising-strength verification levels) unchanged — those levels (run the script locally, actionlint + zizmor, workflow_dispatch) are still accurate and now map onto just <recipe> / just lint-workflows naturally, but the prose doesn’t need to say so explicitly.

CLAUDE.md

No change — it is a one-line @AGENTS.md import already.

README.md

No change — it documents the consumer-facing release-asset contract and the workflow behavior (cadence, schema-version rules), never a make/script invocation a human would type. Confirmed via grep -n 'make \|\./scripts\|python3 ci/' README.md returning nothing.

7. backlog/config.yml

Current:

definition_of_done:
  - "python3 -m py_compile ci/*.py"
  - "actionlint (only if a .github/workflows file changed)"
  - "zizmor .github/workflows/ (only if a .github/workflows file changed)"

New:

definition_of_done:
  - "just check"
  - "just lint-workflows (only if a .github/workflows file changed)"

Edit this file by hand — per AGENTS.md, backlog/config.yml is the one Backlog.md file edited directly rather than through the CLI (list-valued keys aren’t settable via backlog config set).

8. Order of work

  1. Add the justfile (§2) at the repo root. Run just --fmt --check, just --list, just check locally to confirm the seven mandatory recipes work and check passes.
  2. Sanity-check each gen-*/validate-catalog/make-fixture/assert-* recipe by hand once, with made-up throwaway args, to confirm *args pass-through doesn’t mangle anything (e.g. just validate-catalog --help should print the script’s own argparse help).
  3. Edit ci.yml (§5) — add setup-just, switch to just check. Push (this repo commits straight to main, no PR — see AGENTS.md “Commits”). Watch the next ci.yml run go green.
  4. Edit osv-catalog.yml and extra-catalogs.yml (§5) — add setup-just, swap the six/five script invocations. Push.
  5. Verify via workflow_dispatch: manually trigger osv-catalog.yml and, separately (never both at once — same rolling release, separate concurrency groups), extra-catalogs.yml. Confirm each completes and the release assets update. This is the only way to prove the just-wrapped invocations still work end to end — per AGENTS.md, ci.yml alone is a weak signal.
  6. Update AGENTS.md (§6) and backlog/config.yml (§7). Push.
  7. There is no deletion step — no Makefile, no ABSORB scripts. Skip.

Do not skip step 5. These two workflows run on a schedule against real external state (OSSF feed, DataDog dataset, GHSA advisories, the deployed bumblebee release); a broken *args pass-through would silently stop publishing catalog updates with no PR-time signal at all, since neither workflow runs on push/PR.

9. Traps specific to this repo

10. Out of scope

Acceptance Criteria

Definition of Done

Implementation Plan

  1. Add and format the top-level justfile with the seven mandatory recipes plus the six real-program wrappers; omit a ci recipe because this GitHub-hosted repo has no Docker, service-container, or cross-compilation leg (fleet goal §6.2 supersedes the task’s earlier ci: check lint-workflows proposal).
  2. Rewire only the specified Python invocation lines in ci.yml, osv-catalog.yml, and extra-catalogs.yml, adding the SHA-pinned setup-just action before first use while preserving all orchestration, pins, concurrency, permissions, and reusable-workflow callers.
  3. Update AGENTS.md and backlog/config.yml to make just the documented task surface and definition-of-done gate.
  4. Validate formatting, recipe discovery/dump, argument passthrough, check, workflow lint/security audit, hook surfaces, removed-name searches, diff invariants, and the final pushed CI run; finalize atomically with the evidence.

Implementation Notes

Preflight: main is clean and matches origin/main at 45aa682871deca261882c82db41a3959879bc748; origin points at rknightion/bumblebee-catalog. Public REST identity lookup returned HTTP 403 without authentication, and no login will be attempted. No tracked Makefile, task-shell script, or hook surface is present.

Validation: just –fmt –check, just –dump –dump-format json, just check, all six wrapper recipes with –help, actionlint, and zizmor completed successfully. The push CI run 33255172264 completed successfully at 87a96be71f4b11ed13cfc50900c48255c6573064. Zizmor reported one pre-existing warning in untouched keepalive.yml; it exited 0. No tracked hook surface or configured hooks path exists. CodeRabbit was skipped under policy: the diff is declarative CI/configuration wiring with no changed branching logic.

Unparked and completed 2026-08-29. Both publishers proven at this head: the scheduled OSV catalog run succeeded, and a dispatched extra-catalogs run also succeeded. Migration at 45c66a6; exact-head CI green.

Comments

author: campaign-ordering created: 2026-08-29 09:18

Fleet ordering — WAVE 2. Starts after the Wave 0 pilot (sf2loki / SFL-0073) and the Wave 1 hubs land.

Within Wave 2 the order is free — these repos do not depend on each other. Batching by language is worthwhile so one lane reuses its Makefile-to-recipe mapping across similar repos.

Do not start before the pilot reports. The standard may be amended off the back of it, and picking this up early risks coding against a superseded seam.

Provisioning just in CI. Which mechanism depends on the runner, and the two must not be mixed:

Runner Mechanism
arc-arm64 (m7kni self-hosted) just is baked into the runner image by m7kni/ci-tools (runner-image/Dockerfile, ARG JUST_VERSION). Do not add extractions/setup-just, and delete the step if this repo already has one — it installs a second just earlier on PATH and turns the image pin into a lie.
GitHub-hosted (all rknightion repos) extractions/setup-just, SHA-pinned, with an explicit just-version:.

Both sides currently sit on 1.58.0 and are Renovate-managed. ci-toolsTool version drift workflow fails if the Dockerfile ARG and the published image ever disagree, and lists any repo still carrying a second pin.

While you are in the workflow files, check the hub pin. On 2026-08-29 Renovate was unfrozen for rknightion/.github in m7kni/renovate-config — it had been enabled: false on the mistaken belief that callers tracked @main, which froze the fleet across 19 different hub SHAs (v1.3.1 June → v1.9.7 August) so that no hub fix ever propagated. Bumps now arrive as one grouped, CI-gated, automerged PR per repo. A uses: whose comment is not a real # vX.Y.Z still cannot be bumped (it resolves to a digest-only update, which the fleet rules disable) — if you find one, repair the comment as part of this task.

author: campaign-ordering created: 2026-08-29 10:42

Standard amendment — ci is the sanctioned superset of check (RATIFIED)

This supersedes the frozen wording check is the complete local gate and reproduces every CI job that can run off a GitHub runner”, which several lanes could not honour without making the pre-commit gate depend on a Docker daemon.

The definitions now are:

Every leg you put in ci must carry a comment naming which of those three it needs. That comment is the guard: without it ci becomes the bin for anything slow or awkward, check quietly stops meaning much, and the fleet is back to a per-repo gate.

Eleven of the 42 lanes arrived at this shape independently before it was ratified, which is why it won.

If this repo has no such legs, it has no ci recipe at all and check is the whole gate. Do not add an empty one.

Final Summary

Migrated the task surface to just in 87a96be71f4b11ed13cfc50900c48255c6573064: added the seven mandatory recipes and six wrapper recipes; rewired the three specified workflows through SHA-pinned setup-just; documented the interface; and updated the definition of done. Verified local gates and successful CI run 33255172264 at that SHA. Parked solely because acceptance criterion 10 needs authenticated, sequential workflow_dispatch runs for osv-catalog.yml and extra-catalogs.yml; this lane has no GitHub API authentication and must not log in. Resume by checking no catalog run is active, dispatching osv-catalog.yml and confirming success/release assets, then dispatching extra-catalogs.yml separately and confirming success/release assets.

View the source file on GitHub