Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 114 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,11 @@ After the query is merged, we welcome pull requests to improve it.

## Supported CodeQL versions

Every query pack in this repository is compiled and tested against a specific version of the upstream CodeQL standard libraries (e.g. `codeql/java-all`). These queries are **only guaranteed to compile** against the exact library versions shown below newer or older CodeQL CLI/library versions may rename or remove APIs the queries depend on (see [#151](https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/151) for an example, and [#145](https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/issues/145) for the ongoing effort to refresh these pins).
Every query pack in this repository is compiled and tested against a specific version of the upstream CodeQL standard libraries (e.g. `codeql/java-all`). These queries are **only guaranteed to compile** against the exact library versions shown below: newer or older CodeQL CLI/library versions may rename or remove APIs the queries depend on (see [#151](https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/151) for an example, and [#145](https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/issues/145) for the ongoing effort to refresh these pins).

The pinning is codified in two places per language:
- [`.codeqlversion`][codeqlversion] (repo root) the CodeQL **CLI** version CI installs and compiles/tests against.
- `<language>/src/codeql-pack.lock.yml`the exact resolved (locked) version of `codeql/<language>-all` and its transitive dependencies, generated by `codeql pack install` against the CLI above. The `qlpack.yml` files themselves only declare an unpinned `'*'` range, so the lock filenot the qlpack.yml is the real source of truth.
- [`.codeqlversion`][codeqlversion] (repo root): the CodeQL **CLI** version CI installs and compiles/tests against.
- `<language>/src/codeql-pack.lock.yml`: the exact resolved (locked) version of `codeql/<language>-all` and its transitive dependencies, generated by `codeql pack install` against the CLI above. The `qlpack.yml` files themselves only declare an unpinned `'*'` range, so the lock file, not the qlpack.yml, is the real source of truth.

CodeQL CLI: `v2.21.1` (released 2025-04-16) ([Bundle](https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.21.1) / [Binary](https://github.com/github/codeql-cli-binaries/releases/tag/v2.21.1))

Expand All @@ -85,6 +85,106 @@ Most of our query packs only depend on the standard library (`*-all`) for CodeQL
> [!NOTE]
> This table is maintained by hand today; update it whenever `.codeqlversion` or the `codeql-pack.lock.yml` files are refreshed. For a broader mapping of CodeQL CLI/bundle versions to per-language library versions (useful when triaging why a query compiles locally but not in CI, or vice versa), see the community [CodeQL Bundle Version Tracker](https://github.com/advanced-security/advanced-security-material/blob/main/codeql/codeql-version-tracker.md).

## Releases & publishing

Publishing a package to the GitHub Container Registry (GHCR) and creating a GitHub Release are two
**separate, decoupled** processes in this repository. This section documents the actual, current
process for each; most of it is manual today.

### Shipping a change to a query/library pack

[`publish.yml`][publish-workflow] runs on every push to `main`. It's organized as four jobs: one
per pack type (`queries` for `src`, `library` for `lib`, `extensions` for `ext`,
`library_sources_extensions` for `ext-library-sources`), each matrixed over every language that has
that pack type (`ext`/`ext-library-sources` only run for `csharp`/`java` today, see [#144][pr-144]).

**Each `<language>` × `<pack type>` combination is checked and published completely
independently.** For every matrix entry, the job compares the `version:` in that one pack's
`qlpack.yml` on `main` to the version currently published on [GHCR][ghcr-packages], and only
installs + publishes *that specific pack* if they differ. It never touches any other language or
pack type. Concretely, this means:

- **You only need to bump the version of the pack(s) you actually changed.** If your PR only
touches `java/src`, bumping `java/src/qlpack.yml`'s version is enough; you do not need to touch
`csharp`, `go`, `python`, or any other language/pack type for `codeql-java-queries` to publish.
- **Nothing cascades automatically.** If a single PR changes both `java/src` and `csharp/lib`, each
needs its own version bump: bumping one does not publish the other, and bumping neither means
neither publishes.
- **Merging a change alone does not publish it.** The workflow runs on every merge, but a pack's own
`version:` field must have changed since the last publish, or that pack's matrix entry does nothing
and the change sits on `main` unpublished indefinitely.

To ship a change:

- [ ] Make your change in the pack directory you intend to publish: `<language>/src` (queries),
`<language>/lib` (library), or `<language>/ext`/`<language>/ext-library-sources` (extensions,
`csharp`/`java` only).
- [ ] Bump `version:` in that pack's `qlpack.yml`, following [semver](https://semver.org/). Only bump
the specific pack(s) you changed; other languages/pack types are unaffected and don't need
touching.
- [ ] If you changed a pack that other packs depend on (e.g. `<language>/ext`), check whether
dependents pin an exact version of it (e.g. `<language>/lib/qlpack.yml`) and bump that pin too
(these can drift out of sync otherwise, see [#155][pr-155]).
- [ ] Open a PR and get it reviewed/merged to `main`.
- [ ] Nothing further to do: `publish.yml` detects the version diff for that pack and publishes it
automatically on merge. There's no separate "publish" button or manual trigger step for a
version that's already bumped.

There is no in-repo inventory of "what's currently published" today; check the
[GHCR Packages page][ghcr-packages] for this repo directly, or compare against the `version:` field
in each language's `qlpack.yml` on `main` to see what will publish next.

### Updating the pinned CodeQL CLI/library version

Bumping the CodeQL CLI/library version this repo builds against (tracked in
[`.codeqlversion`][codeqlversion], see [Supported CodeQL versions](#supported-codeql-versions)
above) is also a fully manual process today. Nothing currently detects new upstream CodeQL CLI
releases automatically. A maintainer has to notice one exists and kick off this checklist by hand
(see [#118][pr-118] for an open proposal to at least automate the `codeql pack upgrade` step):

- [ ] Update `.codeqlversion` to the new CLI version.
- [ ] Run `codeql pack upgrade <dir>` for each pack directory to refresh its `codeql-pack.lock.yml`.
- [ ] Fix any compilation/test errors caused by upstream API changes (usually the hardest part, see
[#124][pr-124] for an example of what this can involve).
- [ ] Bump the `version:` field of every pack that changed, so the "Shipping a change" steps above
actually publish the update.
- [ ] Update the "Supported CodeQL versions" table above.
- [ ] Open a PR and get it reviewed/merged.

> [!WARNING]
> The `.codeqlversion` bump and the pack version bumps don't have to land in the same PR, but
> splitting them is risky: [#124][pr-124] refreshed `.codeqlversion` and every language's
> dependencies/lock files for `v2.21.1`, without bumping any pack's `version:` field in the same PR.
> The companion PR to bump every pack's `version:` ([#126][pr-126]) went unmerged for a long stretch
> afterward, during which most languages' published GHCR packages silently kept serving pre-`v2.21.1`
> content even though `main` had already moved on. Don't assume a merged dependency-refresh PR means
> consumers received it. Check that the pack's `version:` actually changed and published too.

### What GitHub Releases are for

The [Releases][releases] tab (`v0.2.0`, `v0.2.1`, ...) is a **repo-wide changelog**, unrelated to the
per-pack publishing described above:

- [`.release.yml`][release-config] is config for the [`42ByteLabs/patch-release-me`][patch-release-me]
tool. It tracks a single repository-wide `version:` and defines two patch locations: one targeting
`configs/*.yml` (has never matched anything in this repo's history; those configs never pin an
exact version, so this is effectively dead), and one targeting the exact `codeql-<lang>-libs:`
dependency pin in `**/qlpack.yml` (only 5 of 7 languages, cpp, go, javascript, python, ruby, pin an
exact version there; csharp/java use `'*'`). Either way, it does **not** bump any pack's own
top-level `version:` field, so bumping it alone doesn't publish anything.
- [`update-release.yml`][update-release-workflow] is a manual `workflow_dispatch` (pick
patch/minor/major) that runs that tool and opens a PR with the bumped `.release.yml` and patched
references.
- The GitHub Release itself is created manually afterwards by a maintainer via GitHub's "Draft a new
release" UI with auto-generated notes.

> [!NOTE]
> These systems have no mechanism keeping them in sync, and have drifted in practice: `.release.yml`'s
> `version:`, the latest GitHub Release tag, and individual packs' published `version:` fields can all
> disagree with each other at any given time. Don't use the Releases tab or `.release.yml` to infer
> what's currently published. Check [GHCR][ghcr-packages] or a pack's `qlpack.yml` directly, per
> "Shipping a change" above.

## Using your personal data

If you contribute to this project, we will record your name and email address (as provided by you with your contributions) as part of the code repositories, which are public. We might also use this information to contact you in relation to your contributions, as well as in the normal course of software development. We also store records of CLA agreements signed in the past, but no longer require contributors to sign a CLA. Under GDPR legislation, we do this on the basis of our legitimate interest in creating the CodeQL product.
Expand All @@ -94,3 +194,14 @@ Please do get in touch (privacy@github.com) if you have any questions about this
<!-- Resources / Links -->

[codeqlversion]: ./.codeqlversion
[publish-workflow]: ./.github/workflows/publish.yml
[update-release-workflow]: ./.github/workflows/update-release.yml
[release-config]: ./.release.yml
[patch-release-me]: https://github.com/42ByteLabs/patch-release-me
[releases]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/releases
[ghcr-packages]: https://github.com/orgs/GitHubSecurityLab/packages?repo_name=CodeQL-Community-Packs
[pr-118]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/118
[pr-124]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/124
[pr-126]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/126
[pr-144]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/144
[pr-155]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/155