Skip to content

CI: accept unslothai/llama.cpp PRs in pr-set.json (#30 mirror) #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
oobabooga wants to merge 4 commits into
base: pr-set-unslothai-base
Choose a base branch
from
Open

CI: accept unslothai/llama.cpp PRs in pr-set.json (#30 mirror) #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ef29dd8
Prebuilds: accept unslothai/llama.cpp PRs in pr-set.json
oobabooga Jul 7, 2026
5eed57e
Trim the pr-set.json doc
oobabooga Jul 7, 2026
97f2c2a
Pin unslothai#16 to exercise the fork-pin path (throwaway, CI only)
oobabooga Jul 7, 2026
dd61316
Throwaway: run the resolve mix logic on CI (do not merge)
oobabooga Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
297 changes: 297 additions & 0 deletions .github/workflows/throwaway-resolve-mix-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,297 @@
# Throwaway CI-only workflow: runs the resolve job's script from this branch's
# unsloth-prebuilt.yml byte-for-byte (single deviation: base tag pinned to b9902)
# to exercise the mixed ggml-org + unslothai pin set on a runner. Do not merge.
name: 'Throwaway: resolve mix test (do not merge)'
'on':
push:
branches:
- pr-set-unslothai-ci
permissions:
contents: read
jobs:
resolve-test:
runs-on: ubuntu-22.04
env:
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@v6
- id: r
name: Resolve (verbatim from unsloth-prebuilt.yml, base pinned to b9902)
run: |
set -euo pipefail
REQ='b9902' # throwaway: pinned base (orig: github.event.inputs.tag || latest)
ONLY='${{ github.event.inputs.only_profile || 'all' }}'
GFX_DEFAULT='gfx1151,gfx1150,gfx120X,gfx110X,gfx103X,gfx90a,gfx908'
GFX='${{ github.event.inputs.gfx_target }}'; GFX="${GFX:-$GFX_DEFAULT}"
OS_LIST='${{ github.event.inputs.operating_systems || 'windows,ubuntu' }}'
AGE_H='${{ github.event.inputs.min_age_hours }}'

# publish does delete-then-create on the whole release, and the installer
# treats the published manifest as the authoritative bundle set: a partial
# build drops bundles and silently downgrades uncovered hosts to slow
# source builds. Refuse to publish unless the full default set is built;
# publish=false test runs may use subsets.
if [ "${{ github.event_name }}" = "schedule" ] || [ "${{ inputs.publish }}" = "true" ]; then
[ "$ONLY" = "all" ] || { echo "refusing to publish only_profile=$ONLY: a partial CUDA set clobbers manifest coverage. Use only_profile=all to publish, or publish=false for an artifact-only test." >&2; exit 1; }
[ "$GFX" = "$GFX_DEFAULT" ] || { echo "refusing to publish gfx_target='$GFX': publish requires the full default set ($GFX_DEFAULT); use publish=false for a subset test." >&2; exit 1; }
case "$OS_LIST" in
*windows*ubuntu*|*ubuntu*windows*) : ;;
*) echo "refusing to publish operating_systems='$OS_LIST': both windows and ubuntu ROCm bundles are required. Use publish=false for a subset test." >&2; exit 1 ;;
esac
fi
[ -n "$AGE_H" ] || AGE_H="${UNSLOTH_LLAMA_MIN_RELEASE_AGE_HOURS:-6}"
if [ "$REQ" = "latest" ]; then
# Newest published (non-draft, non-prerelease) release that has been
# public for >= AGE_H hours -- the supply-chain aging window. GitHub
# does not guarantee the list order, so pick the max by published_at
# explicitly rather than trusting `first`.
CUTOFF="$(date -u -d "-${AGE_H} hours" +%s)"
BASE="$(gh api 'repos/ggml-org/llama.cpp/releases?per_page=100' \
--jq "[.[] | select(.draft==false and .prerelease==false) | select((.published_at|fromdateiso8601) <= ${CUTOFF})] | max_by(.published_at|fromdateiso8601) | .tag_name")"
if [ -z "$BASE" ] || [ "$BASE" = "null" ]; then
echo "refusing: no ggml-org release older than ${AGE_H}h in the last 100 releases" >&2
exit 1
fi
echo "selected $BASE (aged >= ${AGE_H}h)"
else
BASE="$REQ" # explicit b#### override skips the aging filter
fi
printf '%s' "$BASE" | grep -qE '^b[0-9]+$' || { echo "refusing non-release tag '$BASE'" >&2; exit 1; }

# Resolve the PR mix set (scripts/unsloth/pr-set.json): PR commit
# urls from ggml-org/llama.cpp or unslothai/llama.cpp (no other
# repos). Pins are mandatory -- only an exact, reviewed commit
# is ever built, so a PR author pushing more commits cannot change
# what the nightly ships. Non-open PRs are skipped, so merged or
# abandoned entries rot away instead of breaking the nightly (but a
# merged unslothai pin drops out of the build entirely; see the
# pr-set.json _doc). unsloth-pr-set-lint.yml runs the same checks on every
# push that edits the file, but that is only a tripwire -- a red
# lint does not stop the schedule, so the gate must live here.
jq -e '.prs | type == "array" and all(.[]; type == "string")' scripts/unsloth/pr-set.json >/dev/null \
|| { echo "scripts/unsloth/pr-set.json: .prs must be an array of PR url strings" >&2; exit 1; }
PRS='[]'
URL_RE='^https://github\.com/(ggml-org|unslothai)/llama\.cpp/pull/([0-9]+)/commits/([0-9a-f]{40})/?$'
for url in $(jq -r '.prs[]' scripts/unsloth/pr-set.json); do
[[ "$url" =~ $URL_RE ]] || { echo "refusing malformed PR url '$url' (expected https://github.com/{ggml-org,unslothai}/llama.cpp/pull/<n>/commits/<40-hex-sha>)" >&2; exit 1; }
SRC="${BASH_REMATCH[1]}/llama.cpp"; NUM="${BASH_REMATCH[2]}"; SHA="${BASH_REMATCH[3]}"
# Abort naming the entry on a gh failure (typo'd numbers 404).
# Title can contain spaces, so it can't ride a space-delimited
# read -- pull each field out on its own.
PR_JSON="$(gh api "repos/${SRC}/pulls/${NUM}")" \
|| { echo "refusing ${SRC}#${NUM}: could not fetch PR metadata (nonexistent PR number in '$url', or a transient API failure); fix the pin in scripts/unsloth/pr-set.json or retry" >&2; exit 1; }
STATE="$(jq -r '.state' <<<"$PR_JSON")"
HEAD="$(jq -r '.head.sha' <<<"$PR_JSON")"
N_COMMITS="$(jq -r '.commits' <<<"$PR_JSON")"
TITLE="$(jq -r '.title' <<<"$PR_JSON")"
if [ "$STATE" != "open" ]; then
echo "skipping ${SRC}#${NUM} (${STATE}): $url"
continue
fi
# A pin pasted from the wrong PR would build arbitrary code while
# the manifest blames PR #<num>; require the pinned commit to be
# a commit of that PR. The commits listing is capped at 250 by
# the API; past that, skip rather than false-fail.
if [ "$N_COMMITS" -gt 250 ]; then
echo "note: ${SRC}#${NUM} has ${N_COMMITS} commits (over the API listing cap); skipping pin membership check"
elif ! gh api "repos/${SRC}/pulls/${NUM}/commits" --paginate --jq '.[].sha' | grep -qx "$SHA"; then
echo "refusing ${SRC}#${NUM}: pinned commit ${SHA} is not a commit of that PR (wrong paste, or force-pushed away); fix the pin in scripts/unsloth/pr-set.json" >&2
exit 1
fi
[ "$SHA" = "$HEAD" ] || echo "note: ${SRC}#${NUM} is pinned to ${SHA} but its head has moved to ${HEAD}"
echo "including ${SRC}#${NUM} @ ${SHA}"
PRS="$(jq -c --arg r "$SRC" --arg n "$NUM" --arg s "$SHA" --arg u "$url" --arg t "$TITLE" '. + [{repo: $r, number: ($n|tonumber), sha: $s, url: $u, title: $t}]' <<<"$PRS")"
done

# Decide the tag and source repo first; the source tree is built
# further down, only if this release isn't already published.
if [ "$(jq length <<<"$PRS")" = 0 ]; then
# Plain build: pristine upstream base, no merge. REPO stays the real
# upstream repo even though we ship our own stamped tarball.
TAG="$BASE"
REPO='ggml-org/llama.cpp'
else
# The synthetic tag embeds a hash of the pinned repo#number:sha
# triples in listed order (merge order can matter for conflicts), so
# a pin update or a reorder yields a new tag and build, while the
# "exists" check below still skips rebuilding an already-published
# set. The repo is part of the key so PR #n in the two repos can't
# hash to the same set. The merged tree exists only as this repo's
# release assets, never upstream.
SETHASH="$(jq -r 'map("\(.repo)#\(.number):\(.sha)") | join("\n")' <<<"$PRS" | sha256sum | cut -c1-7)"
TAG="${BASE}-mix-${SETHASH}"
REPO="$GITHUB_REPOSITORY"
fi
SRC_ARTIFACT="app-source-${TAG}"
COMMIT=""

# Only PUBLISHED releases count as "exists"; a leftover draft (from
# a failed prior publish) should not block today's rebuild.
EXISTS=false
if [ "$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json isDraft --jq .isDraft 2>/dev/null || true)" = "false" ]; then
EXISTS=true
fi

# Build the source tree every child compiles, ONCE, here -- but only
# when something will actually be built. On a scheduled no-op (the aged
# "latest" was already published) skip the whole checkout/merge/tar +
# upload; a manual dispatch always rebuilds. Check out the upstream base,
# merge any pinned PRs (mix builds), then bake the build number/commit
# and the Unsloth fingerprint into cmake/build-info.cmake. The
# result is uploaded as the app-source-* artifact every child extracts --
# one identical tree everywhere, no child clones, fingerprint patch in one
# place. Nothing is pushed (GITHUB_TOKEN may never push commits touching
# .github/workflows, which upstream history routinely does).
if [ "$EXISTS" != "true" ] || [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
git remote add upstream https://github.com/ggml-org/llama.cpp.git
if [ "$(jq length <<<"$PRS")" != 0 ]; then
# Merges need a merge-base, so unshallow first.
if [ "$(git rev-parse --is-shallow-repository)" = "true" ]; then
git fetch -q --unshallow --no-tags origin
fi
git fetch -q --no-tags upstream "refs/tags/${BASE}:refs/tags/${BASE}"
git checkout -q --detach "refs/tags/${BASE}"
# Each pin is fetched from the repo its PR lives in (repo/number/
# sha are machine fields, so the space-delimited read is safe;
# titles stay out of this loop).
while read -r SRC NUM SHA; do
git fetch -q --no-tags "https://github.com/${SRC}.git" "$SHA" \
|| { echo "could not fetch commit ${SHA} for ${SRC}#${NUM}; it may have been pruned after a force-push -- update or remove the pin in scripts/unsloth/pr-set.json" >&2; exit 1; }
git -c user.name='github-actions[bot]' -c user.email='41898282+github-actions[bot]@users.noreply.github.com' \
merge --no-ff --no-edit -m "Merge ${SRC}#${NUM} @ ${SHA}" "$SHA" \
|| { echo "${SRC}#${NUM} (${SHA}) does not merge cleanly onto ${BASE} + the PRs listed before it; reorder or drop it in scripts/unsloth/pr-set.json" >&2; exit 1; }
done < <(jq -r '.[] | "\(.repo) \(.number) \(.sha)"' <<<"$PRS")
else
# Plain build: only the base tree is needed. Shallow is fine -- the
# build number is baked below, so no git history is needed at build time.
git fetch -q --depth 1 --no-tags upstream "refs/tags/${BASE}:refs/tags/${BASE}"
git checkout -q --detach "refs/tags/${BASE}"
fi
COMMIT="$(git rev-parse HEAD)"

# The tarball ships without .git, where cmake/build-info.cmake falls
# back to its defaults; bake real values into those defaults so
# llama-server --version doesn't report b0 (unknown). The build number
# stays the BASE's b#### number (the upstream tag's commit count), so
# anything comparing versions against upstream releases keeps working;
# BUILD_COMMIT identifies the (possibly merged) head.
COUNT="${BASE#b}"
SHORT="$(git rev-parse --short HEAD)"
sed -i "s/^set(BUILD_NUMBER 0)$/set(BUILD_NUMBER ${COUNT})/" cmake/build-info.cmake
sed -i "s/^set(BUILD_COMMIT \"unknown\")$/set(BUILD_COMMIT \"${SHORT}\")/" cmake/build-info.cmake
grep -q "set(BUILD_NUMBER ${COUNT})" cmake/build-info.cmake \
&& grep -q "set(BUILD_COMMIT \"${SHORT}\")" cmake/build-info.cmake \
|| { echo "cmake/build-info.cmake no longer has the expected fallback lines; cannot bake the build number into the source artifact" >&2; exit 1; }

# Unsloth fingerprint: fold "Compiled by the Unsloth team"
# into BUILD_TARGET, which cmake bakes into LLAMA_BUILD_TARGET, so it
# prints on llama-server --version ("built with ... for ...") and shows
# up in `strings` on every binary that links common. Append so it wraps
# whatever BUILD_TARGET the build computes. Keep this string byte-identical
# to MARK in the assemble job's verify gate, which re-checks it landed.
grep -q 'set(BUILD_TARGET' cmake/build-info.cmake \
|| { echo "cmake/build-info.cmake has no BUILD_TARGET line to stamp" >&2; exit 1; }
printf '\n# Unsloth fingerprint: shows in --version and strings.\nset(BUILD_TARGET "${BUILD_TARGET} (Compiled by the Unsloth team)")\n' >> cmake/build-info.cmake
grep -q 'Compiled by the Unsloth team' cmake/build-info.cmake \
|| { echo "failed to stamp the Unsloth fingerprint into cmake/build-info.cmake" >&2; exit 1; }

tar -czf "${RUNNER_TEMP}/llama.cpp-source-${TAG}.tar.gz" --exclude-vcs --transform "s,^\.,llama.cpp-${TAG}," .
echo "prepared ${TAG} (${COMMIT}, build ${COUNT})"
else
echo "release ${TAG} already published; skipping source prep (nothing to build)"
fi

# ubuntu-22.04 is x64 (glibc 2.35). arm64 only has ubuntu-24.04-arm
# available on GitHub-hosted runners (glibc 2.39). arm64 is cuda13-only
# (no cuda12 SBSA) and ships the single "portable" coverage class.
# cuda12 installs via Jimver; cuda13 is pinned to 13.3 (matching
# upstream) which Jimver lacks, so those install via NVIDIA redist in
# the build children -- on the same runners, so the glibc floor holds.
ALL='[
{"profile":"cuda12-older", "arch":"x64", "runner":"ubuntu-22.04", "line":"cuda12","klass":"older", "rank":10,"cuda":"12.8.0","toolkit_line":"12.8","archs":"70 75 80 86 89"},
{"profile":"cuda12-newer", "arch":"x64", "runner":"ubuntu-22.04", "line":"cuda12","klass":"newer", "rank":20,"cuda":"12.8.0","toolkit_line":"12.8","archs":"86 89 90 100 120"},
{"profile":"cuda12-portable","arch":"x64", "runner":"ubuntu-22.04", "line":"cuda12","klass":"portable","rank":30,"cuda":"12.8.0","toolkit_line":"12.8","archs":"70 75 80 86 89 90 100 120"},
{"profile":"cuda13-older", "arch":"x64", "runner":"ubuntu-22.04", "line":"cuda13","klass":"older", "rank":40,"cuda":"13.3","toolkit_line":"13.3","archs":"75 80 86 89"},
{"profile":"cuda13-newer", "arch":"x64", "runner":"ubuntu-22.04", "line":"cuda13","klass":"newer", "rank":50,"cuda":"13.3","toolkit_line":"13.3","archs":"86 89 90 100 120"},
{"profile":"cuda13-portable","arch":"x64", "runner":"ubuntu-22.04", "line":"cuda13","klass":"portable","rank":60,"cuda":"13.3","toolkit_line":"13.3","archs":"75 80 86 89 90 100 120"},
{"profile":"cuda13-portable","arch":"arm64","runner":"ubuntu-24.04-arm","line":"cuda13","klass":"portable","rank":60,"cuda":"13.3","toolkit_line":"13.3","archs":"90 100 120 121"}
]'
if [ "$ONLY" = "all" ]; then
CUDA_INCLUDE="$(echo "$ALL" | jq -c .)"
else
CUDA_INCLUDE="$(echo "$ALL" | jq -c --arg p "$ONLY" '[.[] | select(.profile==$p)]')"
fi
# CUDA Windows reuses the x64 profiles (same arch lists / CUDA
# versions), just on a Windows runner. arm64 has no CUDA Windows target.
WIN_CUDA_INCLUDE="$(echo "$CUDA_INCLUDE" | jq -c '[.[] | select(.arch=="x64") | .runner="windows-2022"]')"
[ -n "$GFX" ] || { echo "refusing empty gfx_target (would publish a CUDA-only release labeled CUDA + ROCm)" >&2; exit 1; }
ROCM_MATRIX="$(jq -cn --arg g "$GFX" '{gfx_target: ($g | split(",") | map(gsub("^\\s+|\\s+$"; "")))}')"

# macOS slices are static: two fixed runners with per-slice deployment
# targets. arm64 builds on macos-26 (newest Metal SDK; avoids the
# M5/A19 "error compiling source" the macos-14 SDK emits) yet pins
# minos 14.0 via deploy_target, so it still loads on macOS 14+. x64
# pins 13.3 (matches upstream's Intel leg, covers 2017 Intel Macs
# capped at Ventura).
MACOS_INCLUDE='[
{"build":"arm64","runner":"macos-26", "expect_arch":"arm64", "deploy_target":"14.0","defines":"-DGGML_METAL_USE_BF16=ON -DGGML_METAL_EMBED_LIBRARY=ON"},
{"build":"x64", "runner":"macos-15-intel","expect_arch":"x86_64","deploy_target":"13.3","defines":"-DGGML_METAL=OFF"}
]'
MACOS_INCLUDE="$(echo "$MACOS_INCLUDE" | jq -c .)"

{
echo "tag=$TAG"
echo "repo=$REPO"
echo "base=$BASE"
echo "prs=$PRS"
echo "source_artifact=$SRC_ARTIFACT"
echo "commit=$COMMIT"
echo "exists=$EXISTS"
echo "cuda_matrix={\"include\":$CUDA_INCLUDE}"
echo "win_cuda_matrix={\"include\":$WIN_CUDA_INCLUDE}"
echo "rocm_matrix=$ROCM_MATRIX"
echo "macos_matrix={\"include\":$MACOS_INCLUDE}"
} >> "$GITHUB_OUTPUT"
echo "Resolved $REQ -> $TAG ($COMMIT); prs=$PRS; source_artifact=${SRC_ARTIFACT:-none}; release exists=$EXISTS; only=$ONLY; gfx=$GFX"
- name: Verify merged tree and outputs
env:
PRS_JSON: ${{ steps.r.outputs.prs }}
run: |
set -euo pipefail
TAG='${{ steps.r.outputs.tag }}'
echo "tag=$TAG"
grep -qE '^b9902-mix-[0-9a-f]{7}$' <<<"$TAG"
[ "$(jq length <<<"$PRS_JSON")" = 4 ]
[ "$(jq -r '.[3].repo' <<<"$PRS_JSON")" = "unslothai/llama.cpp" ]
git log --oneline --first-parent -5
git log --format=%s --first-parent -4 > /tmp/subjects.txt
grep -qx 'Merge unslothai/llama.cpp#16 @ e617ca3a4ddcbef48ef0d5cad8e5c87e39a97ebe' /tmp/subjects.txt
grep -q 'Compiled by the Unsloth team' cmake/build-info.cmake
tar -xzOf "${RUNNER_TEMP}/llama.cpp-source-${TAG}.tar.gz" "llama.cpp-${TAG}/cmake/build-info.cmake" > /tmp/bi.cmake
grep -q 'Compiled by the Unsloth team' /tmp/bi.cmake
echo "all assertions passed"
- name: Render release notes (dry run)
env:
PRS_JSON: ${{ steps.r.outputs.prs }}
run: |
set -eu
PRS="$PRS_JSON"
BASE='${{ steps.r.outputs.base }}'
NOTES="Automated Unsloth llama.cpp CUDA + ROCm + Vulkan + macOS + CPU prebuild for upstream [${BASE}](https://github.com/ggml-org/llama.cpp/releases/tag/${BASE})"
if [ "$(jq length <<<"$PRS")" = 0 ]; then
NOTES="${NOTES}."
else
PR_LIST="$(jq -r 'map("- \(.title) ([\(if .repo == "ggml-org/llama.cpp" then "" else .repo end)#\(.number)](https://github.com/\(.repo)/pull/\(.number)), commit [\(.sha[0:7])](\(.url)))") | join("\n")' <<<"$PRS")"
NOTES="$(printf '%s, merged with:\n\n%s' "$NOTES" "$PR_LIST")"
fi
printf '%s\n' "$NOTES" > /tmp/notes.md
cat /tmp/notes.md
grep -qF '([unslothai/llama.cpp#16](https://github.com/unslothai/llama.cpp/pull/16)' /tmp/notes.md
grep -qF '([#24423](https://github.com/ggml-org/llama.cpp/pull/24423)' /tmp/notes.md
echo "notes render OK"
- uses: actions/upload-artifact@v6
with:
name: ${{ steps.r.outputs.source_artifact }}
path: ${{ runner.temp }}/llama.cpp-source-${{ steps.r.outputs.tag }}.tar.gz
if-no-files-found: error
retention-days: 1
Loading
Loading