Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docker-compose/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ WEB_BASE_URL=http://localhost:3000
BETTER_AUTH_URL=http://localhost:3000
WORKER_PUBLIC_BASE_URL=http://localhost:3000

# Optional image overrides. Use matching commit tags for the app and migrator.
# WEB_IMAGE=ghcr.io/git-ai-project/git-ai-web-ee:abc1234
# CLICKHOUSE_MIGRATOR_IMAGE=ghcr.io/git-ai-project/git-ai-web-ee:abc1234-clickhouse-migrator

# Enterprise license key (required)
LICENSE_KEY=

Expand Down
6 changes: 4 additions & 2 deletions docker-compose/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ BullMQ dashboard is available at `http://localhost:3001`.

This package defaults to `ANALYZE_BATCH_PROVIDER=local` (no extra batch env required).
It also defaults to `STORAGE_BACKEND=local` (no cloud storage bucket required).
Postgres/ClickHouse migrators run from the same EE image as `web`/`worker` using `/app/scripts` and `/app/migrations`.
The Postgres migrator runs from the EE app image. The ClickHouse migrator uses
a dedicated image containing only ClickHouse and its schema migrations.

This package does not configure TLS/reverse proxy automatically. Use your own ingress/reverse proxy in front of `web` as needed.

Expand Down Expand Up @@ -53,7 +54,8 @@ This removes the book demo / booking-gated onboarding screen for that org.

## Image Override

- Set `WEB_IMAGE` in `.env` to pin a specific EE image tag/digest for `web`, `worker`, and both migrators.
- Set `WEB_IMAGE` in `.env` to pin the EE image used by `web`, `worker`, and the Postgres migrator.
- Set `CLICKHOUSE_MIGRATOR_IMAGE` to override the dedicated ClickHouse migrator image.

## BullMQ Dashboard

Expand Down
3 changes: 1 addition & 2 deletions docker-compose/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ services:
node /app/scripts/migrate-postgres.mjs

migrator-clickhouse:
image: ${WEB_IMAGE:-ghcr.io/git-ai-project/git-ai-web-ee:85cd536}
image: ${CLICKHOUSE_MIGRATOR_IMAGE:-ghcr.io/git-ai-project/git-ai-web-ee:7160143-clickhouse-migrator}
restart: "no"
depends_on:
clickhouse:
Expand All @@ -68,7 +68,6 @@ services:
environment:
CLICKHOUSE_HOST: clickhouse
CLICKHOUSE_PORT: 9000
command: ["sh", "/app/scripts/migrate-clickhouse.sh"]

web:
image: ${WEB_IMAGE:-ghcr.io/git-ai-project/git-ai-web-ee:85cd536}
Expand Down
12 changes: 6 additions & 6 deletions docker-compose/docs/10-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,12 +65,12 @@

## Migrator fails to start

- Confirm `WEB_IMAGE` points to an EE image that includes:
- `/app/scripts/migrate-postgres.mjs`
- `/app/scripts/migrate-clickhouse.sh`
- `/app/migrations/postgres`
- `/app/migrations/clickhouse`
- Re-pull image and restart: `docker compose pull && task up`
- Confirm `WEB_IMAGE` includes `/app/scripts/migrate-postgres.mjs` and
`/app/migrations/postgres`.
- Confirm `CLICKHOUSE_MIGRATOR_IMAGE` points to the dedicated migrator image;
it contains `/app/scripts/migrate-clickhouse.sh` and
`/app/migrations/clickhouse`.
- Re-pull the images and restart: `docker compose pull && task up`

## ClickHouse running out of storage

Expand Down
7 changes: 5 additions & 2 deletions docker-compose/docs/11-upgrades.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,15 @@

## Upgrade Image Tag

1. Set/update `WEB_IMAGE` in `.env`
1. Set/update `WEB_IMAGE` and `CLICKHOUSE_MIGRATOR_IMAGE` in `.env`.
For a commit tag such as `abc1234`, the matching migrator tag is
`abc1234-clickhouse-migrator`.
2. Pull and restart:

```bash
docker compose pull
task up
```

Migration scripts/assets ship in the EE image and are executed by the one-shot migrator services.
Postgres migrations ship in the EE image. ClickHouse migrations ship in the
dedicated ClickHouse migrator image.
4 changes: 3 additions & 1 deletion helm/docs/02-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,6 @@ Managed cloud shortcuts:
- AKS app-routing defaults: `task up -- -f values.azure.yaml`
- Istio mode: `task up -- -f values.istio.yaml`

Migration jobs use the same image configured under `image.repository` / `image.tag`.
The Postgres migration job uses `image.repository` / `image.tag`. The
ClickHouse job uses the smaller image configured under
`migrations.clickhouseImage`.
4 changes: 3 additions & 1 deletion helm/docs/09-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@
## Migrations

Migration jobs run as Helm hooks (`post-install,post-upgrade`).
They use the same EE image configured under `image.repository` / `image.tag` and run migration scripts from `/app/scripts` with SQL in `/app/migrations`.
The Postgres job uses the EE app image configured under `image`. The ClickHouse
job uses the dedicated image configured under `migrations.clickhouseImage`.
Each image contains its migration script and SQL assets.

Useful commands:

Expand Down
9 changes: 6 additions & 3 deletions helm/docs/11-upgrades.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@

## Upgrade Image Tag

1. Update `image.repository` and/or `image.tag` in `generated/values.local.yaml`.
1. Update `image.repository` / `image.tag` and
`migrations.clickhouseImage.repository` / `tag` in
`generated/values.local.yaml`. For an app commit tag such as `abc1234`, the
matching migrator tag is `abc1234-clickhouse-migrator`.
2. Re-run release upgrade:

```bash
Expand All @@ -11,5 +14,5 @@ task wait
task doctor
```

Migration hooks run on `task up` (`post-install,post-upgrade`) and use the same EE image.
Migrations are executed from `/app/scripts` and `/app/migrations` inside that image.
Migration hooks run on `task up` (`post-install,post-upgrade`). Postgres uses
the EE app image; ClickHouse uses the dedicated migrator image.
7 changes: 2 additions & 5 deletions helm/templates/migrator-clickhouse-job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,8 @@ spec:
{{- end }}
containers:
- name: migrator-clickhouse
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- sh
- /app/scripts/migrate-clickhouse.sh
image: "{{ .Values.migrations.clickhouseImage.repository }}:{{ .Values.migrations.clickhouseImage.tag }}"
imagePullPolicy: {{ .Values.migrations.clickhouseImage.pullPolicy }}
env:
- name: CLICKHOUSE_HOST
value: {{ include "gitai.clickhouseServiceName" . | quote }}
Expand Down
4 changes: 4 additions & 0 deletions helm/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,10 @@ clickhouse:

migrations:
enabled: true
clickhouseImage:
repository: ghcr.io/git-ai-project/git-ai-web-ee
tag: "7160143-clickhouse-migrator"
pullPolicy: IfNotPresent
resources:
requests:
cpu: 100m
Expand Down
11 changes: 8 additions & 3 deletions scripts/bump-image-tag.sh
Comment thread
batond marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ usage() {
cat <<EOF
Usage: $(basename "$0") [TAG]

Bump the application image tag across this repository, commit, and push.
Bump the application and ClickHouse migrator image tags across this repository,
commit, and push.

If TAG is provided, use it directly.
If omitted, fetch the latest commit SHA from $REPO origin/$BRANCH.
Expand Down Expand Up @@ -90,15 +91,19 @@ while true; do
done

OLD_TAG=$(sed -n 's/^ tag: "\(.*\)"/\1/p' helm/values.yaml)
if [[ "$OLD_TAG" == "$NEW_TAG" ]]; then
echo "Image tag is already $NEW_TAG — nothing to do."
OLD_MIGRATOR_TAG=$(sed -n 's/^ tag: "\(.*-clickhouse-migrator\)"/\1/p' helm/values.yaml)
NEW_MIGRATOR_TAG="$NEW_TAG-clickhouse-migrator"
if [[ "$OLD_TAG" == "$NEW_TAG" && "$OLD_MIGRATOR_TAG" == "$NEW_MIGRATOR_TAG" ]]; then
echo "Image tags are already $NEW_TAG — nothing to do."
exit 0
fi

echo "Bumping image tag: $OLD_TAG → $NEW_TAG"

sed -i '' "s|tag: \"$OLD_TAG\"|tag: \"$NEW_TAG\"|" helm/values.yaml
sed -i '' -E "s|tag: \"[0-9a-fA-F]{7,40}-clickhouse-migrator\"|tag: \"$NEW_MIGRATOR_TAG\"|" helm/values.yaml
sed -i '' "s|ghcr.io/git-ai-project/git-ai-web-ee:$OLD_TAG|ghcr.io/git-ai-project/git-ai-web-ee:$NEW_TAG|g" docker-compose/docker-compose.yml
sed -i '' -E "s|ghcr.io/git-ai-project/git-ai-web-ee:[0-9a-fA-F]{7,40}-clickhouse-migrator|ghcr.io/git-ai-project/git-ai-web-ee:$NEW_MIGRATOR_TAG|g" docker-compose/docker-compose.yml

echo "Updated files:"
git diff --stat
Expand Down