diff --git a/docker-compose/.env.example b/docker-compose/.env.example index 9330169..be97f94 100644 --- a/docker-compose/.env.example +++ b/docker-compose/.env.example @@ -3,6 +3,10 @@ WEB_BASE_URL=http://localhost:3000 BETTER_AUTH_URL=http://localhost:3000 WORKER_PUBLIC_BASE_URL=http://localhost:3000 +# Optional image overrides. Use matching commit tags for the app and migrator. +# WEB_IMAGE=ghcr.io/git-ai-project/git-ai-web-ee:abc1234 +# CLICKHOUSE_MIGRATOR_IMAGE=ghcr.io/git-ai-project/git-ai-web-ee:abc1234-clickhouse-migrator + # Enterprise license key (required) LICENSE_KEY= diff --git a/docker-compose/README.md b/docker-compose/README.md index ad11be2..859c1f7 100644 --- a/docker-compose/README.md +++ b/docker-compose/README.md @@ -18,7 +18,8 @@ BullMQ dashboard is available at `http://localhost:3001`. This package defaults to `ANALYZE_BATCH_PROVIDER=local` (no extra batch env required). It also defaults to `STORAGE_BACKEND=local` (no cloud storage bucket required). -Postgres/ClickHouse migrators run from the same EE image as `web`/`worker` using `/app/scripts` and `/app/migrations`. +The Postgres migrator runs from the EE app image. The ClickHouse migrator uses +a dedicated image containing only ClickHouse and its schema migrations. This package does not configure TLS/reverse proxy automatically. Use your own ingress/reverse proxy in front of `web` as needed. @@ -53,7 +54,8 @@ This removes the book demo / booking-gated onboarding screen for that org. ## Image Override -- Set `WEB_IMAGE` in `.env` to pin a specific EE image tag/digest for `web`, `worker`, and both migrators. +- Set `WEB_IMAGE` in `.env` to pin the EE image used by `web`, `worker`, and the Postgres migrator. +- Set `CLICKHOUSE_MIGRATOR_IMAGE` to override the dedicated ClickHouse migrator image. ## BullMQ Dashboard diff --git a/docker-compose/docker-compose.yml b/docker-compose/docker-compose.yml index a387f72..e62de49 100644 --- a/docker-compose/docker-compose.yml +++ b/docker-compose/docker-compose.yml @@ -58,7 +58,7 @@ services: node /app/scripts/migrate-postgres.mjs migrator-clickhouse: - image: ${WEB_IMAGE:-ghcr.io/git-ai-project/git-ai-web-ee:85cd536} + image: ${CLICKHOUSE_MIGRATOR_IMAGE:-ghcr.io/git-ai-project/git-ai-web-ee:7160143-clickhouse-migrator} restart: "no" depends_on: clickhouse: @@ -68,7 +68,6 @@ services: environment: CLICKHOUSE_HOST: clickhouse CLICKHOUSE_PORT: 9000 - command: ["sh", "/app/scripts/migrate-clickhouse.sh"] web: image: ${WEB_IMAGE:-ghcr.io/git-ai-project/git-ai-web-ee:85cd536} diff --git a/docker-compose/docs/10-troubleshooting.md b/docker-compose/docs/10-troubleshooting.md index 6645c40..5c8e4b0 100644 --- a/docker-compose/docs/10-troubleshooting.md +++ b/docker-compose/docs/10-troubleshooting.md @@ -65,12 +65,12 @@ ## Migrator fails to start -- Confirm `WEB_IMAGE` points to an EE image that includes: - - `/app/scripts/migrate-postgres.mjs` - - `/app/scripts/migrate-clickhouse.sh` - - `/app/migrations/postgres` - - `/app/migrations/clickhouse` -- Re-pull image and restart: `docker compose pull && task up` +- Confirm `WEB_IMAGE` includes `/app/scripts/migrate-postgres.mjs` and + `/app/migrations/postgres`. +- Confirm `CLICKHOUSE_MIGRATOR_IMAGE` points to the dedicated migrator image; + it contains `/app/scripts/migrate-clickhouse.sh` and + `/app/migrations/clickhouse`. +- Re-pull the images and restart: `docker compose pull && task up` ## ClickHouse running out of storage diff --git a/docker-compose/docs/11-upgrades.md b/docker-compose/docs/11-upgrades.md index 7558dff..ef82f1a 100644 --- a/docker-compose/docs/11-upgrades.md +++ b/docker-compose/docs/11-upgrades.md @@ -2,7 +2,9 @@ ## Upgrade Image Tag -1. Set/update `WEB_IMAGE` in `.env` +1. Set/update `WEB_IMAGE` and `CLICKHOUSE_MIGRATOR_IMAGE` in `.env`. + For a commit tag such as `abc1234`, the matching migrator tag is + `abc1234-clickhouse-migrator`. 2. Pull and restart: ```bash @@ -10,4 +12,5 @@ docker compose pull task up ``` -Migration scripts/assets ship in the EE image and are executed by the one-shot migrator services. +Postgres migrations ship in the EE image. ClickHouse migrations ship in the +dedicated ClickHouse migrator image. diff --git a/helm/docs/02-quickstart.md b/helm/docs/02-quickstart.md index eba065f..8ca3d9d 100644 --- a/helm/docs/02-quickstart.md +++ b/helm/docs/02-quickstart.md @@ -42,4 +42,6 @@ Managed cloud shortcuts: - AKS app-routing defaults: `task up -- -f values.azure.yaml` - Istio mode: `task up -- -f values.istio.yaml` -Migration jobs use the same image configured under `image.repository` / `image.tag`. +The Postgres migration job uses `image.repository` / `image.tag`. The +ClickHouse job uses the smaller image configured under +`migrations.clickhouseImage`. diff --git a/helm/docs/09-operations.md b/helm/docs/09-operations.md index adf1744..621dc93 100644 --- a/helm/docs/09-operations.md +++ b/helm/docs/09-operations.md @@ -11,7 +11,9 @@ ## Migrations Migration jobs run as Helm hooks (`post-install,post-upgrade`). -They use the same EE image configured under `image.repository` / `image.tag` and run migration scripts from `/app/scripts` with SQL in `/app/migrations`. +The Postgres job uses the EE app image configured under `image`. The ClickHouse +job uses the dedicated image configured under `migrations.clickhouseImage`. +Each image contains its migration script and SQL assets. Useful commands: diff --git a/helm/docs/11-upgrades.md b/helm/docs/11-upgrades.md index 941f049..3edb9c2 100644 --- a/helm/docs/11-upgrades.md +++ b/helm/docs/11-upgrades.md @@ -2,7 +2,10 @@ ## Upgrade Image Tag -1. Update `image.repository` and/or `image.tag` in `generated/values.local.yaml`. +1. Update `image.repository` / `image.tag` and + `migrations.clickhouseImage.repository` / `tag` in + `generated/values.local.yaml`. For an app commit tag such as `abc1234`, the + matching migrator tag is `abc1234-clickhouse-migrator`. 2. Re-run release upgrade: ```bash @@ -11,5 +14,5 @@ task wait task doctor ``` -Migration hooks run on `task up` (`post-install,post-upgrade`) and use the same EE image. -Migrations are executed from `/app/scripts` and `/app/migrations` inside that image. +Migration hooks run on `task up` (`post-install,post-upgrade`). Postgres uses +the EE app image; ClickHouse uses the dedicated migrator image. diff --git a/helm/templates/migrator-clickhouse-job.yaml b/helm/templates/migrator-clickhouse-job.yaml index 0b62f0f..2d4f1d2 100644 --- a/helm/templates/migrator-clickhouse-job.yaml +++ b/helm/templates/migrator-clickhouse-job.yaml @@ -27,11 +27,8 @@ spec: {{- end }} containers: - name: migrator-clickhouse - image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" - imagePullPolicy: {{ .Values.image.pullPolicy }} - command: - - sh - - /app/scripts/migrate-clickhouse.sh + image: "{{ .Values.migrations.clickhouseImage.repository }}:{{ .Values.migrations.clickhouseImage.tag }}" + imagePullPolicy: {{ .Values.migrations.clickhouseImage.pullPolicy }} env: - name: CLICKHOUSE_HOST value: {{ include "gitai.clickhouseServiceName" . | quote }} diff --git a/helm/values.yaml b/helm/values.yaml index 1628bfe..27286a8 100644 --- a/helm/values.yaml +++ b/helm/values.yaml @@ -138,6 +138,10 @@ clickhouse: migrations: enabled: true + clickhouseImage: + repository: ghcr.io/git-ai-project/git-ai-web-ee + tag: "7160143-clickhouse-migrator" + pullPolicy: IfNotPresent resources: requests: cpu: 100m diff --git a/scripts/bump-image-tag.sh b/scripts/bump-image-tag.sh index 28b46b6..1296a55 100755 --- a/scripts/bump-image-tag.sh +++ b/scripts/bump-image-tag.sh @@ -16,7 +16,8 @@ usage() { cat <