Skip to content

[CI only] Bump PG minors to 16.15 / 17.11 / 18.6 - #8777

Open
ibrahim halatci (ihalatci) wants to merge 5 commits into
release-14.0from
bump-pg-minors-release-14.0-20260815
Open

[CI only] Bump PG minors to 16.15 / 17.11 / 18.6#8777
ibrahim halatci (ihalatci) wants to merge 5 commits into
release-14.0from
bump-pg-minors-release-14.0-20260815

Conversation

@ihalatci

@ihalatci ibrahim halatci (ihalatci) commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Validation PR for the-process #242. Repoints CI at the freshly built dev images carrying the new PostgreSQL minors published 2026-08-14/15.

before after
PG16 16.14 16.15
PG17 17.10 17.11
PG18 18.4 18.6

image_suffix: -dev-b970981

Note: PostgreSQL 18.5 was never released (withdrawn after a regression), so 18.4 -> 18.6 is the correct step.


Fallout from the new minors

Validation surfaced two independent PostgreSQL security changes in this minor set that break Citus.

1. psql COPY FROM STDIN (fixed here)

When a COPY fails early, the new psql silently swallows every line up to and including the next \.. Tests that relied on the old behaviour were adjusted: a terminating \. was added where one was missing, stray statements after a swallowed block were removed, and expected output was trimmed for the lines that no longer execute.

2. output_plugin_libraries (fixed here)

PostgreSQL 14.24, 15.19, 16.15, 17.11 and 18.6 add an output_plugin_libraries GUC. Only the libraries it lists may be used as logical decoding output plugins. It defaults to pgoutput, test_decoding and is superuser-settable, so it takes a reload rather than a restart.

Upgrade note for operators. Citus uses an output plugin named citus for logical replication during non-blocking shard splits. On these minors the following fail until the plugin is allowed:

  • citus_split_shard_by_split_points(..., 'force_logical') and 'auto'
  • create_distributed_table_concurrently()
  • citus_isolate_tenant_to_new_shard(..., 'force_logical' / 'auto')

Shard moves and the rebalancer are unaffected -- they use pgoutput. CDC through the pgoutput shim is unaffected; CDC through wal2json needs the same treatment, exactly as it does on vanilla PostgreSQL.

Remediation, on every node:

ALTER SYSTEM SET output_plugin_libraries = pgoutput, test_decoding, citus;
SELECT pg_reload_conf();

The GUC is GUC_LIST_QUOTE, so quoting the whole list ('pgoutput, test_decoding, citus') stores it as a single name and does not work. Use the bare list above, or quote each element individually.

This PR does not work around the restriction -- allowing a decoder is deliberately an operator action. Instead Citus now fails fast and clearly: a preflight check runs against the source node before any shards, publications or replication slots are created, and raises an error naming the plugin, the node, the current allowlist value, and a ready-to-paste ALTER SYSTEM hint.

Testing

The regression and CDC harnesses start their own clusters, so in CI we are the operator: pg_regress_multi.pl and cdctestlib.pm now write the allowlist themselves, guarded by a probe of postgresql.conf.sample so they stay compatible with older minors where the GUC does not exist.

To keep the un-remediated path covered, a new check-split-output-plugin-denied job deliberately skips that override (via CITUS_TEST_SKIP_OUTPUT_PLUGIN_ALLOWLIST=1) and asserts that a non-blocking split fails with the new error and leaks no shards, publications or replication slots.


Draft: these are -dev- images. Do not merge until the-process #242 is merged and this PR is repointed at the resulting release image tag.

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 77.27273% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.01%. Comparing base (5c292aa) to head (7bd2c56).

❗ There is a different number of reports uploaded between BASE (5c292aa) and HEAD (7bd2c56). Click for more details.

HEAD has 135 uploads less than BASE
Flag BASE (5c292aa) HEAD (7bd2c56)
18_regress_check-query-generator 2 1
17_regress_check-pytest 2 1
17_citus_upgrade 2 1
18_regress_check-pytest 2 1
17_regress_check-query-generator 2 1
18_regress_check-follower-cluster_14.2-1_all 2 1
18_regress_check-follower-cluster_v14.2.0_all 2 1
17_regress_check-columnar-isolation 2 1
18_regress_check-follower-cluster_14.2-1_v14.2.0_workeronly 2 1
18_regress_check-tap 2 1
17_regress_check-add-backup-node 2 1
18_regress_check-enterprise-isolation-logicalrep-3_14.2-1_v14.2.0_workeronly 2 1
17_18_upgrade 2 1
18_regress_check-columnar-isolation_14.2-1_v14.2.0_workeronly 2 1
18_regress_check-follower-cluster_14.2-1_v14.2.0_coordinatoronly 2 1
16_regress_check-follower-cluster 2 1
18_regress_check-enterprise-isolation-logicalrep-3_14.2-1_all 2 1
16_regress_check-enterprise 2 0
16_arbitrary_configs_2 2 0
18_arbitrary_configs_5 1 0
17_regress_check-enterprise-isolation-logicalrep-3 2 1
18_regress_check-vanilla 1 0
18_cdc_installcheck 1 0
16_cdc_installcheck 1 0
16_regress_check-vanilla 1 0
18_regress_check-enterprise-isolation_14.2-1_v14.2.0_workeronly 2 1
18_regress_check-enterprise_14.2-1_all 2 0
18_regress_check-split_14.2-1_v14.2.0_workeronly 2 1
17_arbitrary_configs_2 1 0
18_regress_check-enterprise-isolation_14.2-1_all 1 0
18_regress_check-enterprise-isolation_v14.2.0_all 2 1
17_regress_check-multi-1-create-citus 1 0
18_regress_check-multi-1-create-citus 1 0
17_arbitrary_configs_5 1 0
18_regress_check-columnar-isolation_14.2-1_all 2 1
16_regress_check-tap 2 1
16_regress_check-columnar-isolation 2 1
18_regress_check-multi 1 0
17_regress_check-isolation 2 0
16_arbitrary_configs_0 1 0
18_regress_check-multi_v14.2.0_all 2 1
16_17_upgrade 2 1
18_regress_check-multi-1_14.2-1_v14.2.0_workeronly 2 0
18_regress_check-isolation_v14.2.0_all 2 0
18_regress_check-multi-1_v14.2.0_all 2 0
16_regress_check-isolation 2 0
18_arbitrary_configs_0 2 0
18_regress_check-isolation_14.2-1_v14.2.0_workeronly 1 0
18_regress_check-isolation_14.2-1_v14.2.0_coordinatoronly 2 0
18_regress_check-isolation 2 0
16_regress_check-multi-1 1 0
18_regress_check-multi-1_14.2-1_v14.2.0_coordinatoronly 2 0
18_regress_check-multi-1_14.2-1_all 2 0
18_regress_check-enterprise-isolation-logicalrep-1_14.2-1_v14.2.0_coordinatoronly 2 0
17_regress_check-multi-mx 1 0
16_regress_check-multi-mx 2 0
18_regress_check-multi-mx 2 0
18_regress_check-enterprise-isolation-logicalrep-1 2 0
16_regress_check-enterprise-isolation-logicalrep-3 2 1
18_regress_check-multi-1 2 0
18_regress_check-multi_14.2-1_all 1 0
18_regress_check-operations_14.2-1_all 1 0
16_regress_check-operations 2 0
18_regress_check-operations_14.2-1_v14.2.0_coordinatoronly 2 0
18_regress_check-operations_14.2-1_v14.2.0_workeronly 2 0
18_regress_check-operations 1 0
16_regress_check-multi 2 0
18_regress_check-isolation_14.2-1_all 1 0
16_arbitrary_configs_1 2 0
18_regress_check-multi_14.2-1_v14.2.0_coordinatoronly 2 0
17_arbitrary_configs_1 2 0
17_regress_check-multi 2 0
17_regress_check-multi-1 1 0
18_arbitrary_configs_1 2 0
17_regress_check-tap 2 1
18_regress_check-enterprise_14.2-1_v14.2.0_coordinatoronly 2 0
18_regress_check-enterprise-isolation_14.2-1_v14.2.0_coordinatoronly 2 0
17_arbitrary_configs_3 2 1
18_regress_check-enterprise_v14.2.0_all 2 0
17_regress_check-enterprise 2 0
16_regress_check-multi-1-create-citus 1 0
17_cdc_installcheck 2 0
16_arbitrary_configs_5 1 0
18_regress_check-enterprise-isolation-logicalrep-1_14.2-1_all 2 0
16_regress_check-query-generator 2 1
17_regress_check-enterprise-isolation-logicalrep-1 2 0
16_regress_check-failure 2 1
16_arbitrary_configs_4 2 0
17_regress_check-operations 1 0
18_regress_check-operations_v14.2.0_all 2 0
18_arbitrary_configs_4 2 0
17_arbitrary_configs_0 2 0
17_arbitrary_configs_4 1 0
18_regress_check-enterprise-isolation 1 0
18_regress_check-enterprise-isolation-logicalrep-1_14.2-1_v14.2.0_workeronly 1 0
18_arbitrary_configs_2 1 0
16_regress_check-enterprise-isolation 1 0
Additional details and impacted files
@@               Coverage Diff                @@
##           release-14.0    #8777      +/-   ##
================================================
- Coverage         88.55%   79.01%   -9.54%     
================================================
  Files               289      289              
  Lines             64566    64583      +17     
  Branches           8116     8119       +3     
================================================
- Hits              57177    51031    -6146     
- Misses             5048    10740    +5692     
- Partials           2341     2812     +471     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

PostgreSQL 14.24/15.19/16.15/17.11/18.6 changed psql so that a COPY ... FROM
STDIN which fails before entering copy-data mode now silently swallows every
following line up to and including the next backslash-dot. Previously psql
read those lines as SQL and reported "invalid command \.".

Three shapes are affected:
 * .sql files where a deliberately-failing COPY had no terminator: add one.
 * expected/ files that recorded the now-vanished data / terminator /
   invalid-command lines: drop them.
 * multi_multiuser_copy.sql had a stray bare ";" after the terminator, which
   under the new behaviour re-enters copy mode and swallows the rest of the
   file: remove it.

No test coverage is lost; only psql echo artefacts change.
PostgreSQL 14.24, 15.19, 16.15, 17.11 and 18.6 only allow the logical
decoding output plugins listed in "output_plugin_libraries". Citus uses
the "citus" plugin for non-blocking shard splits, so splits now fail up
front with an actionable error instead of a bare PostgreSQL error after
shards, publications and replication slots have been created.

Also allow the plugin in the regression and CDC test harnesses, and add a
check-split-output-plugin-denied CI job that deliberately leaves it out so
the new error keeps negative coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7c6370b2-06fd-4491-bf92-ecb811d34518
- split_output_plugin_denied: DROP SCHEMA reports the unqualified table
  name, so drop the schema prefix from the expected NOTICE.
- failure_copy_to_reference / failure_copy_on_hash: terminate the psql
  \copy blocks that send no data. Newer psql swallows everything up to
  the next \. when a COPY fails, which silently skipped the following
  test blocks.
- shard_split.c: two blank lines between top-level definitions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7c6370b2-06fd-4491-bf92-ecb811d34518
The PostgreSQL minor bumps have merged in the-process, so switch from the
pull-request dev image tags to the release tags built from the merge commits.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7c6370b2-06fd-4491-bf92-ecb811d34518
@ihalatci
ibrahim halatci (ihalatci) marked this pull request as ready for review August 19, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant