Stamp 16.9.
commit : 6e4ab1b69197e2756192a1019439aebacdea5497
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 May 2025 16:30:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 May 2025 16:30:08 -0400
M configure
M configure.ac
M meson.build
Last-minute updates for release notes.
commit : ce12d66ce731a863abb489dfb9b3f01a4537ee73
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 May 2025 11:29:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 May 2025 11:29:49 -0400
Security: CVE-2025-4207
M doc/src/sgml/release-16.sgml
With GB18030, prevent SIGSEGV from reading past end of allocation.
commit : d1264948fccef7a7eac549b78d327f4d1640e946
author : Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
With GB18030 as source encoding, applications could crash the server via
SQL functions convert() or convert_from(). Applications themselves
could crash after passing unterminated GB18030 input to libpq functions
PQescapeLiteral(), PQescapeIdentifier(), PQescapeStringConn(), or
PQescapeString(). Extension code could crash by passing unterminated
GB18030 input to jsonapi.h functions. All those functions have been
intended to handle untrusted, unterminated input safely.
A crash required allocating the input such that the last byte of the
allocation was the last byte of a virtual memory page. Some malloc()
implementations take measures against that, making the SIGSEGV hard to
reach. Back-patch to v13 (all supported versions).
Author: Noah Misch <noah@leadboat.com>
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 13
Security: CVE-2025-4207
M src/backend/utils/mb/mbutils.c
M src/common/jsonapi.c
M src/common/wchar.c
M src/include/mb/pg_wchar.h
M src/interfaces/libpq/fe-exec.c
M src/interfaces/libpq/fe-misc.c
M src/test/modules/test_escape/test_escape.c
M src/test/regress/expected/conversion.out
M src/test/regress/sql/conversion.sql
Refactor test_escape.c for additional ways of testing.
commit : f3bb0b2c4a1a09efa18a76a153269d24980163d4
author : Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 5 May 2025 04:52:04 -0700
Start the file with static functions not specific to pe_test_vectors
tests. This way, new tests can use them without disrupting the file's
layout. Change report_result() PQExpBuffer arguments to plain strings.
Back-patch to v13 (all supported versions), for the next commit.
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Backpatch-through: 13
Security: CVE-2025-4207
M src/test/modules/test_escape/test_escape.c
Translation updates
commit : 7648d268176377dfab6dd1e0090dd8c9418154eb
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 May 2025 12:17:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 5 May 2025 12:17:53 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 73452f0d3ca43035a492ff657802cc9060561413
M src/backend/po/de.po
M src/backend/po/ja.po
M src/backend/po/ru.po
M src/backend/po/uk.po
M src/bin/initdb/po/uk.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/uk.po
M src/bin/scripts/po/ru.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/ja.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/uk.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/tcl/po/uk.po
Release notes for 17.5, 16.9, 15.13, 14.18, 13.21.
commit : 828edfa3d1fea6b85b37ebf35fc60d4fa1b657ae
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 May 2025 13:52:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 May 2025 13:52:59 -0400
M doc/src/sgml/release-16.sgml
Fix typos in comments.
commit : c8a393537c39077efa20357fa13d136c643cff5a
author : Etsuro Fujita <efujita@postgresql.org>
date : Sat, 3 May 2025 19:10:02 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Sat, 3 May 2025 19:10:02 +0900
Also adjust the phrasing in the comments.
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Author: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAPmGK17%3DPHSDZ%2B0G6jcj12buyyE1bQQc3sbp1Wxri7tODT-SDw%40mail.gmail.com
Backpatch-through: 15
M src/backend/utils/activity/pgstat_database.c
M src/backend/utils/activity/pgstat_function.c
M src/backend/utils/activity/pgstat_relation.c
M src/backend/utils/activity/pgstat_subscription.c
Handle self-referencing FKs correctly in partitioned tables
commit : 1817d62ecbf55f27101b8ad30e24630c33b38325
author : Álvaro Herrera <alvherre@kurilemu.de>
date : Fri, 2 May 2025 21:25:50 +0200
committer: Álvaro Herrera <alvherre@kurilemu.de>
date : Fri, 2 May 2025 21:25:50 +0200
For self-referencing foreign keys in partitioned tables, we weren't
handling creation of pg_constraint rows during CREATE TABLE PARTITION AS
as well as ALTER TABLE ATTACH PARTITION. This is an old bug -- mostly,
we broke this in 614a406b4ff1 while trying to fix it (so 12.13, 13.9,
14.6 and 15.0 and up all behave incorrectly). This commit reverts part
of that with additional fixes for full correctness, and installs more
tests to verify the parts we broke, not just the catalog contents but
also the user-visible behavior.
Backpatch to all live branches. In branches 13 and 14, commit
46a8c27a7226 changed the behavior during DETACH to drop a FK
constraint rather than trying to repair it, because the complete fix of
repairing catalog constraints was problematic due to lack of previous
fixes. For this reason, the test behavior in those branches is a bit
different. However, as best as I can tell, the fix works correctly
there.
In release notes we have to recommend that all self-referencing foreign
keys on partitioned tables be recreated if partitions have been created
or attached after the FK was created, keeping in mind that violating
rows might already be present on the referencing side.
Reported-by: Guillaume Lelarge <guillaume@lelarge.info>
Reported-by: Matthew Gabeler-Lee <fastcat@gmail.com>
Reported-by: Luca Vallisa <luca.vallisa@gmail.com>
Discussion: https://postgr.es/m/CAECtzeWHCA+6tTcm2Oh2+g7fURUJpLZb-=pRXgeWJ-Pi+VU=_w@mail.gmail.com
Discussion: https://postgr.es/m/18156-a44bc7096f0683e6@postgresql.org
Discussion: https://postgr.es/m/CAAT=myvsiF-Attja5DcWoUWh21R12R-sfXECY2-3ynt8kaOqjw@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/expected/triggers.out
M src/test/regress/sql/foreign_key.sql
Doc: correct spelling of meson switch.
commit : 3faac9d14063676089c7e974df54e4baa31aa658
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 May 2025 15:12:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 2 May 2025 15:12:49 -0400
It's --auto-features not --auto_features.
Reported-by: Egor Chindyaskin <kyzevan23@mail.ru>
Discussion: https://postgr.es/m/172465652540.862882.17808523044292761256@wrigleys.postgresql.org
Discussion: https://postgr.es/m/1979661.1746212726@sss.pgh.pa.us
Backpatch-through: 16
M doc/src/sgml/installation.sgml
Doc: stop implying recommendation of insecure search_path value.
commit : 334e508e17da7067d11fca3053b976d7d6f3fb10
author : Noah Misch <noah@leadboat.com>
date : Thu, 1 May 2025 16:51:59 -0700
committer: Noah Misch <noah@leadboat.com>
date : Thu, 1 May 2025 16:51:59 -0700
SQL "SET search_path = 'pg_catalog, pg_temp'" is silently equivalent to
"SET search_path = pg_temp, pg_catalog, "pg_catalog, pg_temp"" instead
of the intended "SET search_path = pg_catalog, pg_temp". (The intent
was a two-element search path. With the single quotes, it instead
specifies one element with a comma and a space in the middle of the
element.) In addition to the SET statement, this affects SET clauses of
CREATE FUNCTION, ALTER ROLE, and ALTER DATABASE. It does not affect the
set_config() SQL function.
Though the documentation did not show an insecure command, remove single
quotes that could entice a reader to write an insecure command.
Back-patch to v13 (all supported versions).
Reported-by: Sven Klemm <sven@timescale.com>
Author: Sven Klemm <sven@timescale.com>
Backpatch-through: 13
M doc/src/sgml/extend.sgml
doc: Warn that ts_headline() output is not HTML-safe.
commit : e14df1015e2ce6b9997e3b6b6ad04e8043cd0811
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 1 May 2025 11:07:17 +0100
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Thu, 1 May 2025 11:07:17 +0100
Add a documentation warning to ts_headline() pointing out that, when
working with untrusted input documents, the output is not guaranteed
to be safe for direct inclusion in web pages. This is because, while
it does remove some XML tags from the input, it doesn't remove all
HTML markup, and so the result may be unsafe (e.g., it might permit
XSS attacks).
To guard against that, all HTML markup should be removed from the
input, making it plain text, or the output should be passed through an
HTML sanitizer.
In addition, document precisely what the default text search parser
recognises as valid XML tags, since that's what determines which XML
tags ts_headline() will remove.
Reported-by: Richard Neill <richard.neill@telos.digital>
Author: Dean Rasheed <dean.a.rasheed@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 13
M doc/src/sgml/textsearch.sgml
Update time zone data files to tzdata release 2025b.
commit : e076120d99ff24c1e0711fde4877627c966b341e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Apr 2025 11:13:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Apr 2025 11:13:49 -0400
DST law changes in Chile: there is a new time zone America/Coyhaique
for Chile's Aysén Region, to account for it changing to UTC-03
year-round and thus diverging from America/Santiago.
Historical corrections for Iran.
Backpatch-through: 13
M src/timezone/data/tzdata.zi
Fix xmin advancement during fast_forward decoding.
commit : 21a7caeeb948e3bdcf36d7aca785b7fb953c0e3a
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Apr 2025 11:09:27 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 28 Apr 2025 11:09:27 +0530
During logical decoding, we advance catalog_xmin of logical too early in
fast_forward mode, resulting in required catalog data being removed by
vacuum. This mode is normally used to advance the slot without processing
the changes, but we still can't let the slot's xmin to advance to an
incorrect value.
Commit f49a80c481 fixed a similar issue where the logical slot's
catalog_xmin was getting advanced prematurely during non-fast-forward
mode. During xl_running_xacts processing, instead of directly advancing
the slot's xmin to the oldest running xid in the record, it allowed the
xmin to be held back for snapshots that can be used for
not-yet-replayed transactions, as those might consider older txns as
running too. However, it missed the fact that the same problem can happen
during fast_forward mode decoding, as we won't build a base snapshot in
that mode, and the future call to get_changes from the same slot can miss
seeing the required catalog changes leading to incorrect reslts.
This commit allows building the base snapshot even in fast_forward mode to
prevent the early advancement of xmin.
Reported-by: Amit Kapila <amit.kapila16@gmail.com>
Author: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com
Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com
M contrib/test_decoding/expected/oldest_xmin.out
M contrib/test_decoding/specs/oldest_xmin.spec
M src/backend/replication/logical/decode.c
Fix typo in test file name added in commit 4909b38af0.
commit : 9987c94662c25ff84644c7931ef0cdc933d0bb9d
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Apr 2025 12:17:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 25 Apr 2025 12:17:40 +0530
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/CANhcyEXsObdjkjxEnq10aJumDpa5J6aiPzgTh_w4KCWRYHLw6Q@mail.gmail.com
M contrib/test_decoding/Makefile
R100 contrib/test_decoding/expected/invalidation_distrubution.out contrib/test_decoding/expected/invalidation_distribution.out
M contrib/test_decoding/meson.build
R100 contrib/test_decoding/specs/invalidation_distrubution.spec contrib/test_decoding/specs/invalidation_distribution.spec
Avoid possibly-theoretical OOM crash hazard in hash_create().
commit : f45bdb13397fc142bc95513783dca3e0106e57af
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Apr 2025 16:04:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Apr 2025 16:04:42 -0400
One place in hash_create() used DynaHashAlloc() as a convenient
shorthand for MemoryContextAlloc(). That was fine when it was
written, but it stopped being fine when 9c911ec06 changed
DynaHashAlloc() to use MCXT_ALLOC_NO_OOM (mea culpa). Change
the code to call plain MemoryContextAlloc() as intended.
I think that this bug may be unreachable in practice, since we now
always create AllocSets with some space already allocated, so that
an OOM failure here for a non-shared hash table should be impossible
(with a hash table name of reasonable length anyway). And there
aren't enough shared hash tables to make a crash for one of those
probable. Nonetheless it's clearly not operating as designed, so
back-patch to v16 where 9c911ec06 came in.
Reported-by: Maksim Korotkov <m.korotkov@postgrespro.ru>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/219bdccd460510efaccf90b57e5e5ef2@postgrespro.ru
Backpatch-through: 16
M src/backend/utils/hash/dynahash.c
Fix an oversight in 3f28b2fcac.
commit : 0de091a4b585bfb3ffa2dc1f6e7465db1d0b3eba
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 23 Apr 2025 10:35:54 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 23 Apr 2025 10:35:54 +0530
Commit 3f28b2fcac tried to ensure that the replication origin shouldn't be
advanced in case of an ERROR in the apply worker, so that it can request
the same data again after restart. However, it is possible that an ERROR
was caught and handled by a (say PL/pgSQL) function, and the apply worker
continues to apply further changes, in which case, we shouldn't reset the
replication origin.
Ensure to reset the origin only when the apply worker exits after an
ERROR.
Commit 3f28b2fcac added new function geterrlevel, which we removed in HEAD
as part of this commit, but kept it in backbranches to avoid breaking any
applications. A separate case can be made to have such a function even for
HEAD.
Reported-by: Shawn McCoy <shawn.the.mccoy@gmail.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 16, where it was introduced
Discussion: https://postgr.es/m/CALsgZNCGARa2mcYNVTSj9uoPcJo-tPuWUGECReKpNgTpo31_Pw@mail.gmail.com
M src/backend/replication/logical/worker.c
M src/test/subscription/t/100_bugs.pl
Remove assertion based on pending_since in pgstat_report_stat()
commit : e9ab8677bfc581bdfbcc1e5d286d7d6269741b90
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Apr 2025 13:54:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Apr 2025 13:54:56 +0900
This assertion, based on pending_since (timestamp used to prevent stats
reports to be too frequent or should a partial flush happen), is reached
when it is found that no data can be flushed but a previous call of
pgstat_report_stat() determined that some stats data has been found as
in need of a flush. So pending_since is set when some stats data is
pending (in non-force mode) or if report attempts are too frequent, and
reset to 0 once all stats have been flushed.
Since 5cbbe70a9cc6, WAL senders have begun to report their stats on a
periodic basis for IO stats in v16~ and backend stats on HEAD, creating
some friction with the concurrent pgstat_report_stat() calls that can
happen in the context of a WAL sender (shutdown callback doing a final
report or backend-related code paths). This problem is the cause of
spurious failures in the TAP tests.
In theory, this assertion can be also reached in v15, even if that's
very unlikely. For example, a process, say a background worker, could
do periodic and direct stats flushes with concurrent calls of
pgstat_report_stat() that could cause conflicting values of
pending_since. This can be done with WAL or SLRU stats flushes using
pgstat_flush_wal() or pgstat_slru_flush(). HEAD makes this situation
easier to happen with custom cumulative stats.
This commit removes the assertion altogether, per discussion, as it is
more useful to keep the state of things as they are for the WAL sender.
The assertion could use a special state based on for example
am_walsender, but I doubt that this would be meaningful in the long run
based on the other arguments raised while discussing this issue.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/1489124.1744685908@sss.pgh.pa.us
Discussion: https://postgr.es/m/dwrkeszz6czvtkxzr5mqlciy652zau5qqnm3cp5f3p2po74ppk@omg4g3cc6dgq
Backpatch-through: 15
M src/backend/utils/activity/pgstat.c
Doc: reword text explaining the --maintenance-db option
commit : d2b065084a4a7b16ea3df1c9244291d571b609b1
author : David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 14:57:03 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 22 Apr 2025 14:57:03 +1200
The previous text was a little clumsy. Here we improve that.
Author: David Rowley <dgrowleyml@gmail.com>
Reported-by: Noboru Saito <noborusai@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAM3qnJtv5YbjpwDfVOYN2gZ9zGSLFM1UGJgptSXmwfifOZJFQ@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
Test restartpoints in archive recovery.
commit : 4664de1826a8b24baa96502121dcfcccbef06e72
author : Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
v14 commit 1f95181b44c843729caaa688f74babe9403b5850 and its v13
equivalent caused timing-dependent failures in archive recovery, at
restartpoints. The symptom was "invalid magic number 0000 in log
segment X, offset 0", "unexpected pageaddr X in log segment Y, offset 0"
[X < Y], or an assertion failure. Commit
3635a0a35aafd3bfa80b7a809bc6e91ccd36606a and predecessors back-patched
v15 changes to fix that. This test reproduces the problem
probabilistically, typically in less than 1000 iterations of the test.
Hence, buildfarm and CI runs would have surfaced enough failures to get
attention within a day.
Reported-by: Arun Thirupathi <arunth@google.com>
Discussion: https://postgr.es/m/20250306193013.36.nmisch@google.com
Backpatch-through: 13
M src/test/recovery/meson.build
A src/test/recovery/t/045_archive_restartpoint.pl
Avoid ERROR at ON COMMIT DELETE ROWS after relhassubclass=f.
commit : 5905e993599917e9d978f31cdda6f540c3b3e4b2
author : Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 20 Apr 2025 08:28:48 -0700
Commit 7102070329d8147246d2791321f9915c3b5abf31 fixed a similar bug, but
it missed the case of database-wide ANALYZE ("use_own_xacts" mode).
Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 changed consequences
from silent discard of a pg_class stats (relpages et al.) update to
ERROR "tuple to be updated was already modified". Losing a relpages
update of an ON COMMIT DELETE ROWS table was negligible, but a
COMMIT-time error isn't negligible. Back-patch to v13 (all supported
versions).
Reported-by: Richard Guo <guofenglinux@gmail.com
Reported-by: Robins Tharakan <tharakan@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-XwMKMKJ_GT=p3_-_=j9rQSEs1FbDFUnW9zHuKPsPNEQ@mail.gmail.com
Backpatch-through: 13
M src/backend/commands/vacuum.c
A src/test/regress/expected/maintain_every.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/maintain_every.sql
Fix issue with ORDER BY / DISTINCT aggregates and FILTER
commit : 887a23237150be514df30b11084e6738ed5f93e3
author : David Rowley <drowley@postgresql.org>
date : Sun, 20 Apr 2025 22:12:59 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sun, 20 Apr 2025 22:12:59 +1200
1349d2790 added support so that aggregate functions with an ORDER BY or
DISTINCT clause could make use of presorted inputs to avoid an implicit
sort within nodeAgg.c. That commit failed to consider that a FILTER
clause may exist that filters rows before the aggregate function
arguments are evaluated. That can be problematic if an aggregate
argument contains an expression which could error out during evaluation.
It's perfectly valid to want to have a FILTER clause which eliminates
such values, and with the pre-sorted path added in 1349d2790, it was
possible that the planner would produce a plan with a Sort node above
the Aggregate to perform the sort on the aggregate's arguments long before
the Aggregate node would filter out the non-matching values.
Here we fix this by inspecting ORDER BY / DISTINCT aggregate functions
which have a FILTER clause to see if the aggregate's arguments are
anything more complex than a Var or a Const. Evaluating these isn't
going to cause an error. If we find any non-Var, non-Const parameters
then the planner will now opt to perform the sort in the Aggregate node
for these aggregates, i.e. disable the presorted aggregate optimization.
An alternative fix would have been to completely disallow the presorted
optimization for Aggrefs with any FILTER clause, but that wasn't done as
that could cause large performance regressions for queries that see
significant gains from 1349d2790 due to presorted results coming in from
an Index Scan.
Backpatch to 16, where 1349d2790 was introduced
Author: David Rowley <dgrowleyml@gmail.com>
Reported-by: Kaimeh <kkaimeh@gmail.com>
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAK-%2BJz9J%3DQ06-M7cDJoPNeYbz5EZDqkjQbJnmRyQyzkbRGsYkA%40mail.gmail.com
Backpatch-through: 16
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/sqljson.out
M src/test/regress/sql/aggregates.sql
Be more wary of corrupt data in pageinspect's heap_page_items().
commit : 2d33cf7b8b699ea0158e27f3b8182b0731f6273b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Apr 2025 16:37:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 19 Apr 2025 16:37:42 -0400
The original intent in heap_page_items() was to return nulls, not
throw an error or crash, if an item was sufficiently corrupt that
we couldn't safely extract data from it. However, commit d6061f83a
utterly missed that memo, and not only put in an un-length-checked
copy of the tuple's data section, but also managed to break the check
on sane nulls-bitmap length. Either mistake could possibly lead to
a SIGSEGV crash if the tuple is corrupt.
Bug: #18896
Reported-by: Dmitry Kovalenko <d.kovalenko@postgrespro.ru>
Author: Dmitry Kovalenko <d.kovalenko@postgrespro.ru>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18896-add267b8e06663e3@postgresql.org
Backpatch-through: 13
M contrib/pageinspect/heapfuncs.c
Fix pg_dump --clean with partitioned indexes.
commit : a25f21d99adf92492065b2af0f4c27e11b92d966
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Apr 2025 13:31:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Apr 2025 13:31:44 -0400
We'd try to drop the partitions of a partitioned index separately,
which is disallowed by the backend, leading to an error during
restore. While the error is harmless, it causes problems if you
try to use --single-transaction mode.
Fortunately, there seems no need to do a DROP at all, since the
partition will go away silently when we drop either the parent index
or the partition's table. So just make the DROP conditional on not
being a partition.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxF0QSdkjFKF4di-JGWN6CSdQYEAhGPmQJJCdkSZtd=oLg@mail.gmail.com
Backpatch-through: 13
M src/bin/pg_dump/pg_dump.c
Fix failure for generated column with a not-null domain constraint.
commit : f04e0faa3702edcc3bebefcb0a21be53ee442e59
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Apr 2025 12:08:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Apr 2025 12:08:34 -0400
If a GENERATED column is declared to have a domain data type where
the domain's constraints disallow null values, INSERT commands failed
because we built a targetlist that included coercing a null constant
to the domain's type. The failure occurred even when the generated
value would have been perfectly OK. This is adjacent to the issues
fixed in 0da39aa76, but we didn't notice for lack of testing a domain
with such a constraint.
We aren't going to use the result of the targetlist entry for the
generated column --- ExecComputeStoredGenerated will overwrite it.
So it's not really necessary that it have the exact datatype of
the generated column. This patch fixes the problem by changing
the targetlist entry to be a null Const of the domain's base type,
which should be sufficiently legal. (We do have to tweak
ExecCheckPlanOutput to accept the situation, though.)
This has been broken since we implemented generated columns.
However, this patch only applies easily as far back as v14, partly
because I (tgl) only carried 0da39aa76 back that far, but mostly
because v14 significantly refactored the handling of INSERT/UPDATE
targetlists. Given the lack of field complaints and the short
remaining support lifetime of v13, I judge the cost-benefit ratio
not good for devising a version that would work in v13.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxG59tip2+9h=rEv-ykOFjt0cbsPVchhi0RTij8bABBA0Q@mail.gmail.com
Backpatch-through: 14
M src/backend/executor/nodeModifyTable.c
M src/backend/optimizer/prep/preptlist.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Fix GIN's shimTriConsistentFn to not corrupt its input.
commit : 4b65b085aff2796f01aef7df053781ef6ab74752
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Apr 2025 12:27:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 12 Apr 2025 12:27:46 -0400
Commit 0f21db36d made an assumption that GIN triConsistentFns
would not modify their input entryRes[] arrays. But in fact,
the "shim" triConsistentFn that we use for opclasses that don't
supply their own did exactly that, potentially leading to wrong
answers from a GIN index search. Through bad luck, none of the
test cases that we have for such opclasses exposed the bug.
One response to this could be that the assumption of consistency check
functions not modifying entryRes[] arrays is a bad one, but it still
seems reasonable to me. Notably, shimTriConsistentFn is itself
assuming that with respect to the underlying boolean consistentFn,
so it's sure being self-centered in supposing that it gets to do so.
Fortunately, it's quite simple to fix shimTriConsistentFn to restore
the entry-time state of entryRes[], so let's do that instead.
This issue doesn't affect any core GIN opclasses, since they all
supply their own triConsistentFns. It does affect contrib modules
btree_gin, hstore, and intarray.
Along the way, I (tgl) noticed that shimTriConsistentFn failed to
pick up on a "recheck" flag returned by its first call to the boolean
consistentFn. This may be only a latent problem, since it would be
unlikely for a consistentFn to set recheck for the all-false case
and not any other cases. (Indeed, none of our contrib modules do
that.) Nonetheless, it's formally wrong.
Reported-by: Vinod Sridharan <vsridh90@gmail.com>
Author: Vinod Sridharan <vsridh90@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAFMdLD7XzsXfi1+DpTqTgrD8XU0i2C99KuF=5VHLWjx4C1pkcg@mail.gmail.com
Backpatch-through: 13
M contrib/intarray/expected/_int.out
M contrib/intarray/sql/_int.sql
M src/backend/access/gin/ginlogic.c
Fix race with synchronous_standby_names at startup
commit : c922ae2c42a6de6a4f226c112f13c907b46b5cad
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 11 Apr 2025 10:02:17 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 11 Apr 2025 10:02:17 +0900
synchronous_standby_names cannot be reloaded safely by backends, and the
checkpointer is in charge of updating a state in shared memory if the
GUC is enabled in WalSndCtl, to let the backends know if they should
wait or not for a given LSN. This provides a strict control on the
timing of the waiting queues if the GUC is enabled or disabled, then
reloaded. The checkpointer is also in charge of waking up the backends
that could be waiting for a LSN when the GUC is disabled.
This logic had a race condition at startup, where it would be possible
for backends to not wait for a LSN even if synchronous_standby_names is
enabled. This would cause visibility issues with transactions that we
should be waiting for but they were not. The problem lasts until the
checkpointer does its initial update of the shared memory state when it
loads synchronous_standby_names.
In order to take care of this problem, the shared memory state in
WalSndCtl is extended to detect if it has been initialized by the
checkpointer, and not only check if synchronous_standby_names is
defined. In WalSndCtlData, sync_standbys_defined is renamed to
sync_standbys_status, a bits8 able to know about two states:
- If the shared memory state has been initialized. This flag is set by
the checkpointer at startup once, and never removed.
- If synchronous_standby_names is known as defined in the shared memory
state. This is the same as the previous sync_standbys_defined in
WalSndCtl.
This method gives a way for backends to decide what they should do until
the shared memory area is initialized, and they now ultimately fall back
to a check on the GUC value in this case, which is the best thing that
can be done.
Fortunately, SyncRepUpdateSyncStandbysDefined() is called immediately by
the checkpointer when this process starts, so the window is very narrow.
It is possible to enlarge the problematic window by making the
checkpointer wait at the beginning of SyncRepUpdateSyncStandbysDefined()
with a hardcoded sleep for example, and doing so has showed that a 2PC
visibility test is indeed failing. On machines slow enough, this bug
would cause spurious failures.
In 17~, we have looked at the possibility of adding an injection point
to have a reproducible test, but as the problematic window happens at
early startup, we would need to invent a way to make an injection point
optionally persistent across restarts when attached, something that
would be fine for this case as it would involve the checkpointer. This
issue is quite old, and can be reproduced on all the stable branches.
Author: Melnikov Maksim <m.melnikov@postgrespro.ru>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/163fcbec-900b-4b07-beaa-d2ead8634bec@postgrespro.ru
Backpatch-through: 13
M src/backend/replication/syncrep.c
M src/backend/replication/walsender.c
M src/include/replication/walsender_private.h
Doc: remove long-obsolete advice about generated constraint names.
commit : 047495f1a80d506781cb7b16a906937b9c70b06d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Apr 2025 14:49:10 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Apr 2025 14:49:10 -0400
It's been twenty years since we generated constraint names that
look like "$N". So this advice about double-quoting such names
is well past its sell-by date, and now it merely seems confusing.
Reported-by: Yaroslav Saburov <y.saburov@gmail.com>
Author: "David G. Johnston" <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/174393459040.678.17810152410419444783@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/ddl.sgml
Fix data loss in logical replication.
commit : 9a2f8b4f01d5a40b8b080f6b108ae314a6730cec
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 10 Apr 2025 12:44:11 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 10 Apr 2025 12:44:11 +0530
Data loss can happen when the DDLs like ALTER PUBLICATION ... ADD TABLE ...
or ALTER TYPE ... that don't take a strong lock on table happens
concurrently to DMLs on the tables involved in the DDL. This happens
because logical decoding doesn't distribute invalidations to concurrent
transactions and those transactions use stale cache data to decode the
changes. The problem becomes bigger because we keep using the stale cache
even after those in-progress transactions are finished and skip the
changes required to be sent to the client.
This commit fixes the issue by distributing invalidation messages from
catalog-modifying transactions to all concurrent in-progress transactions.
This allows the necessary rebuild of the catalog cache when decoding new
changes after concurrent DDL.
We observed performance regression primarily during frequent execution of
*publication DDL* statements that modify the published tables. The
regression is minor or nearly nonexistent for DDLs that do not affect the
published tables or occur infrequently, making this a worthwhile cost to
resolve a longstanding data loss issue.
An alternative approach considered was to take a strong lock on each
affected table during publication modification. However, this would only
address issues related to publication DDLs (but not the ALTER TYPE ...)
and require locking every relation in the database for publications
created as FOR ALL TABLES, which is impractical.
The bug exists in all supported branches, but we are backpatching till 14.
The fix for 13 requires somewhat bigger changes than this fix, so the fix
for that branch is still under discussion.
Reported-by: hubert depesz lubaczewski <depesz@depesz.com>
Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Tested-by: Benoit Lobréau <benoit.lobreau@dalibo.com>
Backpatch-through: 14
Discussion: https://postgr.es/m/de52b282-1166-1180-45a2-8d8917ca74c6@enterprisedb.com
Discussion: https://postgr.es/m/CAD21AoAenVqiMjpN-PvGHL1N9DWnHSq673bfgr6phmBUzx=kLQ@mail.gmail.com
M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/invalidation_distrubution.out
M contrib/test_decoding/meson.build
A contrib/test_decoding/specs/invalidation_distrubution.spec
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/reorderbuffer.h
Stabilize 035_standby_logical_decoding.pl.
commit : 86392e8827d874dd2adc4393ce7a962513620455
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Apr 2025 09:12:59 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 8 Apr 2025 09:12:59 +0530
Some tests try to invalidate logical slots on the standby server by
running VACUUM on the primary. The problem is that xl_running_xacts was
getting generated and replayed before the VACUUM command, leading to the
advancement of the active slot's catalog_xmin. Due to this, active slots
were not getting invalidated, leading to test failures.
We fix it by skipping the generation of xl_running_xacts for the required
tests with the help of injection points. As the required interface for
injection points was not present in back branches, we fixed the failing
tests in them by disallowing the slot to become active for the required
cases (where rows_removed conflict could be generated).
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 16, where it was introduced
Discussion: https://postgr.es/m/Z6oQXc8LmiTLfwLA@ip-10-97-1-34.eu-west-3.compute.internal
M src/test/recovery/t/035_standby_logical_decoding.pl
Flush the IO statistics of active WAL senders more frequently
commit : e2a82cd23a88c7f10c9546fcc9fff94db25f66a0
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Apr 2025 07:58:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Apr 2025 07:58:50 +0900
WAL senders do not flush their statistics until they exit, limiting the
monitoring possible for live processes. This is penalizing when WAL
senders are running for a long time, like in streaming or logical
replication setups, because it is not possible to know the amount of IO
they generate while running.
This commit makes WAL senders more aggressive with their statistics
flush, using an internal of 1 second, with the flush timing calculated
based on the existing GetCurrentTimestamp() done before the sleeps done
to wait for some activity. Note that the sleep done for logical and
physical WAL senders happens in two different code paths, so the stats
flushes need to happen in these two places.
One test is added for the physical WAL sender case, and one for the
logical WAL sender case. This can be done in a stable fashion by
relying on the WAL generated by the TAP tests in combination with a
stats reset while a server is running, but only on HEAD as WAL data has
been added to pg_stat_io in a051e71e28a1.
This issue exists since a9c70b46dbe and the introduction of pg_stat_io,
so backpatch down to v16.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Discussion: https://postgr.es/m/Z73IsKBceoVd4t55@ip-10-97-1-34.eu-west-3.compute.internal
Backpatch-through: 16
M src/backend/replication/walsender.c
doc: Clarify project naming
commit : 00f0bb4eab015ea52bcd84c415c75eb03e880de7
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Apr 2025 00:03:18 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 7 Apr 2025 00:03:18 +0200
Clarify the project naming in the history section of the docs
to match the recent license preamble changes.
Backpatch to all supported versions.
Author: Dave Page <dpage@pgadmin.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CA+OCxozLzK2+Jc14XZyWXSp6L9Ot+3efwXUE35FJG=fsbib2EA@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/history.sgml
Fix unintentional 'NULL' string literal in pg_upgrade.
commit : 57467ec7b3d3769d6e96c3995ca4060ac0167901
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 6 Apr 2025 09:14:42 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 6 Apr 2025 09:14:42 -0700
Introduced in 2a083ab807.
Note: backport of commit 945126234b, which was missed at the time.
Discussion: https://postgr.es/m/e852442da35b4f31acc600ed98bbee0f12e65e0c.camel@j-davis.com
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Backpatch-through: 16
M src/bin/pg_upgrade/pg_upgrade.c
Fix parse_cte.c's failure to examine sub-WITHs in DML statements.
commit : 1980ec2bc99361bb6486bf1aaa836c95dd5d629e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 15:01:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 15:01:33 -0400
makeDependencyGraphWalker thought that only SelectStmt nodes could
contain a WithClause. Which was true in our original implementation
of WITH, but astonishingly we missed updating this code when we added
the ability to attach WITH to INSERT/UPDATE/DELETE (and later MERGE).
Moreover, since it was coded to deliberately block recursion to a
WithClause, even updating raw_expression_tree_walker didn't save it.
The upshot of this was that we didn't see references to outer CTE
names appearing within an inner WITH, and would neither complain about
disallowed recursion nor account for such references when sorting CTEs
into a usable order. The lack of complaints about this is perhaps not
so surprising, because typical usage of WITH wouldn't hit either case.
Still, it's pretty broken; failing to detect recursion here leads to
assert failures or worse later on.
Fix by factoring out the processing of sub-WITHs into a new function
WalkInnerWith, and invoking that for all the statement types that
can have WITH.
Bug: #18878
Reported-by: Yu Liang <luy70@psu.edu>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18878-a26fa5ab6be2f2cf@postgresql.org
Backpatch-through: 13
M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Avoid double transformation of json_array()'s subquery.
commit : ca54f9b706f66f8f7191551aec8eac8b86b0fb9b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 12:13:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Apr 2025 12:13:35 -0400
transformJsonArrayQueryConstructor() applied transformStmt() to
the same subquery tree twice. While this causes no issue in many
cases, there are some where it causes a coredump, thanks to the
parser's habit of scribbling on its input.
Fix by making a copy before the first transformation (compare
0f43083d1). This is quite brute-force, but then so is the
whole business of transforming the input twice. Per discussion
in the bug thread, this implementation of json_array() parsing
should be replaced completely. But that will take some work
and will surely not be back-patchable, so for the moment let's
take the easy way out.
Oversight in 7081ac46a. Back-patch to v16 where that came in.
Bug: #18877
Reported-by: Yu Liang <luy70@psu.edu>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18877-c3c3ad75845833bb@postgresql.org
Backpatch-through: 16
M src/backend/parser/parse_expr.c
M src/test/regress/expected/sqljson.out
M src/test/regress/sql/sqljson.sql
Repair misbehavior with duplicate entries in FK SET column lists.
commit : fb3a77fd9db03610c5af219d9ddfb499d77df505
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 20:11:48 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Apr 2025 20:11:48 -0400
Since v15 we've had an option to apply a foreign key constraint's
ON DELETE SET DEFAULT or SET NULL action to just some of the
referencing columns. There was not a check for duplicate entries in
the list of columns-to-set, though. That caused a potential memory
stomp in CreateConstraintEntry(), which incautiously assumed that
the list of columns-to-set couldn't be longer than the number of key
columns. Even after fixing that, the case doesn't work because you
get an error like "multiple assignments to same column" from the SQL
command that is generated to do the update.
We could either raise an error for duplicate columns or silently
suppress the dups, and after a bit of thought I chose to do the
latter. This is motivated by the fact that duplicates in the FK
column list are legal, so it's not real clear why duplicates
in the columns-to-set list shouldn't be. Of course there's no
need to actually set the column more than once.
I left in the fix in CreateConstraintEntry() too, just because
it didn't seem like such low-level code ought to be making
assumptions about what it's handed.
Bug: #18879
Reported-by: Yu Liang <luy70@psu.edu>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18879-259fc59d072bd4d7@postgresql.org
Backpatch-through: 15
M src/backend/catalog/pg_constraint.c
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Relax assertion in finding correct GiST parent
commit : 41932139882e20724c6a0a8c0363b26211793c36
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 13:49:00 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Apr 2025 13:49:00 +0300
Commit 28d3c2ddcf introduced an assertion that if the memorized
downlink location in the insertion stack isn't valid, the parent's
LSN should've changed too. Turns out that was too strict. In
gistFindCorrectParent(), if we walk right, we update the parent's
block number and clear its memorized 'downlinkoffnum'. That triggered
the assertion on next call to gistFindCorrectParent(), if the parent
needed to be split too. Relax the assertion, so that it's OK if
downlinkOffnum is InvalidOffsetNumber.
Backpatch to v13-, all supported versions. The assertion was added in
commit 28d3c2ddcf in v12.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://www.postgresql.org/message-id/18396-03cac9beb2f7aac3@postgresql.org
M src/backend/access/gist/gist.c
doc: Clarify the system value for sslrootcert
commit : dda1b06035231e3342fcdb695303c9946989eefb
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 4 Apr 2025 09:47:36 +0200
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 4 Apr 2025 09:47:36 +0200
The documentation for the special value "system" for sslrootcert could
be misinterpreted to mean the default operating system CA store, which
it may be, but it's defined to be the default CA store of the SSL lib
used.
Backpatch down to v16 where support for the system value was added.
Author: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: George MacKerron <george@mackerron.co.uk>
Discussion: https://postgr.es/m/B3CBBAA3-6EA3-4AB7-8619-4BBFAB93DDB4@yesql.se
Backpatch-through: 16
M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml
Fix logical decoding test to correctly check slot removal on standby.
commit : 65e9418abe39486e9a7ee4f23c17b5e9fa16e972
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:32:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:32:46 +0900
The regression test for logical decoding verifies whether a logical slot
is correctly dropped on a standby when its associated database is dropped.
However, the test mistakenly retrieved slot information from the primary
instead of the standby, causing incorrect behavior.
This commit fixes the issue by ensuring the test correctly checks the slot
on the standby.
Back-patch to all supported versions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/1fdfd020-a509-403c-bd8f-a04664aba148@oss.nttdata.com
Backpatch-through: 13
M src/test/recovery/t/010_logical_decoding_timelines.pl
Fix logical decoding regression tests to correctly check slot existence.
commit : bc485fce52ebaca3f2497f1f490dc9d42505946a
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:09:06 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 4 Apr 2025 13:09:06 +0900
The regression tests for logical decoding verify whether a logical slot
exists or has been dropped. Previously, these tests attempted to
retrieve "slot_name" from the result of slot(), but since "slot_name" was
not included in the result, slot()->{'slot_name'} always returned undef,
leading to incorrect behavior.
This commit fixes the issue by checking the "plugin" field in the result
of slot() instead, ensuring the tests properly verify slot existence.
Back-patch to all supported versions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB149667EC4E738769CA80B7EA5F5AE2@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 13
M src/test/recovery/t/006_logical_decoding.pl
M src/test/recovery/t/010_logical_decoding_timelines.pl
Restrict copying of invalidated replication slots.
commit : 87e8599e046f62bc4c1302c89ef58a881622e051
author : Masahiko Sawada <msawada@postgresql.org>
date : Thu, 3 Apr 2025 10:30:05 -0700
committer: Masahiko Sawada <msawada@postgresql.org>
date : Thu, 3 Apr 2025 10:30:05 -0700
Previously, invalidated logical and physical replication slots could
be copied using the pg_copy_logical_replication_slot and
pg_copy_physical_replication_slot functions. Replication slots that
were invalidated for reasons other than WAL removal retained their
restart_lsn. This meant that a new slot copied from an invalidated
slot could have a restart_lsn pointing to a WAL segment that might
have already been removed.
This commit restricts the copying of invalidated replication slots.
Backpatch to v16, where slots could retain their restart_lsn when
invalidated for reasons other than WAL removal.
For v15 and earlier, this check is not required since slots can only
be invalidated due to WAL removal, and existing checks already handle
this issue.
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CANhcyEU65aH0VYnLiu%3DOhNNxhnhNhwcXBeT-jvRe1OiJTo_Ayg%40mail.gmail.com
Backpatch-through: 16
M doc/src/sgml/func.sgml
M src/backend/replication/slotfuncs.c
M src/test/recovery/t/035_standby_logical_decoding.pl
Make dblink interruptible, via new libpqsrv APIs.
commit : 82a8f0f465ea4bb0d2774f9856ecd62067796686
author : Noah Misch <noah@leadboat.com>
date : Mon, 8 Jan 2024 11:39:56 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 8 Jan 2024 11:39:56 -0800
This replaces dblink's blocking libpq calls, allowing cancellation and
allowing DROP DATABASE (of a database not involved in the query). Apart
from explicit dblink_cancel_query() calls, dblink still doesn't cancel
the remote side. The replacement for the blocking calls consists of
new, general-purpose query execution wrappers in the libpqsrv facility.
Out-of-tree extensions should adopt these.
The original commit d3c5f37dd543498cc7c678815d3921823beec9e9 did not
back-patch. Back-patch now to v16-v13, bringing coverage to all supported
versions. This back-patch omits the orignal's refactoring in postgres_fdw.
Discussion: https://postgr.es/m/20231122012945.74@rfd.leadboat.com
M contrib/dblink/dblink.c
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/include/libpq/libpq-be-fe-helpers.h
Remove unnecessary type violation in tsvectorrecv().
commit : 5fb9f967c373457fd0babde6c7467edcc245906a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 16:17:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 16:17:43 -0400
compareentry() is declared to work on WordEntryIN structs, but
tsvectorrecv() is using it in two places to work on WordEntry
structs. This is almost okay, since WordEntry is the first
field of WordEntryIN. But on machines with 8-byte pointers,
WordEntryIN will have a larger alignment spec than WordEntry,
and it's at least theoretically possible that the compiler
could generate code that depends on the larger alignment.
Given the lack of field reports, this may be just a hypothetical bug
that upsets nothing except sanitizer tools. Or it may be real on
certain hardware but nobody's tried to use tsvectorrecv() on such
hardware. In any case we should fix it, and the fix is trivial:
just change compareentry() so that it works on WordEntry without any
mention of WordEntryIN. We can also get rid of the quite-useless
intermediate function WordEntryCMP.
Bug: #18875
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18875-07a29c49c825a608@postgresql.org
Backpatch-through: 13
M src/backend/utils/adt/tsvector.c
Remove HeapBitmapScan's skip_fetch optimization
commit : 980727b84107f88cecfd4b94fe80269d948633c4
author : Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 14:25:17 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 2 Apr 2025 14:25:17 -0400
The optimization does not take the removal of TIDs by a concurrent vacuum into
account. The concurrent vacuum can remove dead TIDs and make pages ALL_VISIBLE
while those dead TIDs are referenced in the bitmap. This can lead to a
skip_fetch scan returning too many tuples.
It likely would be possible to implement this optimization safely, but we
don't have the necessary infrastructure in place. Nor is it clear that it's
worth building that infrastructure, given how limited the skip_fetch
optimization is.
In the backbranches we just disable the optimization by always passing
need_tuples=true to table_beginscan_bm(). We can't perform API/ABI changes in
the backbranches and we want to make the change as minimal as possible.
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reported-By: Konstantin Knizhnik <knizhnik@garret.ru>
Discussion: https://postgr.es/m/CAEze2Wg3gXXZTr6_rwC+s4-o2ZVFB5F985uUSgJTsECx6AmGcQ@mail.gmail.com
Backpatch-through: 13
M src/backend/executor/nodeBitmapHeapscan.c
Need to do CommandCounterIncrement after StoreAttrMissingVal.
commit : 053222a97b1310485851ec8d6212c96632f56ff8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 11:13:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 2 Apr 2025 11:13:01 -0400
Without this, an additional change to the same pg_attribute row
within the same command will fail. This is possible at least with
ALTER TABLE ADD COLUMN on a multiple-inheritance-pathway structure.
(Another potential hazard is that immediately-following operations
might not see the missingval.)
Introduced by 95f650674, which split the former coding that
used a single pg_attribute update to change both atthasdef and
atthasmissing/attmissingval into two updates, but missed that
this should entail two CommandCounterIncrements as well. Like
that fix, back-patch through v13.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/025a3ffa-5eff-4a88-97fb-8f583b015965@gmail.com
Backpatch-through: 13
M src/backend/commands/tablecmds.c
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Fix code comment
commit : 548184dd16d5f817e5d48ffe049acc2fda2fc86a
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Apr 2025 14:34:24 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 2 Apr 2025 14:34:24 +0200
The changes made in commit d2b4b4c2259 contained incorrect comments:
They said that certain forward declarations were necessary to "avoid
including pathnodes.h here", but the file is itself pathnodes.h! So
change the comment to just say it's a forward declaration in one case,
and in the other case we don't need the declaration at all because it
already appeared earlier in the file.
M src/include/nodes/pathnodes.h
Doc: add information about partition locking
commit : 9b34f10ee050a115d2e02c1bec6de1bf31933970
author : David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 14:04:10 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 14:04:10 +1300
The documentation around locking of partitions for the executor startup
phase of run-time partition pruning wasn't clear about which partitions
were being locked. Fix that.
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/CAApHDvp738G75HfkKcfXaf3a8s%3D6mmtOLh46tMD0D2hAo1UCzA%40mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/ddl.sgml
Fix planner's failure to identify multiple hashable ScalarArrayOpExprs
commit : 5a0840b767cd6443e8fd4aa02746ea6ca03c24bc
author : David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 11:57:54 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 2 Apr 2025 11:57:54 +1300
50e17ad28 (v14) and 29f45e299 (v15) made it so the planner could identify
IN and NOT IN clauses which have Const lists as right-hand arguments and
when an appropriate hash function is available for the data types, mark
the ScalarArrayOpExpr as hashable so the executor could execute it more
optimally by building and probing a hash table during expression
evaluation.
These commits both worked correctly when there was only a single
ScalarArrayOpExpr in the given expression being processed by the
planner, but when there were multiple, only the first was checked and any
subsequent ones were not identified, which resulted in less optimal
expression evaluation during query execution for all but the first found
ScalarArrayOpExpr.
Backpatch to 14, where 50e17ad28 was introduced.
Author: David Geier <geidav.pg@gmail.com>
Discussion: https://postgr.es/m/29a76f51-97b0-4c07-87b7-ec8e3b5345c9@gmail.com
Backpatch-through: 14
M src/backend/optimizer/util/clauses.c
Fix detection and handling of strchrnul() for macOS 15.4.
commit : a39eb9c77faaf51f8d02da718546e77e00a7622a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Apr 2025 16:49:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Apr 2025 16:49:51 -0400
As of 15.4, macOS has strchrnul(), but access to it is blocked behind
a check for MACOSX_DEPLOYMENT_TARGET >= 15.4. But our does-it-link
configure check finds it, so we try to use it, and fail with the
present default deployment target (namely 15.0). This accounts for
today's buildfarm failures on indri and sifaka.
This is the identical problem that we faced some years ago when Apple
introduced preadv and pwritev in the same way. We solved that in
commit f014b1b9b by using AC_CHECK_DECLS instead of AC_CHECK_FUNCS
to check the functions' availability. So do the same now for
strchrnul(). Interestingly, we already had a workaround for
"the link check doesn't agree with <string.h>" cases with glibc,
which we no longer need since only the header declaration is being
checked.
Testing this revealed that the meson version of this check has never
worked, because it failed to use "-Werror=unguarded-availability-new".
(Apparently nobody's tried to build with meson on macOS versions that
lack preadv/pwritev as standard.) Adjust that while at it. Also,
we had never put support for "-Werror=unguarded-availability-new"
into v13, but we need that now.
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/385134.1743523038@sss.pgh.pa.us
Backpatch-through: 13
M configure
M configure.ac
M meson.build
M src/include/pg_config.h.in
M src/port/snprintf.c
M src/tools/msvc/Solution.pm
Fix MERGE with DO NOTHING actions into a partitioned table.
commit : 8d4cd3b4a88713bb1083ebee9e6e81bd703f4c3b
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 29 Mar 2025 09:51:23 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sat, 29 Mar 2025 09:51:23 +0000
ExecInitPartitionInfo() duplicates much of the logic in
ExecInitMerge(), except that it failed to handle DO NOTHING
actions. This would cause an "unknown action in MERGE WHEN clause"
error if a MERGE with any DO NOTHING actions attempted to insert into
a partition not already initialised by ExecInitModifyTable().
Bug: #18871
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Discussion: https://postgr.es/m/18871-b44e3c96de3bd2e8%40postgresql.org
Backpatch-through: 15
M src/backend/executor/execPartition.c
M src/backend/executor/nodeModifyTable.c
M src/test/regress/expected/merge.out
M src/test/regress/sql/merge.sql
Fix guc_malloc calls for consistency and OOM checks
commit : 8d48e84c56889ea26ad1446ce9f22d12bfdb0c14
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 27 Mar 2025 22:57:34 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Thu, 27 Mar 2025 22:57:34 +0100
check_createrole_self_grant and check_synchronized_standby_slots
were allocating memory on a LOG elevel without checking if the
allocation succeeded or not, which would have led to a segfault
on allocation failure.
On top of that, a number of callsites were using the ERROR level,
relying on erroring out rather than returning false to allow the
GUC machinery handle it gracefully. Other callsites used WARNING
instead of LOG. While neither being not wrong, this changes all
check_ functions do it consistently with LOG.
init_custom_variable gets a promoted elevel to FATAL to keep
the guc_malloc error handling in line with the rest of the
error handling in that function which already call FATAL. If
we encounter an OOM in this callsite there is no graceful
handling to be had, better to error out hard.
Backpatch the fix to check_createrole_self_grant down to v16
and the fix to check_synchronized_standby_slots down to v17
where they were introduced.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: Nikita <pm91.arapov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Bug: #18845
Discussion: https://postgr.es/m/18845-582c6e10247377ec@postgresql.org
Backpatch-through: 16
M src/backend/commands/user.c
Prevent assertion failure in contrib/pg_freespacemap.
commit : 41ed749e48b0b5f59c116a12241171a48eebe05a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Mar 2025 13:20:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Mar 2025 13:20:23 -0400
Applying pg_freespacemap() to a relation lacking storage (such as a
view) caused an assertion failure, although there was no ill effect
in non-assert builds. Add an error check for that case.
Bug: #18866
Reported-by: Robins Tharakan <tharakan@gmail.com>
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Euler Taveira <euler@eulerto.com>
Discussion: https://postgr.es/m/18866-d68926d0f1c72d44@postgresql.org
Backpatch-through: 13
M contrib/pg_freespacemap/pg_freespacemap.c
doc: Correct description of values used in FSM for indexes
commit : 4fd0156a6de96117d9b852b84ec8a6c49c57a503
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 10:20:47 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Mar 2025 10:20:47 +0900
The implementation of FSM for indexes is simpler than heap, where 0 is
used to track if a page is in-use and (BLCKSZ - 1) if a page is free.
One comment in indexfsm.c and one description in the documentation of
pg_freespacemap were incorrect about that.
Author: Alex Friedman <alexf01@gmail.com>
Discussion: https://postgr.es/m/71eef655-c192-453f-ac45-2772fec2cb04@gmail.com
Backpatch-through: 13
M doc/src/sgml/pgfreespacemap.sgml
M src/backend/storage/freespace/indexfsm.c
Keep the decompressed filter in brin_bloom_union
commit : ebcc799a772ee0097ba86e9ae63481dd44277a77
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 26 Mar 2025 16:50:13 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 26 Mar 2025 16:50:13 +0100
The brin_bloom_union() function combines two BRIN summaries, by merging
one filter into the other. With bloom, we have to decompress the filters
first, but the function failed to update the summary to store the merged
filter. As a consequence, the index may be missing some of the data, and
return false negatives.
This issue exists since BRIN bloom indexes were introduced in Postgres
14, but at that point the union function was called only when two
sessions happened to summarize a range concurrently, which is rare. It
got much easier to hit in 17, as parallel builds use the union function
to merge summaries built by workers.
Fixed by storing a pointer to the decompressed filter, and freeing the
original one. Free the second filter too, if it was decompressed. The
freeing is not strictly necessary, because the union is called in
short-lived contexts, but it's tidy.
Backpatch to 14, where BRIN bloom indexes were introduced.
Reported by Arseniy Mukhin, investigation and fix by me.
Reported-by: Arseniy Mukhin
Discussion: https://postgr.es/m/18855-1cf1c8bcc22150e6%40postgresql.org
Backpatch-through: 14
M src/backend/access/brin/brin_bloom.c
Fix integer-overflow problem in scram_SaltedPassword()
commit : de1484736dbef86a1b0b92544030ae655b1b2af3
author : Richard Guo <rguo@postgresql.org>
date : Wed, 26 Mar 2025 17:46:51 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 26 Mar 2025 17:46:51 +0900
Setting the iteration count for SCRAM secret generation to INT_MAX
will cause an infinite loop in scram_SaltedPassword() due to integer
overflow, as the loop uses the "i <= iterations" comparison. To fix,
use "i < iterations" instead.
Back-patch to v16 where the user-settable GUC scram_iterations has
been added.
Author: Kevin K Biju <kevinkbiju@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAM45KeEMm8hnxdTOxA98qhfZ9CzGDdgy3mxgJmy0c+2WwjA6Zg@mail.gmail.com
M src/common/scram-common.c
Fix rare assertion failure in standby, if primary is restarted
commit : 2f33de3cdbc814bcc4270aafd98880a12d265777
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 23 Mar 2025 20:41:16 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sun, 23 Mar 2025 20:41:16 +0200
During hot standby, ExpireAllKnownAssignedTransactionIds() and
ExpireOldKnownAssignedTransactionIds() functions mark old transactions
as no-longer running, but they failed to update xactCompletionCount
and latestCompletedXid. AFAICS it would not lead to incorrect query
results, because those functions effectively turn in-progress
transactions into aborted transactions and an MVCC snapshot considers
both as "not visible". But it could surprise GetSnapshotDataReuse()
and trigger the "TransactionIdPrecedesOrEquals(TransactionXmin,
RecentXmin))" assertion in it, if the apparent xmin in a backend would
move backwards. We saw this happen when GetCatalogSnapshot() would
reuse an older catalog snapshot, when GetTransactionSnapshot() had
already advanced TransactionXmin.
The bug goes back all the way to commit 623a9ba79b in v14 that
introduced the snapshot reuse mechanism, but it started to happen more
frequently with commit 952365cded6 which removed a
GetTransactionSnapshot() call from backend startup. That made it more
likely for ExpireOldKnownAssignedTransactionIds() to be called between
GetCatalogSnapshot() and the first GetTransactionSnapshot() in a
backend.
Andres Freund first spotted this assertion failure on buildfarm member
'skink'. Reproduction and analysis by Tomas Vondra.
Backpatch-through: 14
Discussion: https://www.postgresql.org/message-id/oey246mcw43cy4qw2hqjmurbd62lfdpcuxyqiu7botx3typpax%40h7o7mfg5zmdj
M src/backend/storage/ipc/procarray.c
Fix plpgsql's handling of simple expressions in scrollable cursors.
commit : a28c1fb619bfbb82e4dc4965502d33f5a77bde52
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 11:30:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Mar 2025 11:30:42 -0400
exec_save_simple_expr did not account for the possibility that
standard_planner would stick a Materialize node atop the plan
of even a simple Result, if CURSOR_OPT_SCROLL is set. This led
to an "unexpected plan node type" error.
This is a very old bug, but it'd only be reached by declaring a
cursor for a "SELECT simple-expression" query and explicitly
marking it scrollable, which is an odd thing to do. So the lack
of prior reports isn't too surprising.
Bug: #18859
Reported-by: Olleg Samoylov <splarv@ya.ru>
Author: Andrei Lepikhov <lepihov@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18859-0d5f28ac99a37059@postgresql.org
Backpatch-through: 13
M src/pl/plpgsql/src/expected/plpgsql_simple.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_simple.sql
doc: Remove incorrect description about dropping replication slots.
commit : 9070e5840aa30c51c0e4040d9e4a9ba1964aee07
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 21 Mar 2025 12:56:39 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 21 Mar 2025 12:56:39 +0900
pg_drop_replication_slot() can drop replication slots created on
a different database than the one where it is executed. This behavior
has been in place since PostgreSQL 9.4, when pg_drop_replication_slot()
was introduced.
However, commit ff539d mistakenly added the following incorrect
description in the documentation:
For logical slots, this must be called when connected to
the same database the slot was created on.
This commit removes that incorrect statement. A similar mistake was
also present in the documentation for the DROP_REPLICATION_SLOT
command, which has now been corrected as well.
Back-patch to all supported versions.
Author: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/OSCPR01MB14966C6BE304B5BB2E58D4009F5DE2@OSCPR01MB14966.jpnprd01.prod.outlook.com
Backpatch-through: 13
M doc/src/sgml/func.sgml
M doc/src/sgml/protocol.sgml
meson: Flush stdout in testwrap
commit : d9bc60705274b27eab68604b9458431100c871b3
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Mar 2025 09:04:10 -0400
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Mar 2025 09:04:10 -0400
Otherwise the progress won't reliably be displayed during a test.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/kx6xu7suexal5vwsxpy7ybgkcznx6hgywbuhkr6qabcwxjqax2@i4pcpk75jvaa
Backpatch-through: 16
M src/tools/testwrap
Fix ARRAY_SUBLINK and ARRAY[] for int2vector and oidvector input.
commit : 0405982c7cbab6e16245df848e9d110bd0284fa0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Mar 2025 16:07:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Mar 2025 16:07:55 -0400
If the given input_type yields valid results from both
get_element_type and get_array_type, initArrayResultAny believed the
former and treated the input as an array type. However this is
inconsistent with what get_promoted_array_type does, leading to
situations where the output of an ARRAY() subquery is labeled with
the wrong type: it's labeled as oidvector[] but is really a 2-D
array of OID. That at least results in strange output, and can
result in crashes if further processing such as unnest() is applied.
AFAIK this is only possible with the int2vector and oidvector
types, which are special-cased to be treated mostly as true arrays
even though they aren't quite.
Fix by switching the logic to match get_promoted_array_type by
testing get_array_type not get_element_type, and remove an Assert
thereby made pointless. (We need not introduce a symmetrical
check for get_element_type in the other if-branch, because
initArrayResultArr will check it.) This restores the behavior
that existed before bac27394a introduced initArrayResultAny:
the output really is int2vector[] or oidvector[].
Comparable confusion exists when an input of an ARRAY[] construct
is int2vector or oidvector: transformArrayExpr decides it's dealing
with a multidimensional array constructor, and we end up with
something that's a multidimensional OID array but is alleged to be
of type oidvector. I have not found a crashing case here, but it's
easy to demonstrate totally-wrong results. Adjust that code so
that what you get is an oidvector[] instead, for consistency with
ARRAY() subqueries. (This change also makes these types work like
domains-over-arrays in this context, which seems correct.)
Bug: #18840
Reported-by: yang lei <ylshiyu@126.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18840-fbc9505f066e50d6@postgresql.org
Backpatch-through: 13
M src/backend/parser/parse_expr.c
M src/backend/utils/adt/arrayfuncs.c
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql
Handle interrupts while waiting on Append's async subplans
commit : 004dbbd72f7505105a10d4e8ccb9a5a5d87125ed
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 12 Mar 2025 20:53:09 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 12 Mar 2025 20:53:09 +0200
We did not wake up on interrupts while waiting on async events on an
async-capable append node. For example, if you tried to cancel the
query, nothing would happen until one of the async subplans becomes
readable. To fix, add WL_LATCH_SET to the WaitEventSet.
Backpatch down to v14 where async Append execution was introduced.
Discussion: https://www.postgresql.org/message-id/37a40570-f558-40d3-b5ea-5c2079b3b30b@iki.fi
M src/backend/executor/nodeAppend.c
Build whole-row Vars the same way during parsing and planning.
commit : fec43428c9662ad0217308529ef6cdd211e8b478
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Mar 2025 11:47:19 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 12 Mar 2025 11:47:19 -0400
makeWholeRowVar() has different rules for constructing a
whole-row Var depending on the kind of RTE it's representing.
This turns out to be problematic because the rewriter and planner
can convert view RTEs and set-returning-function RTEs into
subquery RTEs; so a whole-row Var made during planning might
look different from one made by the parser. In isolation this
doesn't cause any problem, but if a query contains Vars made
both ways for the same varno, there are cross-checks in the
executor that will complain. This manifests for UPDATE, DELETE,
and MERGE queries that use whole-row table references.
To fix, we need makeWholeRowVar() to produce the same result
from an inlined RTE as it would have for the original. For
an inlined view, we can use RangeTblEntry.relid to detect
that this had been a view RTE. For inlined SRFs, make a
data structure definition change akin to commit 47bb9db75,
and say that we won't clear RangeTblEntry.functions until
the end of planning. That allows makeWholeRowVar() to
repeat what it would have done with the unmodified RTE.
Reported-by: Duncan Sands <duncan.sands@deepbluecap.com>
Reported-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Diagnosed-by: Tender Wang <tndrwang@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/3518c50a-ab18-482f-b916-a37263622501@deepbluecap.com
Backpatch-through: 13
M src/backend/nodes/makefuncs.c
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/returning.out
M src/test/regress/sql/returning.sql
BRIN: be more strict about required support procs
commit : e0d8f49a3fee56264fdc817b534305e7a281d4bf
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 11 Mar 2025 12:50:35 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 11 Mar 2025 12:50:35 +0100
With improperly defined operator classes, it's possible to get a
Postgres crash because we'd try to invoke a procedure that doesn't
exist. This is because the code is being a bit too trusting that the
opclass is correctly defined. Add some ereport(ERROR)s for cases where
mandatory support procedures are not defined, transforming the crashes
into errors.
The particular case that was reported is an incomplete opclass in
PostGIS.
Backpatch all the way down to 13.
Reported-by: Tobias Wendorff <tobias.wendorff@tu-dortmund.de>
Diagnosed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Discussion: https://postgr.es/m/fb6d9a35-6c8e-4869-af80-0a4944a793a4@tu-dortmund.de
M src/backend/access/brin/brin_bloom.c
M src/backend/access/brin/brin_inclusion.c
M src/backend/access/brin/brin_minmax_multi.c
Fix a few more redundant calls of GetLatestSnapshot()
commit : 324e0b656b4bc90bf64559cd9610542613211f20
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 18:54:58 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 18:54:58 +0200
Commit 2367503177 fixed this in RelationFindReplTupleByIndex(), but I
missed two other similar cases.
Per report from Ranier Vilela.
Discussion: https://www.postgresql.org/message-id/CAEudQArUT1dE45WN87F-Gb7XMy_hW6x1DFd3sqdhhxP-RMDa0Q@mail.gmail.com
Backpatch-through: 13
M src/backend/executor/execReplication.c
Fix snapshot used in logical replication index lookup
commit : 8171d2dae93d8463f2cbba0ce18e3f5c85cac096
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 17:07:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 10 Mar 2025 17:07:38 +0200
The function calls GetLatestSnapshot() to acquire a fresh snapshot,
makes it active, and was meant to pass it to table_tuple_lock(), but
instead called GetLatestSnapshot() again to acquire yet another
snapshot. It was harmless because the heap AM and all other known
table AMs ignore the 'snapshot' argument anyway, but let's be tidy.
In the long run, this perhaps should be redesigned so that snapshot
was not needed in the first place. The table AM API uses TID +
snapshot as the unique identifier for the row version, which is
questionable when the row came from an index scan with a Dirty
snapshot. You might lock a different row version when you use a
different snapshot in the table_tuple_lock() call (a fresh MVCC
snapshot) than in the index scan (DirtySnapshot). However, in the heap
AM and other AMs where the TID alone identifies the row version, it
doesn't matter. So for now, just fix the obvious albeit harmless bug.
This has been wrong ever since the table AM API was introduced in
commit 5db6df0c01, so backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/83d243d6-ad8d-4307-8b51-2ee5844f6230@iki.fi
Backpatch-through: 13
M src/backend/executor/execReplication.c
Doc: improve description of window function processing.
commit : 778f9dd0d2aa84f5c67f0e257a175c3c99d0dc5e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Mar 2025 10:22:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Mar 2025 10:22:08 -0400
The previous wording talked about a "single pass over the data",
which can be read as promising more than intended (to wit, that only
one WindowAgg plan node will be used). What we promise is only what
the SQL spec requires, namely that the data not get re-sorted between
window functions with compatible PARTITION BY/ORDER BY clauses.
Adjust the wording in hopes of making this clearer.
Reported-by: Christopher Inokuchi <cinokuchi@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CABde6B5va2wMsnM79u_x=n9KUgfKQje_pbLROEBmA9Ru5XWidw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/queries.sgml
Don't try to parallelize array_agg() on an anonymous record type.
commit : a7aa9f21f23ead3e04b073de66fdb56f5eb765a2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Mar 2025 13:11:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Mar 2025 13:11:20 -0400
This doesn't work because record_recv requires the typmod that
identifies the specific record type (in our session) and
array_agg_deserialize has no convenient way to get that information.
The result is an "input of anonymous composite types is not
implemented" error.
We could probably make this work if we had to, but it does not seem
worth the trouble, given that it took this long to get a field report.
Just shut off parallelization, as though record_recv didn't exist.
Oversight in commit 16fd03e95. Back-patch to v16 where that
came in.
Reported-by: Kirill Zdornyy <kirill@dineserve.com>
Diagnosed-by: Richard Guo <guofenglinux@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/atLI5Kce2ie1zcYjU0w_kjtVaxiYbYGTihrkLDmGZQnRDD4pnXukIATaABbnIj9pUnelC4ESvCXMm4HAyHg-v61XABaKpERj0A2IXzJZM7g=@dineserve.com
Backpatch-through: 16
M src/backend/parser/parse_agg.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Clear errno before calling strtol() in spell.c.
commit : 61513da08b169fcc58c78bdf8752a81ea1fc92bd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 Mar 2025 11:24:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 8 Mar 2025 11:24:22 -0500
Per POSIX, a caller of strtol() that wishes to check for errors must
set errno to 0 beforehand. Several places in spell.c neglected that,
so that they risked delivering a false overflow error in case errno
had been ERANGE already. Given the lack of field reports, this case
may be unreachable at present --- but it's surely trouble waiting to
happen, so fix it.
Author: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://postgr.es/m/CA+COZaBhsq6EromFm+knMJfzK6nTpG23zJ+K2=nfUQQXcj_xcQ@mail.gmail.com
Backpatch-through: 13
M src/backend/tsearch/spell.c
Doc: correct aggressive vacuum threshold for multixact members storage
commit : 98aa99b6d68293b500ad3a1042dfa22dd2e96918
author : John Naylor <john.naylor@postgresql.org>
date : Fri, 7 Mar 2025 10:22:56 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Fri, 7 Mar 2025 10:22:56 +0700
The threshold is two billion members, which was interpreted as 2GB
in the documentation. Fix to reflect that each member takes up five
bytes, which translates to about 10GB. This is not exact, because of
page boundaries. While at it, mention the maximum size 20GB.
This has been wrong since commit c552e171d16e, so backpatch to
version 14.
Author: Alex Friedman <alexf01@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CACbFw60UOk6fCC02KsyT3OfU9Dnuq5roYxdw2aFisiN_p1L0bg@mail.gmail.com
Backpatch-through: 14
M doc/src/sgml/maintenance.sgml
Fix some performance issues in GIN query startup.
commit : d52221cf0de411613ab7e0fe8b44e30cef64f1aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Mar 2025 11:54:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 6 Mar 2025 11:54:27 -0500
If a GIN index search had a lot of search keys (for example,
"jsonbcol ?| array[]" with tens of thousands of array elements),
both ginFillScanKey() and startScanKey() took O(N^2) time.
Worse, those loops were uncancelable for lack of CHECK_FOR_INTERRUPTS.
The problem in ginFillScanKey() is the brute-force search key
de-duplication done in ginFillScanEntry(). The most expedient
solution seems to be to just stop trying to de-duplicate once
there are "too many" search keys. We could imagine working harder,
say by using a sort-and-unique algorithm instead of brute force
compare-all-the-keys. But it seems unlikely to be worth the trouble.
There is no correctness issue here, since the code already allowed
duplicate keys if any extra_data is present.
The problem in startScanKey() is the loop that attempts to identify
the first non-required search key. In the submitted test case, that
vainly tests all the key positions, and each iteration takes O(N)
time. One part of that is that it's reinitializing the entryRes[]
array from scratch each time, which is entirely unnecessary given
that the triConsistentFn isn't supposed to scribble on its input.
We can easily adjust the array contents incrementally instead.
The other part of it is that the triConsistentFn may itself take
O(N) time (and does in this test case). This is all extremely
brute force: in simple cases with AND or OR semantics, we could
know without any looping whatever that all or none of the keys
are required. But GIN opclasses don't have any API for exposing
that knowledge, so at least in the short run there is little to
be done about that. Put in a CHECK_FOR_INTERRUPTS so that at
least the loop is cancelable.
These two changes together resolve the primary complaint that
the test query doesn't respond promptly to cancel interrupts.
Also, while they don't completely eliminate the O(N^2) behavior,
they do provide quite a nice speedup for mid-sized examples.
Bug: #18831
Reported-by: Niek <niek.brasa@hitachienergy.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18831-e845ac44ebc5dd36@postgresql.org
Backpatch-through: 13
M src/backend/access/gin/ginget.c
M src/backend/access/gin/ginscan.c
ci: Upgrade FreeBSD image
commit : 49389240bb5e8a3f18a72a41323c0d4c64add38b
author : Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 10:29:03 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 5 Mar 2025 10:29:03 -0500
Upgrade to the current stable version. To avoid needing commits like this in
the future, the CI image name now doesn't contain the OS version number
anymore.
Backpatch to all versions with CI support, we don't want to generate CI images
for multiple FreeBSD versions.
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ3_P4JJ6tWZafjf-_XbHgG6DQGXhH-y6Yp78_bwBJjcww@mail.gmail.com
M .cirrus.tasks.yml
Fix ALTER TABLE error message
commit : 9ea5fe5b6c80f9522d2d819592f796d40294d83e
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Mar 2025 20:07:30 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 4 Mar 2025 20:07:30 +0100
This bogus error message was introduced in 2013 by commit f177cbfe676d,
because of misunderstanding the processCASbits() API; at the time, no
test cases were added that would be affected by this change. Only in
ca87c415e2fc was one added (along with a couple of typos), with an XXX
note that the error message was bogus. Fix the whole, add some test
cases.
Backpatch all the way back.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/202503041822.aobpqke3igvb@alvherre.pgsql
M src/backend/parser/gram.y
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Fix broken handling of domains in atthasmissing logic.
commit : edc3bccd0dc79f874bd10ed6206aed7028df6fcf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 12:43:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Mar 2025 12:43:29 -0500
If a domain type has a default, adding a column of that type (without
any explicit DEFAULT clause) failed to install the domain's default
value in existing rows, instead leaving the new column null. This
is unexpected, and it used to work correctly before v11. The cause
is confusion in the atthasmissing mechanism about which default value
to install: we'd only consider installing an explicitly-specified
default, and then we'd decide that no table rewrite is needed.
To fix, take the responsibility for filling attmissingval out of
StoreAttrDefault, and instead put it into ATExecAddColumn's existing
logic that derives the correct value to fill the new column with.
Also, centralize the logic that determines the need for
default-related table rewriting there, instead of spreading it over
four or five places.
In the back branches, we'll leave the attmissingval-filling code
in StoreAttrDefault even though it's now dead, for fear that some
extension may be depending on that functionality to exist there.
A separate HEAD-only patch will clean up the now-useless code.
Reported-by: jian he <jian.universality@gmail.com>
Author: jian he <jian.universality@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACJufxHFssPvkP1we7WMhPD_1kwgbG52o=kQgL+TnVoX5LOyCQ@mail.gmail.com
Backpatch-through: 13
M src/backend/catalog/heap.c
M src/backend/catalog/pg_attrdef.c
M src/backend/commands/tablecmds.c
M src/include/catalog/heap.h
M src/test/regress/expected/fast_default.out
M src/test/regress/sql/fast_default.sql
Fix pg_strtof() to not crash on NULL endptr.
commit : 5c64ece8aaf3249181e1ae0babf56419f577b8d0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Mar 2025 14:22:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 1 Mar 2025 14:22:56 -0500
We had managed not to notice this simple oversight because none
of our calls exercised the case --- until commit 8f427187d.
That led to pg_dump crashing on any platform that uses this code
(currently Cygwin and Mingw).
Even though there's no immediate bug in the back branches, backpatch,
because a non-POSIX-compliant strtof() substitute is trouble waiting
to happen for extensions or future back-patches.
Diagnosed-by: Alexander Lakhin <exclusion@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/339b3902-4e98-4e31-a744-94e43b7b9292@gmail.com
Backpatch-through: 13
M src/port/strtof.c
pg_upgrade: Fix inconsistency in memory freeing
commit : 9ca2145b00fb5d6272e87f274b0f40df77d81a51
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Feb 2025 10:15:34 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 28 Feb 2025 10:15:34 +0900
The function in charge of freeing the memory from a result created by
PQescapeIdentifier() has to be PQfreemem(), to ensure that both
allocation and free come from libpq.
One spot in pg_upgrade was not respecting that for pg_database's
datlocale (daticulocale in v16) when the collation provider is libc (aka
datlocale/daticulocale is NULL) with an allocation done using
pg_strdup() and a free with PQfreemem(). The code is changed to always
use PQescapeLiteral() when processing the input.
Oversight in 9637badd9f92. This commit is similar to 48e4ae9a0707 and
5b94e2753439.
Author: Michael Paquier <michael@paquier.xyz>
Co-authored-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/Z601RQxTmIUohdkV@paquier.xyz
Backpatch-through: 16
M src/bin/pg_upgrade/pg_upgrade.c
pg_amcheck: Fix inconsistency in memory freeing
commit : 816149dc6bf9dfc1fa6bdb03ef52e1453fef736a
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 14:05:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 27 Feb 2025 14:05:57 +0900
The function in charge of freeing the memory from a result created by
PQescapeIdentifier() has to be PQfreemem(), to ensure that both
allocation and free come from libpq, but one spot in pg_amcheck was
missing that.
Oversight in b859d94c6389.
Author: Ranier Vilela <ranier.vf@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAEudQArD_nKSnYCNUZiPPsJ2tNXgRmLbXGSOrH1vpOF_XtP0Vg@mail.gmail.com
Discussion: https://postgr.es/m/CAEudQArbTWVSbxq608GRmXJjnNSQ0B6R7CSffNnj2hPWMUsRNg@mail.gmail.com
Backpatch-through: 14
M src/bin/pg_amcheck/pg_amcheck.c
Fix assertion when decoding XLOG_PARAMETER_CHANGE on promoted primary.
commit : cc628f66187c56e0bfd1befee298755f7dfbf65e
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 24 Feb 2025 14:03:10 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 24 Feb 2025 14:03:10 -0800
When a standby replays an XLOG_PARAMETER_CHANGE record that lowers
wal_level below logical, we invalidate all logical slots in hot
standby mode. However, if this record was replayed while not in hot
standby mode, logical slots could remain valid even after promotion,
potentially causing an assertion failure during WAL record decoding.
To fix this issue, this commit adds a check for hot_standby status
when restoring a logical replication slot on standbys. This check
ensures that logical slots are invalidated when they become
incompatible due to insufficient wal_level during recovery.
Backpatch to v16 where logical decoding on standby was introduced.
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/CAD21AoABoFwGY_Rh2aeE6tEq3HkJxf0c6UeOXn4VV9v6BAQPSw%40mail.gmail.com
Backpatch-through: 16
M src/backend/replication/slot.c
M src/test/recovery/t/035_standby_logical_decoding.pl
Fix pg_dumpall to cope with dangling OIDs in pg_auth_members.
commit : d850a6600cec7cf382f875af229f9df329fbcc30
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Feb 2025 13:37:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Feb 2025 13:37:12 -0500
There is a race condition between "GRANT role" and "DROP ROLE",
which allows GRANT to install pg_auth_members entries that refer to
dropped roles. (Commit 6566133c5 prevented that for the grantor
field, but not for the granted or grantee roles.) We'll soon fix
that, at least in HEAD, but pg_dumpall needs to cope with the
situation in case of pre-existing inconsistency. As pg_dumpall
stands, it will emit invalid commands like 'GRANT foo TO ""',
which causes pg_upgrade to fail. Fix it to emit warnings and skip
those GRANTs, instead.
There was some discussion of removing the problem by changing
dumpRoleMembership's query to use JOIN not LEFT JOIN, but that
would result in silently ignoring such entries. It seems better
to produce a warning.
Pre-v16 branches already coped with dangling grantor OIDs by simply
omitting the GRANTED BY clause. I left that behavior as-is, although
it's somewhat inconsistent with the behavior of later branches.
Reported-by: Virender Singla <virender.cse@gmail.com>
Discussion: https://postgr.es/m/CAM6Zo8woa62ZFHtMKox6a4jb8qQ=w87R2L0K8347iE-juQL2EA@mail.gmail.com
Backpatch-through: 13
M src/bin/pg_dump/pg_dumpall.c
doc: clarify default checksum behavior in non-master branches
commit : b2b112b326ab6a538ac299d23647d694fa063544
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Feb 2025 13:03:29 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Feb 2025 13:03:29 -0500
Also simplify and correct data checksum wording in master now that it is
the default. PG 13 did not have the awkward wording.
Reported-by: Felix <afripowered@gmail.com>
Reviewed-by: Laurenz Albe
Discussion: https://postgr.es/m/173928241056.707.3989867022954178032@wrigleys.postgresql.org
Backpatch-through: 14
M doc/src/sgml/wal.sgml
Make test portlock logic work with meson
commit : 28fcaa6a2cc0d261ae23d56507ca52f928e54340
author : Andres Freund <andres@anarazel.de>
date : Fri, 21 Feb 2025 11:16:57 -0500
committer: Andres Freund <andres@anarazel.de>
date : Fri, 21 Feb 2025 11:16:57 -0500
Previously the portlock logic, added in 9b4eafcaf41, didn't actually work
properly when the tests were run via meson. 9b4eafcaf41 used the
MESON_BUILD_ROOT environment variable to determine the directory for the port
lock directory, but that's never set for running the tests. That meant that
each test used its own portlock dir, unless the PG_TEST_PORT_DIR environment
variable was set.
Fix the problem by setting top_builddir for the environment. That's also used
for the autoconf/make build.
Backpatch back to 16, where meson support was added.
Reported-by: Zharkov Roman <r.zharkov@postgrespro.ru>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Backpatch-through: 16
M meson.build
M src/test/perl/PostgreSQL/Test/Cluster.pm
Fix cross-version upgrades with XMLSERIALIZE(NO INDENT)
commit : 514d47dfb69caec14b62b294f899b2e953cc2baa
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 20:37:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 20:37:38 +0900
Dumps from versions older than v16 do not know about NO INDENT in a
XMLSERIALIZE() clause. This commit adjusts AdjustUpgrade.pm so as NO
INDENT is discarded in the contents of the new dump adjusted for
comparison when the old version is v15 or older. This should be enough
to make the cross-version upgrade tests pass.
Per report from buildfarm member crake. Oversight in 984410b92326.
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/88b183f1-ebf9-4f51-9144-3704380ccae7@dunslane.net
Backpatch-through: 16
M src/test/perl/PostgreSQL/Test/AdjustUpgrade.pm
Fix a WARNING for data origin discrepancies.
commit : 1c2a2354c3848d9f03dc627270bdca073a5a37bc
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 21 Feb 2025 14:08:27 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 21 Feb 2025 14:08:27 +0530
Previously, a WARNING was issued at the time of defining a subscription
with origin=NONE only when the publisher subscribed to the same table from
other publishers, indicating potential data origination from different
origins. However, the publisher can subscribe to the partition ancestors
or partition children of the table from other publishers, which could also
result in mixed-origin data inclusion. So, give a WARNING in those cases
as well.
Reported-by: Sergey Tatarintsev <s.tatarintsev@postgrespro.ru>
Author: Hou Zhijie <houzj.fnst@fujitsu.com>
Author: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Backpatch-through: 16, where it was introduced
Discussion: https://postgr.es/m/5eda6a9c-63cf-404d-8a49-8dcb116a29f3@postgrespro.ru
M doc/src/sgml/ref/create_subscription.sgml
M src/backend/commands/subscriptioncmds.c
M src/test/subscription/t/030_origin.pl
Add missing deparsing of [NO] IDENT to XMLSERIALIZE()
commit : 0af3ae468180586789060133ceeecbf950fa62ae
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 17:31:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 21 Feb 2025 17:31:02 +0900
NO INDENT is the default, and is added if no explicit indentation
flag was provided with XMLSERIALIZE().
Oversight in 483bdb2afec9.
Author: Jim Jones <jim.jones@uni-muenster.de>
Discussion: https://postgr.es/m/bebd457e-5b43-46b3-8fc6-f6a6509483ba@uni-muenster.de
Backpatch-through: 16
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql
Fix FATAL message for invalid recovery timeline at beginning of recovery
commit : b4969a29657dfeb05b61baaa7d7be72b957ac262
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 10:43:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 10:43:38 +0900
If the requested recovery timeline is not reachable, the logged
checkpoint and timeline should to be the values read from the
backup_label when it is defined. The message generated used the values
from the control file in this case, which is fine when recovering from
the control file without a backup_label, but not if there is a
backup_label.
Issue introduced in ee994272ca50. v15 has introduced xlogrecovery.c and
more simplifications in this area (4a92a1c3d1c3, a27048cbcb58), making
this change a bit simpler to think about, so backpatch only down to this
version.
Author: David Steele <david@pgbackrest.org>
Reviewed-by: Andrey M. Borodin <x4mmm@yandex-team.ru>
Reviewed-by: Benoit Lobréau <benoit.lobreau@dalibo.com>
Discussion: https://postgr.es/m/c3d617d4-1696-4aa7-8a4d-5a7d19cc5618@pgbackrest.org
Backpatch-through: 15
M src/backend/access/transam/xlogrecovery.c
test_escape: Fix output of --help
commit : 9fe0cad166df87b99f71374f7ff7acecf3cc600c
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 09:31:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 20 Feb 2025 09:31:03 +0900
The short option name -f was not listed, only its long option name
--force-unsupported.
Author: Japin Li
Discussion: https://postgr.es/m/ME0P300MB04452BD1FB1B277D4C1C20B9B6C52@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
Backpatch-through: 13
M src/test/modules/test_escape/test_escape.c
tests: BackgroundPsql: Fix potential for lost errors on windows
commit : 4c9397c374d9cf567837682fe2672df421209921
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 10:45:48 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 10:45:48 -0500
This addresses various corner cases in BackgroundPsql:
- On windows stdout and stderr may arrive out of order, leading to errors not
being reported, or attributed to the wrong statement.
To fix, emit the "query-separation banner" on both stdout and stderr and
wait for both.
- Very occasionally the "query-separation banner" would not get removed, because
we waited until the banner arrived, but then replaced the banner plus
newline.
To fix, wait for banner and newline.
- For interactive psql replacing $banner\n is not sufficient, interactive psql
outputs \r\n.
- For interactive psql, where commands are echoed to stdout, the \echo
command, rather than its output, would be matched.
This would sometimes lead to output from the prior query, or wait_connect(),
being returned in the next command.
This also affected wait_connect(), leading to sometimes sending queries to
psql before the connection actually was established.
While debugging these issues I also found that it's hard to know whether a
query separation banner was attributed to the right query. Make that easier by
counting the queries each BackgroundPsql instance has emitted and include the
number in the banner.
Also emit psql stdout/stderr in query() and wait_connect() as Test::More
notes, without that it's rather hard to debug some issues in CI and buildfarm.
As this can cause issues not just to-be-added tests, but also existing ones,
backpatch the fix to all supported versions.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft
Backpatch-through: 13
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
backport: Extend background_psql() to be able to start asynchronously
commit : 6ab58d506bba28fa46a6c993a7e82d6053651546
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 09:41:08 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 09:41:08 -0500
This is a backport of ba08edb0654. Originally it was only applied to master,
but I (Andres) am planning to fix a few bugs in BackgroundPsql, which would be
somewhat harder with the behavioural differences across branches. It's also
generally good for test infrastructure to behave similarly across branches, to
avoid pain during backpatching.
Discussion: https://postgr.es/m/ilcctzb5ju2gulvnadjmhgatnkxsdpac652byb2u3d3wqziyvx@fbuqcglker46
Michael's original commit message:
This commit extends the constructor routine of BackgroundPsql.pm with a
new "wait" parameter. If set to 0, the routine returns without waiting
for psql to start, ready to consume input.
background_psql() in Cluster.pm gains the same "wait" parameter. The
default behavior is still to wait for psql to start. It becomes now
possible to not wait, giving to TAP scripts the possibility to perform
actions between a BackgroundPsql startup and its wait_connect() call.
Author: Jacob Champion
Discussion: https://postgr.es/m/CAOYmi+=60deN20WDyCoHCiecgivJxr=98s7s7-C8SkXwrCfHXg@mail.gmail.com
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/perl/PostgreSQL/Test/Cluster.pm
backport: Improve handling of empty query results in BackgroundPsql
commit : fbfd38662f7273660fe45f59e7a5e4e3ec9f7ada
author : Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 09:39:49 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 19 Feb 2025 09:39:49 -0500
This is a backport of 70291a3c66e. Originally it was only applied to master,
but I (Andres) am planning to fix a few bugs in BackgroundPsql that are harder
to fix in the backbranches with the old behavior. It's also generally good for
test infrastructure to behave similarly across branches, to avoid pain during
backpatching. 70291a3c66e changes the behavior in some cases, but after
discussing it, we are ok with that, it seems unlikely that there are
out-of-core tests relying on the prior behavior.
Discussion: https://postgr.es/m/ilcctzb5ju2gulvnadjmhgatnkxsdpac652byb2u3d3wqziyvx@fbuqcglker46
Michael's original commit message:
A newline is not added at the end of an empty query result, causing the
banner of the hardcoded \echo to not be discarded. This would reflect
on scripts that expect an empty result by showing the "QUERY_SEPARATOR"
in the output returned back to the caller, which was confusing.
This commit changes BackgroundPsql::query() so as empty results are able
to work correctly, making the first newline before the banner optional,
bringing more flexibility.
Note that this change affects 037_invalid_database.pl, where three
queries generated an empty result, with the script relying on the data
from the hardcoded banner to exist in the expected output. These
queries are changed to use query_safe(), leading to a simpler script.
The author has also proposed a test in a different patch where empty
results would exist when using BackgroundPsql.
Author: Jacob Champion
Reviewed-by: Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/CAOYmi+=60deN20WDyCoHCiecgivJxr=98s7s7-C8SkXwrCfHXg@mail.gmail.com
M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/recovery/t/037_invalid_database.pl
Avoid null pointer dereference crash after OOM in Snowball stemmers.
commit : c0c364fa16094d2e6257908634b02ccaac0c8c8d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Feb 2025 21:23:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 18 Feb 2025 21:23:59 -0500
Absorb upstream bug fix (their commit
e322673a841d9abd69994ae8cd20e191090b6ef4), which prevents a null
pointer dereference crash if SN_create_env() gets a malloc failure
at just the wrong point.
Thanks to Maksim Korotkov for discovering the null-pointer
bug and submitting the fix to upstream snowball.
Reported-by: Maksim Korotkov <m.korotkov@postgrespro.ru>
Author: Maksim Korotkov <m.korotkov@postgrespro.ru>
Discussion: https://postgr.es/m/1d1a46-67ab1000-21-80c451@83151435
Backpatch-through: 13
M src/backend/snowball/libstemmer/api.c
Fix unsafe access to BufferDescriptors
commit : 4c6b5dd5f5cfaf42f2370f587df7238d5aba48b8
author : Richard Guo <rguo@postgresql.org>
date : Wed, 19 Feb 2025 11:05:35 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Wed, 19 Feb 2025 11:05:35 +0900
When considering a local buffer, the GetBufferDescriptor() call in
BufferGetLSNAtomic() would be retrieving a shared buffer with a bad
buffer ID. Since the code checks whether the buffer is shared before
using the retrieved BufferDesc, this issue did not lead to any
malfunction. Nonetheless this seems like trouble waiting to happen,
so fix it by ensuring that GetBufferDescriptor() is only called when
we know the buffer is shared.
Author: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Discussion: https://postgr.es/m/CAHewXNku-o46-9cmUgyv6LkSZ25doDrWq32p=oz9kfD8ovVJMg@mail.gmail.com
Backpatch-through: 13
M src/backend/storage/buffer/bufmgr.c
test_escape: Fix handling of short options in getopt_long()
commit : 8d0756250313c42b20f55ac62bece342c82a783d
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 09:45:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 19 Feb 2025 09:45:56 +0900
This addresses two errors in the module, based on the set of options
supported:
- '-c', for --conninfo, was not listed.
- '-f', for --force-unsupported, was not listed.
While on it, these are now listed in an alphabetical order.
Author: Japin Li
Discussion: https://postgr.es/m/ME0P300MB04451FB20CE0346A59C25CADB6FA2@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
Backpatch-through: 13
M src/test/modules/test_escape/test_escape.c