Stamp 17.3.
commit : 3819b30c4c5bbb4e772a6f209fa7bbccccb91a11
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 18:21:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 18:21:12 -0500
M configure
M configure.ac
M meson.build
Last-minute updates for release notes.
commit : 1ea6e890b225fa45911f58602b8c425ae0581897
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 18:16:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 18:16:25 -0500
Security: CVE-2025-1094
M doc/src/sgml/release-17.sgml
Adapt appendPsqlMetaConnect() to the new fmtId() encoding expectations.
commit : 85c1fcc6563843d7ee7ae6f81f29ef813e77a4b6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 16:30:03 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Feb 2025 16:30:03 -0500
We need to tell fmtId() what encoding to assume, but this function
doesn't know that. Fortunately we can fix that without changing the
function's API, because we can just use SQL_ASCII. That's because
database names in connection requests are effectively binary not text:
no encoding-aware processing will happen on them.
This fixes XversionUpgrade failures seen in the buildfarm. The
alternative of having pg_upgrade use setFmtEncoding() is unappetizing,
given that it's connecting to multiple databases that may have
different encodings.
Andres Freund, Noah Misch, Tom Lane
Security: CVE-2025-1094
M src/fe_utils/string_utils.c
Fix type in test_escape test
commit : 05abb0f8303a78921f7113bee1d72586142df99e
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 12:09:23 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 12:09:23 -0500
On machines where char is unsigned this could lead to option parsing looping
endlessly. It's also too narrow a type on other hardware.
Found via Tom Lane's monitoring of the buildfarm.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Security: CVE-2025-1094
Backpatch-through: 13
M src/test/modules/test_escape/test_escape.c
docs: EUC_TW can be up to four bytes wide, not three
commit : dd3c1eb38e9add293f8be59b6aec7574e8584bdb
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
Backpatch-through: 13
Security: CVE-2025-1094
M doc/src/sgml/charset.sgml
Add test of various escape functions
commit : 02d4d87ac20e2698b5375b347c451c55045e388d
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
As highlighted by the prior commit, writing correct escape functions is less
trivial than one might hope.
This test module tries to verify that different escaping functions behave
reasonably. It e.g. tests:
- Invalidly encoded input to an escape function leads to invalidly encoded
output
- Trailing incomplete multi-byte characters are handled sensibly
- Escaped strings are parsed as single statement by psql's parser (which
derives from the backend parser)
There are further tests that would be good to add. But even in the current
state it was rather useful for writing the fix in the prior commit.
Reviewed-by: Noah Misch <noah@leadboat.com>
Backpatch-through: 13
Security: CVE-2025-1094
M src/test/modules/Makefile
M src/test/modules/meson.build
A src/test/modules/test_escape/.gitignore
A src/test/modules/test_escape/Makefile
A src/test/modules/test_escape/meson.build
A src/test/modules/test_escape/t/001_test_escape.pl
A src/test/modules/test_escape/test_escape.c
M src/tools/pgindent/typedefs.list
Fix handling of invalidly encoded data in escaping functions
commit : 43a77239d412db194a69b18b7850580e3b78218f
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
Previously invalidly encoded input to various escaping functions could lead to
the escaped string getting incorrectly parsed by psql. To be safe, escaping
functions need to ensure that neither invalid nor incomplete multi-byte
characters can be used to "escape" from being quoted.
Functions which can report errors now return an error in more cases than
before. Functions that cannot report errors now replace invalid input bytes
with a byte sequence that cannot be used to escape the quotes and that is
guaranteed to error out when a query is sent to the server.
The following functions are fixed by this commit:
- PQescapeLiteral()
- PQescapeIdentifier()
- PQescapeString()
- PQescapeStringConn()
- fmtId()
- appendStringLiteral()
Reported-by: Stephen Fewer <stephen_fewer@rapid7.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 13
Security: CVE-2025-1094
M src/fe_utils/string_utils.c
M src/interfaces/libpq/fe-exec.c
Specify the encoding of input to fmtId()
commit : 61ad93cdd48ecc8c6edf943f4d888a9325b66882
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
This commit adds fmtIdEnc() and fmtQualifiedIdEnc(), which allow to specify
the encoding as an explicit argument. Additionally setFmtEncoding() is
provided, which defines the encoding when no explicit encoding is provided, to
avoid breaking all code using fmtId().
All users of fmtId()/fmtQualifiedId() are either converted to the explicit
version or a call to setFmtEncoding() has been added.
This commit does not yet utilize the now well-defined encoding, that will
happen in a subsequent commit.
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Backpatch-through: 13
Security: CVE-2025-1094
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/psql/command.c
M src/bin/scripts/common.c
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/string_utils.c
M src/include/fe_utils/string_utils.h
Add pg_encoding_set_invalid()
commit : 7d43ca6fe068015b403ffa1762f4df4efdf68b69
author : Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
committer: Andres Freund <andres@anarazel.de>
date : Mon, 10 Feb 2025 10:03:38 -0500
There are cases where we cannot / do not want to error out for invalidly
encoded input. In such cases it can be useful to replace e.g. an incomplete
multi-byte characters with bytes that will trigger an error when getting
validated as part of a larger string.
Unfortunately, until now, for some encoding no such sequence existed. For
those encodings this commit removes one previously accepted input combination
- we consider that to be ok, as the chosen bytes are outside of the valid
ranges for the encodings, we just previously failed to detect that.
As we cannot add a new field to pg_wchar_table without breaking ABI, this is
implemented "in-line" in the newly added function.
Author: Noah Misch <noah@leadboat.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Backpatch-through: 13
Security: CVE-2025-1094
M src/common/wchar.c
M src/include/mb/pg_wchar.h
M src/test/regress/expected/conversion.out
M src/test/regress/regress.c
M src/test/regress/sql/conversion.sql
Translation updates
commit : 439776ba64d9ad8c47293568cd6d2930c6d5218a
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 Feb 2025 15:00:55 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 Feb 2025 15:00:55 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: d20e57fd51cef4a7c57d9802f8719ef6f442c9c3
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ko.po
M src/backend/po/ru.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/ja.po
M src/bin/initdb/po/ko.po
M src/bin/pg_amcheck/po/es.po
M src/bin/pg_amcheck/po/ko.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/ko.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ja.po
M src/bin/pg_basebackup/po/ko.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/ko.po
M src/bin/pg_combinebackup/po/LINGUAS
M src/bin/pg_combinebackup/po/es.po
M src/bin/pg_combinebackup/po/ka.po
A src/bin/pg_combinebackup/po/ko.po
M src/bin/pg_combinebackup/po/ru.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/ko.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/ko.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/ko.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/ja.po
M src/bin/pg_dump/po/ko.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/ko.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/ko.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/ko.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_test_timing/po/ko.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/ko.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/ko.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ko.po
M src/bin/pg_walsummary/po/LINGUAS
M src/bin/pg_walsummary/po/es.po
A src/bin/pg_walsummary/po/ko.po
M src/bin/psql/po/de.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/ka.po
M src/bin/psql/po/ko.po
M src/bin/psql/po/ru.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/ko.po
M src/bin/scripts/po/ru.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/ko.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/ja.po
M src/interfaces/libpq/po/ka.po
M src/interfaces/libpq/po/ko.po
M src/interfaces/libpq/po/ru.po
M src/pl/plperl/po/es.po
M src/pl/plperl/po/ko.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/ko.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/ru.po
M src/pl/tcl/po/es.po
M src/pl/tcl/po/ko.po
M src/pl/tcl/po/ru.po
Release notes for 17.3, 16.7, 15.11, 14.16, 13.19.
commit : 878635daaae548d4e6a58224c0bb04bea1ea2bcb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Feb 2025 13:58:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Feb 2025 13:58:53 -0500
M doc/src/sgml/release-17.sgml
Fix pgbench performance issue induced by commit af35fe501.
commit : e35d396ec3546cdfdb452ac520cc9384df0bab81
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 13:41:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 13:41:42 -0500
Commit af35fe501 caused "pgbench -i" to emit a '\r' character
for each data row loaded (when stderr is a terminal).
That's effectively invisible on-screen, but it causes the
connected terminal program to consume a lot of cycles.
It's even worse if you're connected over ssh, as the data
then has to pass through the ssh tunnel.
Simplest fix is to move the added logic inside the if-tests
that check whether to print a progress line. We could do
it another way that avoids duplicating these few lines,
but on the whole this seems the most transparent way to
write it.
Like the previous commit, back-patch to all supported versions.
Reported-by: Andres Freund <andres@anarazel.de>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/4k4drkh7bcmdezq6zbkhp25mnrzpswqi2o75d5uv2eeg3aq6q7@b7kqdmzzwzgb
Backpatch-through: 13
M src/bin/pgbench/pgbench.c
Doc: clarify behavior of timestamptz input some more.
commit : 480acbbd426863fbaddf0606325e9a49e40bd86c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 12:40:41 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 12:40:41 -0500
Try to make it absolutely plain that we don't retain the
originally specified time zone, only the UTC timestamp.
While at it, make glossary entries for "UTC" and "GMT".
Author: Robert Treat <rob@xzilla.net>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/173796426022.1064.9135167366862649513@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/datatype.sgml
M doc/src/sgml/glossary.sgml
First-draft release notes for 17.3.
commit : 572ec03cbec4690eeb5c1047b378626fe475d218
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 11:59:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 7 Feb 2025 11:59:32 -0500
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
M doc/src/sgml/release-17.sgml
meson: Fix linking using old OpenSSL lib names
commit : 0951d4ee42f13405343373a849b2f143528012ca
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 7 Feb 2025 15:09:13 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 7 Feb 2025 15:09:13 +0100
Before OpenSSL 1.1.0 the legacy names ssleay32 and libeay32 were
still used on Windows, and while we have support for this auto-
conf the meson buildsystem only used the new names on all plat-
forms. This adds support for the old name scheme when building
on Windows.
This patch only applies to 17 and 16 as master no longer support
OpenSSL 1.0.2.
Author: Darek Ślusarczyk <dslusarczyk@splunk.com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAN55FZ1Nk8wqY=mTrN78H026TuGV50h2H6uq1PwxhTauPYi3ug@mail.gmail.com
Backpatch-through: 16
M meson.build
radixtree: Fix crash when non-creator begins iteration over shared tree.
commit : 9af2b3435844526ff503f2509c5246749b9c48b0
author : Masahiko Sawada <msawada@postgresql.org>
date : Thu, 6 Feb 2025 11:35:51 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Thu, 6 Feb 2025 11:35:51 -0800
Previously, if a backend that attached to a shared tree attempted to
start iteration, it resulted in a crash. This commit resolves the
issue by ensuring iter_context is created in RT_ATTACH().
This fix applies only to v17, where radixtree.h was introduced. In the
master branch, this issue was separately resolved by 960013f2a1, which
eliminated the iter_context entirely.
Reviewed-by: John Naylor
Discussion: https://postgr.es/m/CAD21AoBB2U47V=F+wQRB1bERov_of5=BOZGaybjaV8FLQyqG3Q@mail.gmail.com
M src/include/lib/radixtree.h
doc: Update links which returned 404
commit : 32770ea03247bc42b38ccc53b84711e0c13d1498
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 5 Feb 2025 13:58:40 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Wed, 5 Feb 2025 13:58:40 +0100
Two links in the isn module documentation were pointing to tools
which had been moved, resulting in 404 error responses. Update
to the new URLs for the tools. The link to the Sequoia 2000 page
in the history section was no longer working, and since the page
is no longer available online update our link to point at the
paper instead which is on a stable URL.
These links exist in all versions of the documentation so backpatch
to all supported branches.
Author: Daniel Gustafsson <daniel@yesql.se>
Reported-by: charukiewicz@protonmail.com
Discussion: https://postgr.es/m/173679670185.705.8565555804465055355@wrigleys.postgresql.org
Backpatch-through: 13
M doc/src/sgml/biblio.sgml
M doc/src/sgml/history.sgml
M doc/src/sgml/isn.sgml
meson: ci: ensure tests are built before running them
commit : d38fd59d2a16469a9bf5797176094fb418792fae
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:57 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:57 -0500
Meson 1.7 stopped building all the dependencies of tests as part of the
default build target. But it does breaks CI because we only built the default
target before running the test, and ran the tests with --no-rebuild.
The simplest fix would be to remove --no-rebuild from MTEST_ARGS, but it seems
better to explicitly build the test dependencies, so compiler warnings /
errors are visible as part of the build step.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Backpatch: 16-, where meson was added
M .cirrus.tasks.yml
meson: Add missing dependencies for libpq tests
commit : 73ed502ea67779aedd335a8c322c3c9eef3c77b3
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:57 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:57 -0500
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite libpq
This is a bit more complicated than other related fixes, because until now
libpq's tests depended on 'frontend_code', which includes a dependency on
fe_utils, which in turns on libpq. That in turn required
src/interfaces/libpq/test to be entered from the top-level, not from
libpq/meson.build. Because of that the test definitions in libpq/meson.build
could not declare a dependency on the binaries defined in
libpq/test/meson.build.
To fix this, this commit creates frontend_no_fe_utils_code, which allows us to
recurse into libpq/test from withing libpq/meson.build.
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M meson.build
M src/interfaces/libpq/meson.build
M src/interfaces/libpq/test/meson.build
meson: Add missing dependencies to libpq_pipeline test
commit : 09df23cf321e558376f8e24d4e890900098b0c28
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite libpq_pipeline
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M src/test/modules/libpq_pipeline/meson.build
meson: Add test dependencies for test_json_parser
commit : 493398de2c4ec69fefa4a60b5d41d0f8632f521e
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
This is required to ensure correct test dependencies, previously
the test binaries would not necessarily be built.
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && m test --no-rebuild --suite setup --suite test_json_parser
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Author: Peter Eisentraut <peter@eisentraut.org>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 17-, where test_json_parser was added
M src/test/modules/test_json_parser/meson.build
meson: Add pg_regress_ecpg to ecpg test dependencies
commit : f52bc23882779b79f19f0b2a8b0c643c00beb01d
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
This is required to ensure correct test dependencies, previously
pg_regress_ecpg would not necessarily be built.
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite ecpg
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M src/interfaces/ecpg/test/meson.build
meson: Improve dependencies for tmp_install test target
commit : 25cb7769412804fe671d1799009b95e46998213b
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
The missing dependency was, e.g., visible when doing
ninja clean && ninja meson-test-prereq && meson test --no-rebuild --suite setup --suite cube
because meson (and thus its internal meson-test-prereq target) did not know
about a lot of the required targets.
Previously tmp_install did not actually depend on the relevant files being
built. That was mostly not visible, because "meson test" currently uses the
'default' targets as a test's dependency if no dependency is specified.
However, there are plans to narrow that on the meson side, to make it quicker
to run tests.
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M meson.build
meson: Narrow dependencies for 'install-quiet' target
commit : 6a4815c49226b78d84823a1a189ddbc3dfe5bc84
author : Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
committer: Andres Freund <andres@anarazel.de>
date : Tue, 4 Feb 2025 17:45:56 -0500
Previously test dependencies, which are not actually installed, were
unnecessarily built.
Apply this to all branches with meson support, as part of an effort to fix
incorrect test dependencies that can lead to test failures.
Discussion: https://postgr.es/m/CAGECzQSvM3iSDmjF+=Kof5an6jN8UbkP_4cKKT9w6GZavmb5yQ@mail.gmail.com
Discussion: https://postgr.es/m/bdba588f-69a9-4f3e-9b95-62d07210a32e@eisentraut.org
Backpatch: 16-, where meson support was added
M meson.build
pg_controldata: Fix possible errors on corrupted pg_control
commit : 1b8a9533f07a31eae6f93aa99cd996931f1b1911
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 5 Feb 2025 00:15:17 +0200
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Wed, 5 Feb 2025 00:15:17 +0200
Protect against malformed timestamps. Also protect against negative WalSegSz
as it triggers division by zero:
((0x100000000UL) / (WalSegSz)) can turn into zero in
XLogFileName(xlogfilename, ControlFile->checkPointCopy.ThisTimeLineID,
segno, WalSegSz);
because if WalSegSz is -1 then by arithmetic rules in C we get
0x100000000UL / 0xFFFFFFFFFFFFFFFFUL == 0.
Author: Ilyasov Ian <ianilyasov@outlook.com>
Author: Anton Voloshin <a.voloshin@postgrespro.ru>
Backpatch-through: 13
M src/bin/pg_controldata/pg_controldata.c
vacuumdb: Add missing PQfinish() calls to vacuum_one_database().
commit : 8703cdec2055d055583ca17ef2d2b5052b064a15
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 4 Feb 2025 13:26:57 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 4 Feb 2025 13:26:57 -0600
A few of the version checks in vacuum_one_database() do not call
PQfinish() before exiting. This precedent was unintentionally
established in commit 00d1e88d36, and while it's probably not too
problematic, it seems better to properly close the connection.
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/Z6JAwqN1I8ljTuXp%40nathan
Backpatch-through: 13
M src/bin/scripts/vacuumdb.c
Mention jsonlog in description of logging_collector in GUC table
commit : 254834943388faf950acd5b00308177886b31805
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 2 Feb 2025 11:31:26 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 2 Feb 2025 11:31:26 +0900
logging_collector was only mentioning stderr and csvlog, and forgot
about jsonlog. Oversight in dc686681e079, that has added support for
jsonlog in log_destination.
While on it, the description in the GUC table is tweaked to be more
consistent with the documentation and postgresql.conf.sample.
Author: Umar Hayat
Reviewed-by: Ashutosh Bapat, Tom Lane
Discussion: https://postgr.es/m/CAD68Dp1K_vBYqBEukHw=1jF7e76t8aszGZTFL2ugi=H7r=a7MA@mail.gmail.com
Backpatch-through: 13
M src/backend/utils/misc/guc_tables.c
doc: Fix pg_buffercache_evict() title
commit : 0181d34e8ca0f91583658ec6eae44b2057473dab
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 10:44:21 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 31 Jan 2025 10:44:21 +0100
Use <function> rather than <structname> in the <title> to be consistent
with how other functions in this module are documented. Also suffix the
function name with () for consistency.
Backpatch to v17 where pg_buffercache_evict was introduced.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAExHW5uKWH8CuZc9NCb8XxSQc6uzvACV0cScebm54kF763ERAw@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/pgbuffercache.sgml
Fix comment of StrategySyncStart()
commit : f10676dc93fa7226190bb4d7d138bbc11dabe681
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 11:06:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 31 Jan 2025 11:06:07 +0900
The top comment of StrategySyncStart() mentions BufferSync(), but this
function calls BgBufferSync(), not BufferSync().
Oversight in 9cd00c457e6a.
Author: Ashutosh Bapat
Discussion: https://postgr.es/m/CAExHW5tgkjag8i-s=RFrCn5KAWDrC4zEPPkfUKczfccPOxBRQQ@mail.gmail.com
Backpatch-through: 13
M src/backend/storage/buffer/freelist.c
Avoid integer overflow while testing wal_skip_threshold condition.
commit : 1e25cdb214543d8b661cf01bbdb6f8e2b1a0381e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 15:36:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 30 Jan 2025 15:36:07 -0500
smgrDoPendingSyncs had two distinct risks of integer overflow while
deciding which way to ensure durability of a newly-created relation.
First, it accumulated the total size of all forks in a variable of
type BlockNumber (uint32). While we restrict an individual fork's
size to fit in that, I don't believe there's such a restriction on
all of them added together. Second, it proceeded to multiply the
sum by BLCKSZ, which most certainly could overflow a uint32.
(The exact expression is total_blocks * BLCKSZ / 1024. The
compiler might choose to optimize that to total_blocks * 8,
which is not at quite as much risk of overflow as a literal
reading would be, but it's still wrong.)
If an overflow did occur it could lead to a poor choice to
shove a very large relation into WAL instead of fsync'ing it.
This wouldn't be fatal, but it could be inefficient.
Change total_blocks to uint64 which should be plenty, and
rearrange the comparison calculation to be overflow-safe.
I noticed this while looking for ramifications of the proposed
change in MAX_KILOBYTES. It's not entirely clear to me why
wal_skip_threshold is limited to MAX_KILOBYTES in the
first place, but in any case this code is unsafe regardless
of the range of wal_skip_threshold.
Oversight in c6b92041d which introduced wal_skip_threshold,
so back-patch to v13.
Discussion: https://postgr.es/m/1a01f0-66ec2d80-3b-68487680@27595217
Backpatch-through: 13
M src/backend/catalog/storage.c
Handle default NULL insertion a little better.
commit : 6e41e9e5e0deb39420446f10b26c36a84f7248d2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 15:31:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 15:31:55 -0500
If a column is omitted in an INSERT, and there's no column default,
the code in preptlist.c generates a NULL Const to be inserted.
Furthermore, if the column is of a domain type, we wrap the Const
in CoerceToDomain, so as to throw a run-time error if the domain
has a NOT NULL constraint. That's fine as far as it goes, but
there are two problems:
1. We're being sloppy about the type/typmod that the Const is
labeled with. It really should have the domain's base type/typmod,
since it's the input to CoerceToDomain not the output. This can
result in coerce_to_domain inserting a useless length-coercion
function (useless because it's being applied to a null). The
coercion would typically get const-folded away later, but it'd
be better not to create it in the first place.
2. We're not applying expression preprocessing (specifically,
eval_const_expressions) to the resulting expression tree.
The planner's primary expression-preprocessing pass already happened,
so that means the length coercion step and CoerceToDomain node miss
preprocessing altogether.
This is at the least inefficient, since it means the length coercion
and CoerceToDomain will actually be executed for each inserted row,
though they could be const-folded away in most cases. Worse, it
seems possible that missing preprocessing for the length coercion
could result in an invalid plan (for example, due to failing to
perform default-function-argument insertion). I'm not aware of
any live bug of that sort with core datatypes, and it might be
unreachable for extension types as well because of restrictions of
CREATE CAST, but I'm not entirely convinced that it's unreachable.
Hence, it seems worth back-patching the fix (although I only went
back to v14, as the patch doesn't apply cleanly at all in v13).
There are several places in the rewriter that are building null
domain constants the same way as preptlist.c. While those are
before the planner and hence don't have any reachable bug, they're
still applying a length coercion that will be const-folded away
later, uselessly wasting cycles. Hence, make a utility routine
that all of these places can call to do it right.
Making this code more careful about the typmod assigned to the
generated NULL constant has visible but cosmetic effects on some
of the plans shown in contrib/postgres_fdw's regression tests.
Discussion: https://postgr.es/m/1865579.1738113656@sss.pgh.pa.us
Backpatch-through: 14
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/optimizer/prep/preptlist.c
M src/backend/parser/parse_coerce.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/rewrite/rewriteManip.c
M src/include/parser/parse_coerce.h
Avoid breaking SJIS encoding while de-backslashing Windows paths.
commit : 0b713b94b3b07b01ed9cee7c6257799e8d8c6285
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 14:24:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 29 Jan 2025 14:24:36 -0500
When running on Windows, canonicalize_path() converts '\' to '/'
to prevent confusing the Windows command processor. It was
doing that in a non-encoding-aware fashion; but in SJIS there
are valid two-byte characters whose second byte matches '\'.
So encoding corruption ensues if such a character is used in
the path.
We can fairly easily fix this if we know which encoding is
in use, but a lot of our utilities don't have much of a clue
about that. After some discussion we decided we'd settle for
fixing this only in psql, and assuming that its value of
client_encoding matches what the user is typing.
It seems hopeless to get the server to deal with the problematic
characters in database path names, so we'll just declare that
case to be unsupported. That means nothing need be done in
the server, nor in utility programs whose only contact with
file path names is for database paths. But psql frequently
deals with client-side file paths, so it'd be good if it
didn't mess those up.
Bug: #18735
Reported-by: Koichi Suzuki <koichi.suzuki@enterprisedb.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Koichi Suzuki <koichi.suzuki@enterprisedb.com>
Discussion: https://postgr.es/m/18735-4acdb3998bb9f2b1@postgresql.org
Backpatch-through: 13
M src/bin/psql/command.c
M src/bin/psql/copy.c
M src/include/port.h
M src/port/path.c
Revert "Speed up tail processing when hashing aligned C strings, take two"
commit : 6555fe19791476b034179149e33760b07f676c87
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 29 Jan 2025 13:35:43 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 29 Jan 2025 13:35:43 +0700
This reverts commit a365d9e2e8c1ead27203a4431211098292777d3b.
Older versions of Valgrind raise an error, so go back to the bytewise
loop for the final word in the input.
Reported-by: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Discussion: https://postgr.es/m/a3a959f6-14b8-4819-ac04-eaf2aa2e868d@postgrespro.ru
Backpatch-through: 17
M src/include/common/hashfn_unstable.h
At update of non-LP_NORMAL TID, fail instead of corrupting page header.
commit : f4af4515bb5f3591d49bc16b6cb8ddbf66f98374
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
The right mix of DDL and VACUUM could corrupt a catalog page header such
that PageIsVerified() durably fails, requiring a restore from backup.
This affects only catalogs that both have a syscache and have DDL code
that uses syscache tuples to construct updates. One of the test
permutations shows a variant not yet fixed.
This makes !TransactionIdIsValid(TM_FailureData.xmax) possible with
TM_Deleted. I think core and PGXN are indifferent to that.
Per bug #17821 from Alexander Lakhin. Back-patch to v13 (all supported
versions). The test case is v17+, since it uses INJECTION_POINT.
Discussion: https://postgr.es/m/17821-dd8c334263399284@postgresql.org
M src/backend/access/heap/heapam.c
M src/backend/utils/cache/inval.c
M src/include/access/tableam.h
M src/test/modules/injection_points/Makefile
A src/test/modules/injection_points/expected/syscache-update-pruned.out
A src/test/modules/injection_points/expected/syscache-update-pruned_1.out
M src/test/modules/injection_points/injection_points–1.0.sql
M src/test/modules/injection_points/meson.build
A src/test/modules/injection_points/regress_injection.c
A src/test/modules/injection_points/specs/syscache-update-pruned.spec
Merge copies of converting an XID to a FullTransactionId.
commit : 1587f7b9fc1a16df8b01b4f5f9e3c949325160b2
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
Assume twophase.c is the performance-sensitive caller, and preserve its
choice of unlikely() branch hint. Add some retrospective rationale for
that choice. Back-patch to v17, for the next commit to use it.
Reviewed (in earlier versions) by Michael Paquier.
Discussion: https://postgr.es/m/17821-dd8c334263399284@postgresql.org
Discussion: https://postgr.es/m/20250116010051.f3.nmisch@google.com
M contrib/amcheck/verify_heapam.c
M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlogreader.c
M src/backend/utils/adt/xid8funcs.c
M src/include/access/transam.h
Disable runningcheck for src/test/modules/injection_points/specs.
commit : eca456edc1caf4e93b5d4480b201cd091acede1d
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
Directory "injection_points" has specified NO_INSTALLCHECK since before
commit c35f419d6efbdf1a050250d84b687e6705917711 added the specs, but
that commit neglected to disable the corresponding meson runningcheck.
The alternative would be to enable "make installcheck" for ISOLATION,
but the GNU make build system lacks a concept of setting NO_INSTALLCHECK
for REGRESS without also setting it for ISOLATION. Back-patch to v17,
where that commit first appeared, to avoid surprises when back-patching
additional specs.
Discussion: https://postgr.es/m/17821-dd8c334263399284@postgresql.org
M src/test/modules/injection_points/meson.build
Test ECPG decadd(), decdiv(), decmul(), and decsub() for risnull() input.
commit : 3637905c3cdfea95feda901ce570a077964c8d94
author : Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 25 Jan 2025 11:28:14 -0800
Since commit 757fb0e5a9a61ac8d3a67e334faeea6dc0084b3f, these
Informix-compat functions return 0 without changing the output
parameter. Initialize the output parameter before the test call, making
that obvious. Before this, the expected test output has been depending
on freed stack memory. "gcc -ftrivial-auto-var-init=pattern" revealed
that. Back-patch to v13 (all supported versions).
Discussion: https://postgr.es/m/20250106192748.cf.nmisch@google.com
M src/interfaces/ecpg/test/compat_informix/dec_test.pgc
M src/interfaces/ecpg/test/expected/compat_informix-dec_test.c
M src/interfaces/ecpg/test/expected/compat_informix-dec_test.stdout
Doc: recommend "psql -X" for restoring pg_dump scripts.
commit : 76601c0c885aca203966d8f6f6d6683726a0823b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Jan 2025 12:42:05 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 25 Jan 2025 12:42:05 -0500
This practice avoids possible problems caused by non-default psql
options, such as disabling AUTOCOMMIT.
Author: Shinya Kato <Shinya11.Kato@oss.nttdata.com>
Reviewed-by: Robert Treat <rob@xzilla.net>
Discussion: https://postgr.es/m/96ff23a5d858ff72ca8e823a014d16fe@oss.nttdata.com
Backpatch-through: 13
M doc/src/sgml/backup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
Use the correct sizeof() in BufFileLoadBuffer
commit : 50cd651254354397ca2f2b795dd6aac300826076
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 25 Jan 2025 00:36:48 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 25 Jan 2025 00:36:48 +0100
The sizeof() call should reference buffer.data, because that's the
buffer we're reading data into, not the whole PGAlignedBuffer union.
This was introduced by 44cac93464, which replaced the simple buffer
with a PGAlignedBuffer field.
It's benign, because the buffer is the largest field of the union, so
the sizes are the same. But it's easy to trip over this in a patch, so
fix and backpatch. Commit 44cac93464 went into 12, but that's EOL.
Backpatch-through: 13
Discussion: https://postgr.es/m/928bdab1-6567-449f-98c4-339cd2203b87@vondra.me
M src/backend/storage/file/buffile.c
meson: Fix sepgsql installation
commit : 24c5b73eb6ec744b07bb91e6865cd461a3a17f42
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 10:26:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 24 Jan 2025 10:26:12 +0100
The sepgsql.sql file should be installed under share/contrib/, not
share/extension/, since it is not an extension. This makes it match
what make install does.
Discussion: https://www.postgresql.org/message-id/flat/651a5baf-5c45-4a5a-a202-0c8453a4ebf8@eisentraut.org
M contrib/sepgsql/meson.build
Don't ask for bug reports about pthread_is_threaded_np() != 0.
commit : 6c9a83f216a0d6264728f5d372e4ab10cfb60804
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 14:23:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jan 2025 14:23:04 -0500
We thought that this condition was unreachable in ExitPostmaster,
but actually it's possible if you have both a misconfigured locale
setting and some other mistake that causes PostmasterMain to bail
out before reaching its own check of pthread_is_threaded_np().
Given the lack of other reports, let's not ask for bug reports if
this occurs; instead just give the same hint as in PostmasterMain.
Bug: #18783
Reported-by: anani191181515@gmail.com
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/18783-d1873b95a59b9103@postgresql.org
Discussion: https://postgr.es/m/206317.1737656533@sss.pgh.pa.us
Backpatch-through: 13
M src/backend/postmaster/postmaster.c
Repair incorrect handling of AfterTriggerSharedData.ats_modifiedcols.
commit : 2b72fed2dd741914064422ea510088563e413c2f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 22 Jan 2025 11:58:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 22 Jan 2025 11:58:20 -0500
This patch fixes two distinct errors that both ultimately trace
to commit 71d60e2aa, which added the ats_modifiedcols field.
The more severe error is that ats_modifiedcols wasn't accounted for
in afterTriggerAddEvent's scanning loop that looks for a pre-existing
duplicate AfterTriggerSharedData. Thus, a new event could be
incorrectly matched to an AfterTriggerSharedData that has a different
value of ats_modifiedcols, resulting in the wrong tg_updatedcols
bitmap getting passed to the trigger whenever it finally gets fired.
We'd not noticed because (a) few triggers consult tg_updatedcols,
and (b) we had no tests exercising a case where such a trigger was
called as an AFTER trigger. In the test case added by this commit,
contrib/lo's trigger fails to remove a large object when expected
because (without this fix) it thinks the LO OID column hasn't changed.
The other problem was introduced by commit ce5aaea8c, which copied the
modified-columns bitmap into trigger-related storage. It made a copy
for every trigger event, whereas what we really want is to make a new
copy only when we make a new AfterTriggerSharedData entry. (We could
imagine adding extra logic to reduce the number of bitmap copies still
more, but it doesn't look worthwhile at the moment.) In a simple test
of an UPDATE of 10000000 rows with a single AFTER trigger, this thinko
roughly tripled the amount of memory consumed by the pending-triggers
data structures, from 160446744 to 480443440 bytes.
Fixing the first problem requires introducing a bms_equal() call into
afterTriggerAddEvent's scanning loop, which is slightly annoying from
a speed perspective. However, getting rid of the excessive bms_copy()
calls from the second problem balances that out; overall speed of
trigger operations is the same or slightly better, in my tests.
Discussion: https://postgr.es/m/3496294.1737501591@sss.pgh.pa.us
Backpatch-through: 13
M contrib/lo/expected/lo.out
M contrib/lo/sql/lo.sql
M src/backend/commands/trigger.c
Reword recent error messages: "should" -> "must"
commit : dbc3499a0bcb544d30ec4f6fe576b28ce0a7951f
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 15:24:49 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 15:24:49 +0100
Most were introduced in the 17 timeframe. The ones in wparser_def.c are
very old.
I also changed "JSON path expression for column \"%s\" should return
single item without wrapper" to "JSON path expression for column \"%s\"
must return single item when no wrapper is requested" to avoid
ambiguity.
Backpatch to 17.
Crickets: https://postgr.es/m/202501131819.26ors7oouafu@alvherre.pgsql
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/sqljson_jsontable.out
M src/test/regress/expected/sqljson_queryfuncs.out
Fix detach of a partition that has a toplevel FK to a partitioned table
commit : 2f30847d1d04bafa51a17886e132892e38df093f
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 14:53:46 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jan 2025 14:53:46 +0100
In common cases, foreign keys are defined on the toplevel partitioned
table; but if instead one is defined on a partition and references a
partitioned table, and the referencing partition is detached, we would
examine the pg_constraint row on the partition being detached, and fail
to realize that the sub-constraints must be left alone. This causes the
ALTER TABLE DETACH process to fail with
ERROR: could not find ON INSERT check triggers of foreign key constraint NNN
This is similar but not quite the same as what was fixed by
53af9491a043. This bug doesn't affect branches earlier than 15, because
the detach procedure was different there, so we only backpatch down to
15.
Fix by skipping such modifying constraints that are children of other
constraints being detached.
Author: Amul Sul <sulamul@gmail.com>
Diagnosys-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/CAAJ_b97GuPh6wQPbxQS-Zpy16Oh+0aMv-w64QcGrLhCOZZ6p+g@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Update time zone data files to tzdata release 2025a.
commit : e292ba3334ee1eccfaa5577da62a433788f2826b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 16:49:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 16:49:15 -0500
DST law changes in Paraguay.
Historical corrections for the Philippines.
Backpatch-through: 13
M src/timezone/data/tzdata.zi
Avoid using timezone Asia/Manila in regression tests.
commit : b7bad919e1b0ea3155514258fb19f45851536686
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 15:47:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jan 2025 15:47:53 -0500
The freshly-released 2025a version of tzdata has a refined estimate
for the longitude of Manila, changing their value for LMT in
pre-standardized-timezone days. This changes the output of one of
our test cases. Since we need to be able to run with system tzdata
files that may or may not contain this update, we'd better stop
making that specific test.
I switched it to use Asia/Singapore, which has a roughly similar UTC
offset. That LMT value hasn't changed in tzdb since 2003, so we can
hope that it's well established.
I also noticed that this set of make_timestamptz tests only exercises
zones east of Greenwich, which seems rather sad, and was not the
original intent AFAICS. (We've already changed these tests once
to stabilize their results across tzdata updates, cf 66b737cd9;
it looks like I failed to consider the UTC-offset-sign aspect then.)
To improve that, add a test with Pacific/Honolulu. That LMT offset
is also quite old in tzdb, so we'll cross our fingers that it doesn't
get improved.
Reported-by: Christoph Berg <cb@df7cb.de>
Discussion: https://postgr.es/m/Z46inkznCxesvDEb@msg.df7cb.de
Backpatch-through: 13
M src/include/datatype/timestamp.h
M src/test/regress/expected/timestamptz.out
M src/test/regress/sql/timestamptz.sql
Fix latch event policy that hid socket events.
commit : 44f400fbc6a402bca9911a5646ad17801bfcdc25
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 20 Jan 2025 15:17:47 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 20 Jan 2025 15:17:47 +1300
If a WaitEventSetWait() caller asks for multiple events, an already set
latch would previously prevent other events from being reported at the
same time. Now, we'll also poll the kernel for other events that would
fit in the caller's output buffer with a zero wait time. This policy
change doesn't affect callers that ask for only one event.
The main caller affected is the postmaster. If its latch is set
extremely frequently by backends launching workers and workers exiting,
we don't want it to handle only those jobs and ignore incoming client
connections.
Back-patch to 16 where the postmaster began using the API. The
fast-return policy changed here is older than that, but doesn't cause
any known problems in earlier releases.
Reported-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://postgr.es/m/Z1n5UpAiGDmFcMmd%40nathan
M src/backend/storage/ipc/latch.c
Fix header check for continuation records where standbys could be stuck
commit : e6767c0ed16f34435b632a78b01d2d5397b3bb09
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 20 Jan 2025 09:30:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 20 Jan 2025 09:30:33 +0900
XLogPageRead() checks immediately for an invalid WAL record header on a
standby, to be able to handle the case of continuation records that need
to be read across two different sources. As written, the check was too
generic, applying to any target LSN. Based on an analysis by Kyotaro
Horiguchi, what really matters is to make sure that the page header is
checked when attempting to read a LSN at the boundary of a segment, to
handle the case of a continuation record that spawns across multiple
pages when dealing with multiple segments, as WAL receivers are spawned
they request WAL from the beginning of a segment. This fix has been
proposed by Kyotaro Horiguchi.
This could cause standbys to loop infinitely when dealing with a
continuation record during a timeline jump, in the case where the
contents of the record in the follow-up page are invalid.
Some regression tests are added to check such scenarios, able to
reproduce the original problem. In the test, the contents of a
continuation record are overwritten with junk zeros on its follow-up
page, and replayed on standbys. This is inspired by 039_end_of_wal.pl,
and is enough to show how standbys should react on promotion by not
being stuck. Without the fix, the test would fail with a timeout. The
test to reproduce the problem has been written by Alexander Kukushkin.
The original check has been introduced in 066871980183, for a similar
problem.
Author: Kyotaro Horiguchi, Alexander Kukushkin
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com
Backpatch-through: 13
M src/backend/access/transam/xlogrecovery.c
M src/test/recovery/meson.build
A src/test/recovery/t/043_no_contrecord_switch.pl
Revert recent changes related to handling of 2PC files at recovery
commit : d1bf86a6221177248669db56f9572bae5add026d
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Jan 2025 13:27:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 17 Jan 2025 13:27:42 +0900
This commit reverts 8f67f994e8ea (down to v13) and c3de0f9eed38 (down to
v17), as these are proving to not be completely correct regarding two
aspects:
- In v17 and newer branches, c3de0f9eed38's check for epoch handling is
incorrect, and does not correctly handle frozen epochs. A logic closer
to widen_snapshot_xid() should be used. The 2PC code should try to
integrate deeper with FullTransactionIds, 5a1dfde8334b being not enough.
- In v13 and newer branches, 8f67f994e8ea is a workaround for the real
issue, which is that we should not attempt CLOG lookups without reaching
consistency. This exists since 728bd991c3c4, and this is reachable with
ProcessTwoPhaseBuffer() called by restoreTwoPhaseData() at the beginning
of recovery.
Per discussion with Noah Misch.
Discussion: https://postgr.es/m/20250116010051.f3.nmisch@google.com
Backpatch-through: 13
M src/backend/access/transam/twophase.c
M src/test/recovery/t/009_twophase.pl
Fix setrefs.c's failure to do expression processing on prune steps.
commit : 0671a71e0210f7a3be55af240e9e7bbbea3cb44d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 20:40:07 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 16 Jan 2025 20:40:07 -0500
We should run the expression subtrees of PartitionedRelPruneInfo
structs through fix_scan_expr. Failure to do so means that
AlternativeSubPlans within those expressions won't be cleaned up
properly, resulting in "unrecognized node type" errors since v14.
It seems fairly likely that at least some of the other steps done
by fix_scan_expr are important here as well, resulting in as-yet-
undetected bugs. Therefore, I've chosen to back-patch this to
all supported branches including v13, even though the known
symptom doesn't manifest in v13.
Per bug #18778 from Alexander Lakhin.
Discussion: https://postgr.es/m/18778-24cd399df6c806af@postgresql.org
M src/backend/optimizer/plan/setrefs.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Move routines to manipulate WAL into PostgreSQL::Test::Cluster
commit : 149ed87e22ce70608cc584f2f54c2f12d3a96680
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 16 Jan 2025 09:26:25 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 16 Jan 2025 09:26:25 +0900
These facilities were originally in the recovery TAP test
039_end_of_wal.pl. A follow-up bug fix with a TAP test doing similar
WAL manipulations requires them, and all these had better not be
duplicated due to their complexity. The routine names are tweaked to
use "wal" more consistently, similarly to the existing "advance_wal".
In v14 and v13, the new routines are moved to PostgresNode.pm.
039_end_of_wal.pl is updated to use the refactored routines, without
changing its coverage.
Reviewed-by: Alexander Kukushkin
Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com
Backpatch-through: 13
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/recovery/t/039_end_of_wal.pl
Avoid symbol collisions between pqsignal.c and legacy-pqsignal.c.
commit : a0dfeae0dc9aca33f3d70eea0fd5f81992562ea0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Jan 2025 18:50:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Jan 2025 18:50:24 -0500
In the name of ABI stability (that is, to avoid a library major
version bump for libpq), libpq still exports a version of pqsignal()
that we no longer want to use ourselves. However, since that has
the same link name as the function exported by src/port/pqsignal.c,
there is a link ordering dependency determining which version will
actually get used by code that uses libpq as well as libpgport.a.
It now emerges that the wrong version has been used by pgbench and
psql since commit 06843df4a rearranged their link commands. This
can result in odd failures in pgbench with the -T switch, since its
SIGALRM handler will now not be marked SA_RESTART. psql may have
some edge-case problems in \watch, too.
Since we don't want to depend on link ordering effects anymore,
let's fix this in the same spirit as b6c7cfac8: use macros to change
the actual link names of the competing functions. We cannot change
legacy-pqsignal.c's exported name of course, so the victim has to be
src/port/pqsignal.c.
In master, rename its exported name to be pqsignal_fe in frontend or
pqsignal_be in backend. (We could perhaps have gotten away with using
the same symbol in both cases, but since the FE and BE versions now
work a little differently, it seems advisable to use different names.)
In back branches, rename to pqsignal_fe in frontend but keep it as
pqsignal in backend. The frontend change could affect third-party
code that is calling pqsignal from libpgport.a or libpgport_shlib.a,
but only if the code is compiled against port.h from a different minor
release than libpgport. Since we don't support using libpgport as a
shared library, it seems unlikely that there will be such a problem.
I left the backend symbol unchanged to avoid an ABI break for
extensions. This means that the link ordering hazard still exists
for any extension that links against libpq. However, none of our own
extensions use both pqsignal() and libpq, and we're not making things
any worse for third-party extensions that do.
Report from Andy Fan, diagnosis by Fujii Masao, patch by me.
Back-patch to all supported branches, as 06843df4a was.
Discussion: https://postgr.es/m/87msfz5qv2.fsf@163.com
M src/include/port.h
M src/interfaces/libpq/legacy-pqsignal.c
M src/port/pqsignal.c
ecpg: Restore detection of unsupported COPY FROM STDIN.
commit : ba2dbedd539616734bb9c2a6ce74e593a9ee4052
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 15 Jan 2025 01:24:24 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 15 Jan 2025 01:24:24 +0900
The ecpg command includes code to warn about unsupported COPY FROM STDIN
statements in input files. However, since commit 3d009e45bd,
this functionality has been broken due to a bug introduced in that commit,
causing ecpg to fail to detect the statement.
This commit resolves the issue, restoring ecpg's ability to detect
COPY FROM STDIN and issue a warning as intended.
Back-patch to all supported versions.
Author: Ryo Kanbayashi
Reviewed-by: Hayato Kuroda, Tom Lane
Discussion: https://postgr.es/m/CANOn0Ez_t5uDCUEV8c1YORMisJiU5wu681eEVZzgKwOeiKhkqQ@mail.gmail.com
M src/interfaces/ecpg/preproc/ecpg.addons
Fix catcache invalidation of a list entry that's being built
commit : 96e61b2792a5a4820504aa5e1defaa8582858cd0
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 14 Jan 2025 14:28:49 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 14 Jan 2025 14:28:49 +0200
If a new catalog tuple is inserted that belongs to a catcache list
entry, and cache invalidation happens while the list entry is being
built, the list entry might miss the newly inserted tuple.
To fix, change the way we detect concurrent invalidations while a
catcache entry is being built. Keep a stack of entries that are being
built, and apply cache invalidation to those entries in addition to
the real catcache entries. This is similar to the in-progress list in
relcache.c.
Back-patch to all supported versions.
Reviewed-by: Noah Misch
Discussion: https://www.postgresql.org/message-id/2234dc98-06fe-42ed-b5db-ac17384dc880@iki.fi
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/include/utils/catcache.h
M src/test/modules/test_misc/meson.build
A src/test/modules/test_misc/t/007_catcache_inval.pl
M src/tools/pgindent/typedefs.list
Fix potential integer overflow in bringetbitmap()
commit : e027ee9902fa414d766c8dc2589c6a3d7aeaa53a
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 15:13:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jan 2025 15:13:14 +0900
This function expects an "int64" as result and stores the number of
pages to add to the index scan bitmap as an "int", multiplying its final
result by 10. For a relation large enough, this can theoretically
overflow if counting more than (INT32_MAX / 10) pages, knowing that the
number of pages is upper-bounded by MaxBlockNumber.
To avoid the overflow, this commit redefines "totalpages", used to
calculate the result, to be an "int64" rather than an "int".
Reported-by: Evgeniy Gorbanyov
Author: James Hunter
Discussion: https://www.postgresql.org/message-id/07704817-6fa0-460c-b1cf-cd18f7647041@basealt.ru
Backpatch-through: 13
M src/backend/access/brin/brin.c
Fix HBA option count
commit : dc24c9ad527fea408ec381208902f2dd412ba9bd
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sun, 12 Jan 2025 23:44:39 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Sun, 12 Jan 2025 23:44:39 +0100
Commit 27a1f8d108 missed updating the max HBA option count to
account for the new option added. Fix by bumping the counter
and adjust the relevant comment to match. Backpatch down to
all supported branches like the erroneous commit.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/286764.1736697356@sss.pgh.pa.us
Backpatch-through: v13
M src/backend/utils/adt/hbafuncs.c
Fix JsonExpr deparsing to quote variable names in the PASSING clause.
commit : d037cc2af14ae59f0d1d0cfae826a2c7b0aa1a37
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 13:36:44 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 13:36:44 +0000
When deparsing a JsonExpr, variable names in the PASSING clause were
not quoted. However, since they are parsed as ColLabel tokens, some
variable names require double quotes to ensure that they are properly
interpreted. Fix by using quote_identifier() in the deparsing code.
This oversight was limited to the SQL/JSON query functions
JSON_EXISTS(), JSON_QUERY(), and JSON_VALUE().
Back-patch to v17, where these functions were added.
Dean Rasheed, reviewed by Tom Lane.
Discussion: https://postgr.es/m/CAEZATCXTpAS%3DncfLNTZ7YS6O5puHeLg_SUYAit%2Bcs7wsrd9Msg%40mail.gmail.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/sqljson_queryfuncs.out
M src/test/regress/sql/sqljson_queryfuncs.sql
Fix XMLTABLE() deparsing to quote namespace names if necessary.
commit : 61b12135f5b71df3721e8f13eab812b38c0de11f
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 12:56:52 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Sun, 12 Jan 2025 12:56:52 +0000
When deparsing an XMLTABLE() expression, XML namespace names were not
quoted. However, since they are parsed as ColLabel tokens, some names
require double quotes to ensure that they are properly interpreted.
Fix by using quote_identifier() in the deparsing code.
Back-patch to all supported versions.
Dean Rasheed, reviewed by Tom Lane.
Discussion: https://postgr.es/m/CAEZATCXTpAS%3DncfLNTZ7YS6O5puHeLg_SUYAit%2Bcs7wsrd9Msg%40mail.gmail.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql
Repair memory leaks in plpython.
commit : e98df02df3f2463771b0f008832023cc609c49b2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Jan 2025 11:45:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Jan 2025 11:45:56 -0500
PLy_spi_execute_plan (PLyPlan.execute) and PLy_cursor_plan
(plpy.cursor) use PLy_output_convert to convert Python values
into Datums that can be passed to the query-to-execute. But they
failed to pay much attention to its warning that it can leave "cruft
generated along the way" behind. Repeated use of these methods can
result in a substantial memory leak for the duration of the calling
plpython function.
To fix, make a temporary memory context to invoke PLy_output_convert
in. This also lets us get rid of the rather fragile code that was
here for retail pfree's of the converted Datums. Indeed, we don't
need the PLyPlanObject.values field anymore at all, though I left it
in place in the back branches in the name of ABI stability.
Mat Arye and Tom Lane, per report from Mat Arye. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/CADsUR0DvVgnZYWwnmKRK65MZg7YLUSTDLV61qdnrwtrAJgU6xw@mail.gmail.com
M src/pl/plpython/plpy_cursorobject.c
M src/pl/plpython/plpy_spi.c
Fix missing ldapscheme option in pg_hba_file_rules()
commit : 8ed9bf0a3217c5fb18e9bba43a655b0a947f6f36
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 10 Jan 2025 22:02:58 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Fri, 10 Jan 2025 22:02:58 +0100
The ldapscheme option was missed when inspecing the HbaLine for
assembling rows for the pg_hba_file_rules function. Backpatch
to all supported versions.
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reported-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Bug: 18769
Discussion: https://postgr.es/m/18769-dd8610cbc0405172@postgresql.org
Backpatch-through: v13
M src/backend/utils/adt/hbafuncs.c
Fix UNION planner datatype issue
commit : 5db9367e51136cb270feb79c741f68c1413dce78
author : David Rowley <drowley@postgresql.org>
date : Fri, 10 Jan 2025 14:31:31 +1300
committer: David Rowley <drowley@postgresql.org>
date : Fri, 10 Jan 2025 14:31:31 +1300
66c0185a3 gave the planner the ability to have union child queries
provide the union planner with pre-sorted input so that UNION queries
could be more efficiently implemented using Merge Append.
That commit overlooked checking that the UNION target list and the union
child target list's types all match. In some corner cases, this could
result in the planner producing sorts using the sort operator of the
top-level UNION's target list type rather than of the union child's
target list's type. The implications of this range from silently
working correctly, despite using the wrong sort operator all the way up
to a segmentation fault.
Here we fix by adjusting the planner so it makes no attempt to have the
subquery produce pre-sorted results when the data type of the UNION
target list and the types from the subquery target list don't match
exactly.
Backpatch to 17, where 66c0185a3 was introduced.
Reported-by: Jason Smith <dqetool@126.com>
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Bug: 18764
Discussion: https://postgr.es/m/18764-63ad667ea26e877a%40postgresql.org
Backpatch-through: 17
M src/backend/optimizer/plan/planner.c
Fix an ALTER GROUP ... DROP USER error message.
commit : c559f61b547188f20ff9c54231f54cd9ef99b867
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 9 Jan 2025 17:10:13 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 9 Jan 2025 17:10:13 -0600
This error message stated the privileges required to add a member
to a group even if the user was trying to drop a member:
postgres=> alter group a drop user b;
ERROR: permission denied to alter role
DETAIL: Only roles with the ADMIN option on role "a" may add members.
Since the required privileges for both operations are the same, we
can fix this by modifying the message to mention both adding and
dropping members:
postgres=> alter group a drop user b;
ERROR: permission denied to alter role
DETAIL: Only roles with the ADMIN option on role "a" may add or drop members.
Author: ChangAo Chen
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/tencent_FAA0D00E3514AAF0BBB6322542A6094FEF05%40qq.com
Backpatch-through: 16
M src/backend/commands/user.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix SLRU bank selection code
commit : ffd9b813465843c5eda04229a09d2167ed4a4a71
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jan 2025 07:33:52 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jan 2025 07:33:52 +0100
The originally submitted code (using bit masking) was correct when the
number of slots was restricted to be a power of two -- but that
limitation was removed during development that led to commit
53c2a97a9266, which made the bank selection code incorrect. This led to
always using a smaller number of banks than available. Change said code
to use integer modulo instead, which works correctly with an arbitrary
number of banks.
It's likely that we could improve on this to avoid runtime use of
integer division. But with this change we're, at least, not wasting
memory on unused banks, and more banks mean less contention, which is
likely to have a much higher performance impact than a single
instruction's latency.
Author: Yura Sokolov <y.sokolov@postgrespro.ru>
Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/9444dc46-ca47-43ed-9058-89c456316306@postgrespro.ru
M src/backend/access/transam/slru.c
M src/include/access/slru.h
Fix off_t overflow in pg_basebackup on Windows.
commit : faee3185aafa3a6055c2558b6fe7b5deb64d5460
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 13:17:36 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 13:17:36 +1300
walmethods.c used off_t to navigate around a pg_wal.tar file that could
exceed 2GB, which doesn't work on Windows and would fail with misleading
errors. Use pgoff_t instead.
Back-patch to all supported branches.
Author: Davinder Singh <davinder.singh@enterprisedb.com>
Reported-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Discussion: https://postgr.es/m/CAKZiRmyM4YnokK6Oenw5JKwAQ3rhP0YTz2T-tiw5dAQjGRXE3Q%40mail.gmail.com
M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h
Provide 64-bit ftruncate() and lseek() on Windows.
commit : af109e3399b63355f126f61916dcd16d833d8481
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 12:10:26 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 9 Jan 2025 12:10:26 +1300
Change our ftruncate() macro to use the 64-bit variant of chsize(), and
add a new macro to redirect lseek() to _lseeki64().
Back-patch to all supported releases, in preparation for a bug fix.
Tested-by: Davinder Singh <davinder.singh@enterprisedb.com>
Discussion: https://postgr.es/m/CAKZiRmyM4YnokK6Oenw5JKwAQ3rhP0YTz2T-tiw5dAQjGRXE3Q%40mail.gmail.com
M src/include/port.h
M src/include/port/win32_port.h
M src/pl/plperl/plperl_system.h
Fix C error reported by Oracle compiler.
commit : 45aef9f6bb0fd5784e2f596f93aae3619c20b1d8
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 8 Jan 2025 16:54:45 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 8 Jan 2025 16:54:45 +1300
Commit 66aaabe7 (branches 13 - 17 only) was not acceptable to the Oracle
Developer Studio compiler on build farm animal wrasse. It accidentally
used a C++ style return statement to wrap a void function. None of the
usual compilers complained, but it is right, that is not allowed in C.
Fix.
Reported-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/Z33vgfVgvOnbFLN9%40paquier.xyz
M src/backend/storage/smgr/smgr.c
Restore smgrtruncate() prototype in back-branches.
commit : 66aaabe7a18f2bc5d22cd832ad612ba5bab18d7b
author : Thomas Munro <tmunro@postgresql.org>
date : Wed, 8 Jan 2025 07:50:30 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Wed, 8 Jan 2025 07:50:30 +1300
It's possible that external code is calling smgrtruncate(). Any
external callers might like to consider the recent changes to
RelationTruncate(), but commit 38c579b0 should not have changed the
function prototype in the back-branches, per ABI stability policy.
Restore smgrtruncate()'s traditional argument list in the back-branches,
but make it a wrapper for a new function smgrtruncate2(). The three
callers in core can use smgrtruncate2() directly. In master (18-to-be),
smgrtruncate2() is effectively renamed to smgrtruncate(), so this wart
is cleaned up.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKG%2BThae6x6%2BjmQiuALQBT2Ae1ChjMh1%3DkMvJ8y_SBJZrvA%40mail.gmail.com
M contrib/pg_visibility/pg_visibility.c
M src/backend/catalog/storage.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/smgr.h
Use PqMsg_* macros in postgres.c.
commit : e43537cdc36146f1becc0084b1acc24a46074ae6
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 15:34:19 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 7 Jan 2025 15:34:19 -0600
Commit f4b54e1ed9, which introduced macros for protocol characters,
missed updating a couple of places in postgres.c.
Author: Dave Cramer
Reviewed-by: Fabrízio de Royes Mello
Discussion: https://postgr.es/m/CADK3HHJUVBPoVOmFesPB-fN8_dYt%2BQELV2UB6jxOW2Z40qF-qw%40mail.gmail.com
Backpatch-through: 17
M src/backend/tcop/postgres.c
Fix error message wording
commit : d7905aa1da00115d56ca199b848bcc93fbedf2c8
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jan 2025 20:07:32 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jan 2025 20:07:32 +0100
The originals are ambiguous and a bit out of style.
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/202412141243.efesjyyvzxsz@alvherre.pgsql
M src/backend/parser/parse_expr.c
M src/interfaces/ecpg/test/expected/sql-sqljson.stderr
M src/test/regress/expected/sqljson.out
Remove duplicate definitions in proc.h
commit : e998901b8873d2cc06fb1a8df8d862fda9630775
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 6 Jan 2025 11:56:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 6 Jan 2025 11:56:03 +0200
These are also present in procnumber.h
Reported-by: Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/bd04d675-4672-4f87-800a-eb5d470c15fc@eisentraut.org
M src/include/storage/proc.h
Document strange jsonb sort order for empty top level arrays
commit : 591128f9c9c0e0b9c511c5eba6f889bd1daf3c4c
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 3 Jan 2025 09:23:46 -0500
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 3 Jan 2025 09:23:46 -0500
Slightly faulty logic in the original jsonb code (commit d9134d0a355)
results in an empty top level array sorting less than a json null. We
can't change the sort order now since it would affect btree indexes over
jsonb, so document the anomaly.
Backpatch to all live branches (13 .. 17)
In master, also add a code comment noting the anomaly.
Reported-by: Yan Chengpen
Reviewed-by: Jian He
Discussion: https://postgr.es/m/OSBPR01MB45199DD8DA2D1CECD50518188E272@OSBPR01MB4519.jpnprd01.prod.outlook.com
M doc/src/sgml/json.sgml
Ignore nullingrels when looking up statistics
commit : 297b280abd7a2e57263cf786a3dd5584cb7e0892
author : Richard Guo <rguo@postgresql.org>
date : Thu, 2 Jan 2025 17:59:32 +0900
committer: Richard Guo <rguo@postgresql.org>
date : Thu, 2 Jan 2025 17:59:32 +0900
When looking up statistical data about an expression, we do not need
to concern ourselves with the outer joins that could null the
Vars/PHVs contained in the expression. Accounting for nullingrels in
the expression could cause estimate_num_groups to count the same Var
multiple times if it's marked with different nullingrels. This is
incorrect, and could lead to "ERROR: corrupt MVNDistinct entry" when
searching for multivariate n-distinct.
Furthermore, the nullingrels could prevent us from matching an
expression to expressional index columns or to the expressions in
extended statistics, leading to inaccurate estimates.
To fix, strip out all the nullingrels from the expression before we
look up statistical data about it. There is one ensuing plan change
in the regression tests, but it looks reasonable and does not
compromise its original purpose.
This patch could result in plan changes, but it fixes an actual bug,
so back-patch to v16 where the outer-join-aware-Var infrastructure was
introduced.
Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4-2Z4k+nFTiZe0Qbu5n8juUWenDAtMzi98bAZQtwHx0-w@mail.gmail.com
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Update copyright for 2025
commit : 759620716adb347c1d8c8b2e6f7d88b947a54c98
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 1 Jan 2025 11:21:54 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 1 Jan 2025 11:21:54 -0500
Backpatch-through: 13
M COPYRIGHT
M doc/src/sgml/legal.sgml
Fix memory leak in pgoutput with relation attribute map
commit : 836435424ba871973766455e939a2fbb9ef98118
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 13:33:58 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 13:33:58 +0900
pgoutput caches the attribute map of a relation, that is free()'d only
when validating a RelationSyncEntry. However, this code path is not
taken when calling any of the SQL functions able to do some logical
decoding, like pg_logical_slot_{get,peek}_changes(), leaking some memory
into CacheMemoryContext on repeated calls.
To address this, a relation's attribute map is allocated in
PGOutputData's cachectx, free()'d at the end of the execution of these
SQL functions when logical decoding ends. This is available down to 15.
v13 and v14 have a similar leak, which will be dealt with later.
Reported-by: Masahiko Sawada
Author: Vignesh C
Reviewed-by: Hou Zhijie
Discussion: https://postgr.es/m/CAD21AoDkAhQVSukOfH3_reuF-j4EU0-HxMqU3dU+bSTxsqT14Q@mail.gmail.com
Discussion: https://postgr.es/m/CALDaNm1hewNAsZ_e6FF52a=9drmkRJxtEPrzCB6-9mkJyeBBqA@mail.gmail.com
Backpatch-through: 15
M src/backend/replication/pgoutput/pgoutput.c
Fix failures with incorrect epoch handling for 2PC files at recovery
commit : c3de0f9eed384f96d846d56f59b68850598e8005
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 09:58:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 09:58:09 +0900
At the beginning of recovery, an orphaned two-phase file in an epoch
different than the one defined in the checkpoint record could not be
removed based on the assumptions that AdjustToFullTransactionId() relies
on, assuming that all files would be either from the current epoch or
from the previous epoch.
If the checkpoint epoch was 0 while the 2PC file was orphaned and in the
future, AdjustToFullTransactionId() would underflow the epoch used to
build the 2PC file path. In non-assert builds, this would create a
WARNING message referring to a 2PC file with an epoch of "FFFFFFFF" (or
UINT32_MAX), as an effect of the underflow calculation, leaving the
orphaned file around.
Some tests are added with dummy 2PC files in the past and the future,
checking that these are properly removed.
Issue introduced by 5a1dfde8334b, that has switched two-phase state
files to use FullTransactionIds.
Reported-by: Vitaly Davydov
Author: Michael Paquier
Reviewed-by: Vitaly Davydov
Discussion: https://postgr.es/m/13b5b6-676c3080-4d-531db900@47931709
Backpatch-through: 17
M src/backend/access/transam/twophase.c
M src/test/recovery/t/009_twophase.pl
Fix handling of orphaned 2PC files in the future at recovery
commit : 03c46e1f26a118c279930d64b343d2dcf8684013
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 08:06:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 30 Dec 2024 08:06:40 +0900
Before 728bd991c3c4, that has improved the support for 2PC files during
recovery, the initial logic scanning files in pg_twophase was done so as
files in the future of the transaction ID horizon were checked first,
followed by a check if a transaction ID is aborted or committed which
could involve a pg_xact lookup. After this commit, these checks have
been done in reverse order.
Files detected as in the future do not have a state that can be checked
in pg_xact, hence this caused recovery to fail abruptly should an
orphaned 2PC file in the future of the transaction ID horizon exist in
pg_twophase at the beginning of recovery.
A test is added to check for this scenario, using an empty 2PC with a
transaction ID large enough to be in the future when running the test.
This test is added in 16 and older versions for now. 17 and newer
versions are impacted by a second bug caused by the addition of the
epoch in the 2PC file names. An equivalent test will be added in these
branches in a follow-up commit, once the second set of issues reported
are fixed.
Author: Vitaly Davydov, Michael Paquier
Discussion: https://postgr.es/m/11e597-676ab680-8d-374f23c0@145466129
Backpatch-through: 13
M src/backend/access/transam/twophase.c
Exclude parallel workers from connection privilege/limit checks.
commit : 15b4c46c328b25be9463db6d2960eeb16a784aad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 16:08:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 16:08:50 -0500
Cause parallel workers to not check datallowconn, rolcanlogin, and
ACL_CONNECT privileges. The leader already checked these things
(except for rolcanlogin which might have been checked for a different
role). Re-checking can accomplish little except to induce unexpected
failures in applications that might not even be aware that their query
has been parallelized. We already had the principle that parallel
workers rely on their leader to pass a valid set of authorization
information, so this change just extends that a bit further.
Also, modify the ReservedConnections, datconnlimit and rolconnlimit
logic so that these limits are only enforced against regular backends,
and only regular backends are counted while checking if the limits
were already reached. Previously, background processes that had an
assigned database or role were subject to these limits (with rather
random exclusions for autovac workers and walsenders), and the set of
existing processes that counted against each limit was quite haphazard
as well. The point of these limits, AFAICS, is to ensure the
availability of PGPROC slots for regular backends. Since all other
types of processes have their own separate pools of PGPROC slots, it
makes no sense either to enforce these limits against them or to count
them while enforcing the limit.
While edge-case failures of these sorts have been possible for a
long time, the problem got a good deal worse with commit 5a2fed911
(CVE-2024-10978), which caused parallel workers to make some of these
checks using the leader's current role where before we had used its
AuthenticatedUserId, thus allowing parallel queries to fail after
SET ROLE. The previous behavior was fairly accidental and I have
no desire to return to it.
This patch includes reverting 73c9f91a1, which was an emergency hack
to suppress these same checks in some cases. It wasn't complete,
as shown by a recent bug report from Laurenz Albe. We can also revert
fd4d93d26 and 492217301, which hacked around the same problems in one
regression test.
In passing, remove the special case for autovac workers in
CheckMyDatabase; it seems cleaner to have AutoVacWorkerMain pass
the INIT_PG_OVERRIDE_ALLOW_CONNS flag, now that that does what's
needed.
Like 5a2fed911, back-patch to supported branches (which sadly no
longer includes v12).
Discussion: https://postgr.es/m/1808397.1735156190@sss.pgh.pa.us
M src/backend/access/transam/parallel.c
M src/backend/access/transam/twophase.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/postmaster.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/lmgr/proc.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/init/postinit.c
M src/include/miscadmin.h
M src/include/storage/proc.h
M src/test/modules/worker_spi/worker_spi.c
Reserve a PGPROC slot and semaphore for the slotsync worker process.
commit : 14141bbbc103bf5db41c8f9a8206dfc9ca626e9f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 12:30:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 12:30:42 -0500
The need for this was missed in commit 93db6cbda, with the result
being that if we launch a slotsync worker it would consume one of
the PGPROCs in the max_connections pool. That could lead to inability
to launch the worker, or to subsequent failures of connection requests
that should have succeeded according to the configured settings.
Rather than create some one-off infrastructure to support this,
let's group the slotsync worker with the existing autovac launcher
in a new category of "special worker" processes. These are kind of
like auxiliary processes, but they cannot use that infrastructure
because they need to be able to run transactions.
For the moment, make these processes share the PGPROC freelist
used for autovac workers (which previously supplied the autovac
launcher too). This is partly to avoid an ABI change in v17,
and partly because it seems silly to have a freelist with
at most two members. This might be worth revisiting if we grow
enough workers in this category.
Tom Lane and Hou Zhijie. Back-patch to v17.
Discussion: https://postgr.es/m/1808397.1735156190@sss.pgh.pa.us
M src/backend/storage/lmgr/proc.c
M src/backend/utils/init/postinit.c
M src/include/miscadmin.h
M src/include/storage/proc.h
Try to avoid semaphore-related test failures on NetBSD/OpenBSD.
commit : a46311ed7214741ad0d38893da4285b2c2b468ad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 11:48:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Dec 2024 11:48:13 -0500
These two platforms have a remarkably tight default limit on the
number of SysV semaphores in the system: SEMMNS is only 60
out-of-the-box. Unless manual action is taken to raise that,
we'll only be able to allocate 3 sets of 16 usable semaphores
each, leading to initdb setting max_connections to just 20.
That's problematic because the core regression tests expect
to be able to launch 20 concurrent sessions, leaving us with
no headroom. This seems to be the cause of intermittent
buildfarm failures on some machines.
While there's no getting around the fact that you'd better raise
SEMMNS for production use on these platforms, it does seem desirable
for "make check" to pass reliably without that. We can make that
happen, at least for awhile longer, with two small changes:
* Change sysv_sema.c's SEMAS_PER_SET to 19, so that we can eat up
all of the available semas not just most of them.
* Change initdb to make the smallest max_connections value it will
consider be 25 not 20.
This is a back-patch of recent HEAD commit 38da05346 into v17.
The motivation for doing this now is that an upcoming bug-fix
patch will give the new-in-17 slotsync worker process its own
reserved PGPROC and hence also semaphore. With that patch but
without this change, v17 would fail to start at all under the
default SEMMNS on these platforms.
Discussion: https://postgr.es/m/db2773a2-aca0-43d0-99c1-060efcd9954e@gmail.com
Discussion: https://postgr.es/m/1808397.1735156190@sss.pgh.pa.us
M doc/src/sgml/runtime.sgml
M src/backend/port/sysv_sema.c
M src/bin/initdb/initdb.c
In REASSIGN OWNED of a database, lock the tuple as mandated.
commit : fa6131377054d04f7c938b196b39eeb22784951b
author : Noah Misch <noah@leadboat.com>
date : Sat, 28 Dec 2024 07:16:22 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 28 Dec 2024 07:16:22 -0800
Commit aac2c9b4fde889d13f859c233c2523345e72d32b mandated such locking
and attempted to fulfill that mandate, but it missed REASSIGN OWNED.
Hence, it remained possible to lose VACUUM's inplace update of
datfrozenxid if a REASSIGN OWNED processed that database at the same
time. This didn't affect the other inplace-updated catalog, pg_class.
For pg_class, REASSIGN OWNED calls ATExecChangeOwner() instead of the
generic AlterObjectOwner_internal(), and ATExecChangeOwner() fulfills
the locking mandate.
Like in GRANT, implement this by following the locking protocol for any
catalog subject to the generic AlterObjectOwner_internal(). It would
suffice to do this for IsInplaceUpdateOid() catalogs only. Back-patch
to v13 (all supported versions).
Kirill Reshke. Reported by Alexander Kukushkin.
Discussion: https://postgr.es/m/CAFh8B=mpKjAy4Cuun-HP-f_vRzh2HSvYFG3rhVfYbfEBUhBAGg@mail.gmail.com
M src/backend/catalog/objectaddress.c
M src/backend/commands/alter.c
M src/include/catalog/objectaddress.h
M src/test/regress/expected/database.out
M src/test/regress/sql/database.sql
meson: Export all libcommon functions in Windows builds
commit : d8b0c64116483c1ce9711e66700dddd709941a22
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 19:22:25 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 19:22:25 +0200
This fixes "unresolved external symbol" errors with extensions that
use functions from libpgport that need special CFLAGS to
compile. Currently, that includes the CRC-32 functions.
Commit 2571c1d5cc did this for libcommon, but I missed that libpqport
has the same issue.
Reported-by: Tom Lane
Backpatch-through: 16, where Meson was introduced
Discussion: https://www.postgresql.org/message-id/CAOdR5yF0krWrxycA04rgUKCgKugRvGWzzGLAhDZ9bzNv8g0Lag@mail.gmail.com
M src/port/meson.build
meson: Export all libcommon functions in Windows builds
commit : c80acbc6fae828fc1f88ef7fc1584bdbccb8de65
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 18:14:18 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 25 Dec 2024 18:14:18 +0200
This fixes "unresolved external symbol" errors with extensions that
use functions from libcommon. This was reported with pgvector.
Reported-by: Andrew Kane
Author: Vladlen Popolitov
Backpatch-through: 16, where Meson was introduced
Discussion: https://www.postgresql.org/message-id/CAOdR5yF0krWrxycA04rgUKCgKugRvGWzzGLAhDZ9bzNv8g0Lag@mail.gmail.com
M src/common/meson.build
postgres_fdw: re-issue cancel requests a few times if necessary.
commit : 89962bfef624822786f8fc6268307407fd10bc7f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2024 15:14:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 23 Dec 2024 15:14:30 -0500
Despite the best efforts of commit 0e5c82380, we're still seeing
occasional failures of postgres_fdw's query_cancel test in the
buildfarm. Investigation suggests that its 100ms timeout is
still not enough to reliably ensure that the remote side starts
the query before receiving the cancel request --- and if it
hasn't, it will just discard the request because it's idle.
We discussed allowing a cancel request to kill the next-received
query, but that would have wide and perhaps unpleasant side-effects.
What seems safer is to make postgres_fdw do what a human user would
likely do, which is issue another cancel request if the first one
didn't seem to do anything. We'll keep the same overall 30 second
grace period before concluding things are broken, but issue additional
cancel requests after 1 second, then 2 more seconds, then 4, then 8.
(The next one in series is 16 seconds, but we'll hit the 30 second
timeout before that.)
Having done that, revert the timeout in query_cancel.sql to 10 ms.
That will still be enough on most machines, most of the time, for
the remote query to start; but now we're intentionally risking the
race condition occurring sometimes in the buildfarm, so that the
repeat-cancel code path will get some testing.
As before, back-patch to v17. We might eventually contemplate
back-patching this further, and/or adding similar logic to dblink.
But given the lack of field complaints to date, this feels like
mostly an exercise in test case stabilization, so v17 is enough.
Discussion: https://postgr.es/m/colnv3lzzmc53iu5qoawynr6qq7etn47lmggqr65ddtpjliq5d@glkveb4m6nop
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/query_cancel.out
M contrib/postgres_fdw/sql/query_cancel.sql
Fix memory leak in pgoutput with publication list cache
commit : bbe68c13abe0f622cb29322651b9a0b67789358f
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 23 Dec 2024 12:48:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 23 Dec 2024 12:48:06 +0900
The pgoutput module caches publication names in a list and frees it upon
invalidation. However, the code forgot to free the actual publication
names within the list elements, as publication names are pstrdup()'d in
GetPublication(). This would cause memory to leak in
CacheMemoryContext, bloating it over time as this context is not
cleaned.
This is a problem for WAL senders running for a long time, as an
accumulation of invalidation requests would bloat its cache memory
usage. A second case, where this leak is easier to see, involves a
backend calling SQL functions like pg_logical_slot_{get,peek}_changes()
which create a new decoding context with each execution. More
publications create more bloat.
To address this, this commit adds a new memory context within the
logical decoding context and resets it each time the publication names
cache is invalidated, based on a suggestion from Amit Kapila. This
ensures that the lifespan of the publication names aligns with that of
the logical decoding context.
Contrary to the HEAD-only commit f0c569d71515 that has changed
PGOutputData to track this new child memory context, the context is
tracked with a static variable whose state is reset with a MemoryContext
reset callback attached to PGOutputData->context, so as ABI
compatibility is preserved in stable branches. This approach is based
on an suggestion from Amit Kapila.
Analyzed-by: Michael Paquier, Jeff Davis
Author: Masahiko Sawada
Reviewed-by: Amit Kapila, Michael Paquier, Euler Taveira, Hou Zhijie
Discussion: https://postgr.es/m/Z0khf9EVMVLOc_YY@paquier.xyz
Backpatch-through: 13
M src/backend/replication/pgoutput/pgoutput.c
Update TransactionXmin when MyProc->xmin is updated
commit : 7cfdb4d1e778349b47e34413a26e931d9b07dda6
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 21 Dec 2024 23:42:39 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Sat, 21 Dec 2024 23:42:39 +0200
GetSnapshotData() set TransactionXmin = MyProc->xmin, but when
SnapshotResetXmin() advanced MyProc->xmin, it did not advance
TransactionXmin correspondingly. That meant that TransactionXmin could
be older than MyProc->xmin, and XIDs between than TransactionXmin and
the real MyProc->xmin could be vacuumed away. One known consequence is
in pg_subtrans lookups: we might try to look up the status of an XID
that was already truncated away.
Back-patch to all supported versions.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/d27a046d-a1e4-47d1-a95c-fbabe41debb4@iki.fi
M src/backend/utils/time/snapmgr.c
Fix corruption when relation truncation fails.
commit : 0350b876b074dc307b82ba18cd3c7cad46066baf
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 20 Dec 2024 21:53:25 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 20 Dec 2024 21:53:25 +1300
RelationTruncate() does three things, while holding an
AccessExclusiveLock and preventing checkpoints:
1. Logs the truncation.
2. Drops buffers, even if they're dirty.
3. Truncates some number of files.
Step 2 could previously be canceled if it had to wait for I/O, and step
3 could and still can fail in file APIs. All orderings of these
operations have data corruption hazards if interrupted, so we can't give
up until the whole operation is done. When dirty pages were discarded
but the corresponding blocks were left on disk due to ERROR, old page
versions could come back from disk, reviving deleted data (see
pgsql-bugs #18146 and several like it). When primary and standby were
allowed to disagree on relation size, standbys could panic (see
pgsql-bugs #18426) or revive data unknown to visibility management on
the primary (theorized).
Changes:
* WAL is now unconditionally flushed first
* smgrtruncate() is now called in a critical section, preventing
interrupts and causing PANIC on file API failure
* smgrtruncate() has a new parameter for existing fork sizes,
because it can't call smgrnblocks() itself inside a critical section
The changes apply to RelationTruncate(), smgr_redo() and
pg_truncate_visibility_map(). That last is also brought up to date with
other evolutions of the truncation protocol.
The VACUUM FileTruncate() failure mode had been discussed in older
reports than the ones referenced below, with independent analysis from
many people, but earlier theories on how to fix it were too complicated
to back-patch. The more recently invented cancellation bug was
diagnosed by Alexander Lakhin. Other corruption scenarios were spotted
by me while iterating on this patch and earlier commit 75818b3a.
Back-patch to all supported releases.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reported-by: rootcause000@gmail.com
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18146-04e908c662113ad5%40postgresql.org
Discussion: https://postgr.es/m/18426-2d18da6586f152d6%40postgresql.org
M contrib/pg_visibility/pg_visibility.c
M src/backend/catalog/storage.c
M src/backend/storage/smgr/md.c
M src/backend/storage/smgr/smgr.c
M src/include/storage/md.h
M src/include/storage/smgr.h
Avoid nbtree index scan SAOP scanBehind confusion.
commit : 9e85b20da7cd89eb0fc503d4dd48f5fc86b45f64
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 19 Dec 2024 11:08:53 -0500
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 19 Dec 2024 11:08:53 -0500
Consistently reset so->scanBehind at the beginning of nbtree array
advancement, even during sktrig_required=false calls (calls where array
advancement is triggered by an unsatisfied non-required array scan key).
Otherwise, it's possible for queries to fail to return all relevant
tuples to the scan given a low-order required scan key that was
previously deemed "satisfied" by a truncated high key attribute value.
This only happened at the point where a later non-required array scan
key needed to be "advanced" once on the next leaf page (that is, once
the right sibling of the truncated high key page was reached).
The underlying issue was that later code within _bt_advance_array_keys
assumed that the so->scanBehind flag must have been set using the
current page's high key (not the previous page's high key). Any later
successful recheck call to _bt_check_compare would therefore spuriously
be prevented from making _bt_advance_array_keys return true, based on
the faulty belief that the truncated attribute must be from the scan's
current tuple (i.e. the non-pivot tuple at the start of the next page).
_bt_advance_array_keys would return false for the tuple, ultimately
resulting in _bt_checkkeys failing to return a matching tuple.
Oversight in commit 5bf748b8, which enhanced nbtree ScalarArrayOp
execution.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WzkJKncfqyAUTeuB5GgRhT1vhsWO2q11dbZNqKmvjopP_g@mail.gmail.com
Backpatch: 17-, where commit 5bf748b8 first appears.
M src/backend/access/nbtree/nbtutils.c
Fix Assert failure in WITH RECURSIVE UNION queries
commit : 7b8d45d278de62b23777b36e1515c34d2eaeaf66
author : David Rowley <drowley@postgresql.org>
date : Thu, 19 Dec 2024 13:12:18 +1300
committer: David Rowley <drowley@postgresql.org>
date : Thu, 19 Dec 2024 13:12:18 +1300
If the non-recursive part of a recursive CTE ended up using
TTSOpsBufferHeapTuple as the table slot type, then a duplicate value
could cause an Assert failure in CheckOpSlotCompatibility() when
checking the hash table for the duplicate value. The expected slot type
for the deform step was TTSOpsMinimalTuple so the Assert failed when the
TTSOpsBufferHeapTuple slot was used.
This is a long-standing bug which we likely didn't notice because it
seems much more likely that the non-recursive term would have required
projection and used a TTSOpsVirtual slot, which CheckOpSlotCompatibility
is ok with.
There doesn't seem to be any harm done here other than the Assert
failure. Both TTSOpsMinimalTuple and TTSOpsBufferHeapTuple slot types
require tuple deformation, so the EEOP_*_FETCHSOME ExprState step would
have properly existed in the ExprState.
The solution is to pass NULL for the ExecBuildGroupingEqual's 'lops'
parameter. This means the ExprState's EEOP_*_FETCHSOME step won't
expect a fixed slot type. This makes CheckOpSlotCompatibility() happy as
no checking is performed when the ExprEvalStep is not expecting a fixed
slot type.
Reported-by: Richard Guo
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAMbWs4-8U9q2LAtf8+ghV11zeUReA3AmrYkxzBEv0vKnDxwkKA@mail.gmail.com
Backpatch-through: 13, all supported versions
M src/backend/executor/execGrouping.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql
Fix memory leak in pg_restore with zstd-compressed data.
commit : 04b86019818067b32ce51883a3322f875d573d15
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 22:31:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 17 Dec 2024 22:31:26 -0500
EndCompressorZstd() neglected to free everything. This was
most visible with a lot of large objects in the dump.
Per report from Tomasz Szypowski. Back-patch to v16
where this code came in.
Discussion: https://postgr.es/m/DU0PR04MB94193D038A128EF989F922D199042@DU0PR04MB9419.eurprd04.prod.outlook.com
M src/bin/pg_dump/compress_zstd.c
Accommodate very large dshash tables.
commit : 18452b70acee293d709f475c4551b3b242d4abd1
author : Nathan Bossart <nathan@postgresql.org>
date : Tue, 17 Dec 2024 15:24:45 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Tue, 17 Dec 2024 15:24:45 -0600
If a dshash table grows very large (e.g., the dshash table for
cumulative statistics when there are millions of tables), resizing
it may fail with an error like:
ERROR: invalid DSA memory alloc request size 1073741824
To fix, permit dshash resizing to allocate more than 1 GB by
providing the DSA_ALLOC_HUGE flag.
Reported-by: Andreas Scherbaum
Author: Matthias van de Meent
Reviewed-by: Cédric Villemain, Michael Paquier, Andres Freund
Discussion: https://postgr.es/m/80a12d59-0d5e-4c54-866c-e69cd6536471%40pgug.de
Backpatch-through: 13
M src/backend/lib/dshash.c
Detect version mismatch in brin_page_items
commit : 3668c1d506bfd061ee6c474502cdbbe3f794a91e
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 16:47:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 16:47:23 +0100
Commit dae761a87ed modified brin_page_items() to return the new "empty"
flag for each BRIN range. But the new output parameter was added in the
middle, which may cause crashes when using the new binary with old
function definition.
The ideal solution would be to introduce API versioning similar to what
pg_stat_statements does, but it's too late for that as PG17 was already
released (so we can't introduce a new extension version). We could do
something similar in brin_page_items() by checking the number of output
columns (and ignoring the new flag), but it doesn't seem very nice.
Instead, simply error out and suggest updating the extension to the
latest version. pageinspect is a superuser-only extension, and there's
not much reason to run an older version. Moreover, there's a precedent
for this approach in 691e8b2e18.
Reported by Ľuboslav Špilák, investigation and patch by me. Backpatch to
17, same as dae761a87ed.
Reported-by: Ľuboslav Špilák
Reviewed-by: Michael Paquier, Hayato Kuroda, Peter Geoghegan
Backpatch-through: 17
Discussion: https://postgr.es/m/VI1PR02MB63331C3D90E2104FD12399D38A5D2@VI1PR02MB6333.eurprd02.prod.outlook.com
Discussion: https://postgr.es/m/flat/3385a58f-5484-49d0-b790-9a198a0bf236@vondra.me
M contrib/pageinspect/brinfuncs.c
M contrib/pageinspect/expected/oldextversions.out
M contrib/pageinspect/sql/oldextversions.sql
Update comments about index parallel builds
commit : 42eae257cfcaa1d20e4f555d14934388ac434c61
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 15:40:07 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 17 Dec 2024 15:40:07 +0100
Commit b43757171470 allowed parallel builds for BRIN, but left behind
two comments claiming only btree indexes support parallel builds.
Reported by Egor Rogov, along with similar issues in SGML docs.
Backpatch to 17, where parallel builds for BRIN were introduced.
Reported-by: Egor Rogov
Backpatch-through: 17
Discussion: https://postgr.es/m/114e2d5d-125e-07d8-94aa-5ad175fb7443@postgrespro.ru
M src/backend/catalog/index.c
M src/backend/optimizer/plan/planner.c
Doc: Fix the wrong link on pg_createsubscriber page.
commit : 88baa2791418c6a1e57064fd48392bb0ebce694b
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 17 Dec 2024 15:02:14 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 17 Dec 2024 15:02:14 +0530
Commit 84db9a0eb1 has added the incorrect link to
'initial data synchronization'. It was a subsection of Row Filter and
didn't provide the required information.
Author: Peter Smith
Reviewed-by: Vignesh C, Pavel Luzanov
Backpatch-through: 17, where it was introduced
Discussion: https://postgr.es/m/CAHut+PtnA4DB_pcv4TDr4NjUSM1=P2N_cuZx5DX09k7LVmaqUA@mail.gmail.com
M doc/src/sgml/ref/pg_createsubscriber.sgml
pg_combinebackup: Fix PITR comparison test in 002_compare_backups
commit : 94e7e77a95ba501955788e1e124b577a665cd85d
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 09:24:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 17 Dec 2024 09:24:18 +0900
The test was creating both the dumps to compare from the same database
on the same node, so it would never detect any mismatches when comparing
the logical dumps of the two servers.
Fixing this issue has revealed that there is a difference in the dumps:
the tablespaces paths are different. This commit uses compare_text()
with a custom comparison function to erase the difference (slightly
tweaked to be able to work with WIN32 and non-WIN32 paths). This way,
the non-relevant parts of the tablespace path are ignored from the check
with the basic structure of the query string still compared.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/87h67653ns.fsf@wibble.ilmari.org
Backpatch-through: 17
M src/bin/pg_combinebackup/t/002_compare_backups.pl
doc: Mention BRIN indexes support parallel builds
commit : f5dec4e861522cb255c97e3125a505cfc77df2f1
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 19:08:52 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 16 Dec 2024 19:08:52 +0100
Two places in the documentation suggest B-tree is the only index access
method allowing parallel builds. Commit b4375717 added parallel builds
for BRIN too, but failed to update the docs. So fix that, and backpatch
to 17, where parallel BRIN builds were introduced.
Author: Egor Rogov
Backpatch-through: 17
Discussion: https://postgr.es/m/114e2d5d-125e-07d8-94aa-5ad175fb7443@postgrespro.ru
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/create_index.sgml
Make 009_twophase.pl test pass with recovery_min_apply_delay set
commit : d1253c67fa29a8cfd15874f9154ddf86e05921a9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 16 Dec 2024 15:56:38 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 16 Dec 2024 15:56:38 +0200
The test failed if you ran the regression tests with TEMP_CONFIG with
recovery_min_apply_delay = '500ms'. Fix the race condition by waiting
for transaction to be applied in the replica, like in a few other
tests.
The failing test was introduced in commit cbfbda7841. Backpatch to all
supported versions like that commit (except v12, which is no longer
supported).
Reported-by: Alexander Lakhin
Discussion: https://www.postgresql.org/message-id/09e2a70a-a6c2-4b5c-aeae-040a7449c9f2@gmail.com
M src/test/recovery/t/009_twophase.pl
pgbench: fix misprocessing of some nested \if constructs.
commit : ff9dc96f370aa33a5c1c578c43c598473705d82f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Dec 2024 14:14:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Dec 2024 14:14:15 -0500
An \if command appearing within a false (not-to-be-executed) \if
branch was incorrectly treated the same as \elif. This could allow
statements within the inner \if to be executed when they should
not be. Also the missing inner \if stack entry would result in an
assertion failure (in assert-enabled builds) when the final \endif
is reached.
Report and patch by Michail Nikolaev. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/CANtu0oiA1ke=SP6tauhNqkUdv5QFsJtS1p=aOOf_iU+EhyKkjQ@mail.gmail.com
M src/bin/pgbench/pgbench.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
doc: Clarify old WAL files are kept until they are summarized.
commit : 87e16d632622c622244dba8d1bfc8afbf222d614
author : Fujii Masao <fujii@postgresql.org>
date : Sun, 15 Dec 2024 11:18:18 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sun, 15 Dec 2024 11:18:18 +0900
The documentation in wal.sgml explains that old WAL files cannot be
removed or recycled until they are archived (when WAL archiving is used)
or replicated (when using replication slots). However, it did not mention
that, similarly, old WAL files are also kept until they are summarized
if WAL summarization is enabled. This commit adds that clarification
to the documentation.
Back-patch to v17 where WAL summarization was added.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/fd0eb0a5-f43b-4e06-b450-cbca011b6cff@oss.nttdata.com
M doc/src/sgml/wal.sgml
contrib/earthdistance: Use SQL-standard function bodies.
commit : 3652de36e4323e9bd528011b181ee46252c4ce58
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Dec 2024 16:07:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Dec 2024 16:07:18 -0500
The @extschema:name@ feature added by 72a5b1fc8 allows us to
make earthdistance's references to the cube extension fully
search-path-secure, so long as all those references are
resolved at extension installation time not runtime.
To do that, we must convert earthdistance's SQL functions to
the new SQL-standard style; but we wanted to do that anyway.
The functions can be updated in our customary style by running
CREATE OR REPLACE FUNCTION in an extension update script.
However, there's still problems in the "CREATE DOMAIN earth"
command: its references to cube functions could be captured
by hostile objects in earthdistance's installation schema,
if that's not where the cube extension is. Worse, the reference
to the cube type itself as the domain's base could be captured,
and that's not something we could fix after-the-fact in the
update script.
What I've done about that is to change the "CREATE DOMAIN earth"
command in the base script earthdistance--1.1.sql. Ordinarily,
changing a released extension script is forbidden; but I think
it's okay here since the results of successful (non-trojaned)
script execution will be identical to before.
A good deal of care is still needed to make the extension's scripts
proof against search-path-based attacks. We have to make sure that
all the function and operator invocations have exact argument-type
matches, to forestall attacks based on supplying a better match.
Fortunately earthdistance isn't very big, so I've just gone through
it and inspected each call to be sure of that. The only actual code
changes needed were to spell all floating-point constants in the style
'-1'::float8, rather than depending on runtime type conversions and/or
negations. (I'm not sure that the shortcuts previously used were
attackable, but removing run-time effort is a good thing anyway.)
I believe that this fixes earthdistance enough that we could
mark it trusted and remove the warnings about it that were
added by 7eeb1d986; but I've not done that here.
The primary reason for dealing with this now is that we've
received reports of pg_upgrade failing for databases that use
earthdistance functions in contexts like generated columns.
That's a consequence of 2af07e2f7 having restricted the search_path
used while evaluating such expressions. The only way to fix that
is to make the earthdistance functions independent of run-time
search_path. This patch is very much nicer than the alternative of
attaching "SET search_path" clauses to earthdistance's functions:
it is more secure and doesn't create a run-time penalty. Therefore,
I've chosen to back-patch this to v16 where @extschema:name@
was added. It won't help unless users update to 16.7 and issue
"ALTER EXTENSION earthdistance UPDATE" before upgrading to 17,
but at least there's now a way to deal with the problem without
manual intervention in the dump/restore process.
Tom Lane and Ronan Dunklau
Discussion: https://postgr.es/m/3316564.aeNJFYEL58@aivenlaptop
Discussion: https://postgr.es/m/6a6439f1-8039-44e2-8fb9-59028f7f2014@mailbox.org
M contrib/earthdistance/Makefile
A contrib/earthdistance/earthdistance–1.1–1.2.sql
M contrib/earthdistance/earthdistance–1.1.sql
M contrib/earthdistance/earthdistance.control
M contrib/earthdistance/meson.build
Fix possible crash in pg_dump with identity sequences.
commit : ad950ea98e54ab0b00305a3c4e7c5cec4996460e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Dec 2024 14:21:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 13 Dec 2024 14:21:36 -0500
If an owned sequence is considered interesting, force its owning
table to be marked interesting too. This ensures, in particular,
that we'll fetch the owning table's column names so we have the
data needed for ALTER TABLE ... ADD GENERATED. Previously there were
edge cases where pg_dump could get SIGSEGV due to not having filled in
the column names. (The known case is where the owning table has been
made part of an extension while its identity sequence is not a member;
but there may be others.)
Also, if it's an identity sequence, force its dumped-components mask
to exactly match the owning table: dump definition only if we're
dumping the table's definition, dump data only if we're dumping the
table's data, etc. This generalizes the code introduced in commit
b965f2617 that set the sequence's dump mask to NONE if the owning
table's mask is NONE. That's insufficient to prevent failures,
because for example the table's mask might only request dumping ACLs,
which would lead us to still emit ALTER TABLE ADD GENERATED even
though we didn't create the table. It seems better to treat an
identity sequence as though it were an inseparable aspect of the
table, matching the treatment used in the backend's dependency logic.
Perhaps this policy needs additional refinement, but let's wait to
see some field use-cases before changing it further.
While here, add a comment in pg_dump.h warning against writing tests
like "if (dobj->dump == DUMP_COMPONENT_NONE)", which was a bug in this
case. There is one other example in getPublicationNamespaces, which
if it's not a bug is at least remarkably unclear and under-documented.
Changing that requires a separate discussion, however.
Per report from Artur Zakirov. Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAKNkYnwXFBf136=u9UqUxFUVagevLQJ=zGd5BsLhCsatDvQsKQ@mail.gmail.com
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Revert "Don't truncate database and user names in startup packets."
commit : d09fbf645ecea4f36309dc34731e6fd66a342131
author : Nathan Bossart <nathan@postgresql.org>
date : Thu, 12 Dec 2024 15:52:04 -0600
committer: Nathan Bossart <nathan@postgresql.org>
date : Thu, 12 Dec 2024 15:52:04 -0600
This reverts commit 562bee0fc13dc95710b8db6a48edad2f3d052f2e.
We received a report from the field about this change in behavior,
so it seems best to revert this commit and to add proper
multibyte-aware truncation as a follow-up exercise.
Fixes bug #18711.
Reported-by: Adam Rauch
Reviewed-by: Tom Lane, Bertrand Drouvot, Bruce Momjian, Thomas Munro
Discussion: https://postgr.es/m/18711-7503ee3e449d2c47%40postgresql.org
Backpatch-through: 17
M src/backend/tcop/backend_startup.c
Improve reporting of pg_upgrade log files on test failure
commit : 8afff7d6db25ab52f155d8a220f692d7d6d84152
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 11 Dec 2024 08:48:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 11 Dec 2024 08:48:52 +0900
On failure, the pg_upgrade log files are automatically appended to the
test log file, but the information reported was inconsistent.
A header, with the log file name, was reported with note(), while the
log contents and a footer used print(), making it harder to diagnose
failures when these are split into console output and test log file
because the pg_upgrade log file path in the header may not be included
in the test log file.
The output is now consolidated so as the header uses print() rather than
note(). An extra note() is added to inform that the contents of a
pg_upgrade log file are appended to the test log file.
The diffs from the regression test suite and dump files all use print()
to show their contents on failure.
Author: Joel Jacobson
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/49f7e64a-b9be-4a90-a9fe-210a7740405e@app.fastmail.com
Backpatch-through: 15
M src/bin/pg_upgrade/t/002_pg_upgrade.pl
Fix elog(FATAL) before PostmasterMain() or just after fork().
commit : 4bd9de3f4190242d3dbc3ddb2b1a5f87027efdd0
author : Noah Misch <noah@leadboat.com>
date : Tue, 10 Dec 2024 13:51:59 -0800
committer: Noah Misch <noah@leadboat.com>
date : Tue, 10 Dec 2024 13:51:59 -0800
Since commit 97550c0711972a9856b5db751539bbaf2f88884c, these failed with
"PANIC: proc_exit() called in child process" due to uninitialized or
stale MyProcPid. That was reachable if close() failed in
ClosePostmasterPorts() or setlocale(category, "C") failed, both
unlikely. Back-patch to v13 (all supported versions).
Discussion: https://postgr.es/m/20241208034614.45.nmisch@google.com
M src/backend/main/main.c
M src/backend/postmaster/fork_process.c
M src/backend/postmaster/postmaster.c
M src/port/pqsignal.c
Fix comments of GUC hooks for timezone_abbreviations
commit : 67ef403d0e8f069368612adb917e42cd62cc6377
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 13:02:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 13:02:24 +0900
The GUC assign and check hooks used "assign_timezone_abbreviations",
which was incorrect.
Issue noticed while browsing this area of the code, introduced in
0a20ff54f5e6.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/Z1eV6Y8yk77GZhZI@paquier.xyz
Backpatch-through: 16
M src/backend/commands/variable.c
Fix outdated comment of scram_build_secret()
commit : 416d1bd66c2859b2a40d265b77299c1f1d5dfc5b
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 12:54:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 10 Dec 2024 12:54:14 +0900
This routine documented that "iterations" would use a default value if
set to 0 by the caller. However, the iteration should always be set by
the caller to a value strictly more than 0, as documented by an
assertion.
Oversight in b577743000cd, that has made the iteration count of SCRAM
configurable.
Author: Matheus Alcantara
Discussion: https://postgr.es/m/ac858943-4743-44cd-b4ad-08a0c10cbbc8@gmail.com
Backpatch-through: 16
M src/common/scram-common.c
Include necessary header files in radixtree.h.
commit : a12b0f3a9dc08ed18711fdcd38763cb3f6bc9f69
author : Masahiko Sawada <msawada@postgresql.org>
date : Mon, 9 Dec 2024 13:07:03 -0800
committer: Masahiko Sawada <msawada@postgresql.org>
date : Mon, 9 Dec 2024 13:07:03 -0800
When #include'ing radixtree.h with RT_SHMEM, it could happen to raise
compiler errors due to missing some declarations of types and
functions.
This commit also removes the inclusion of postgres.h since it's
against our usual convention.
Backpatch to v17, where radixtree.h was introduced.
Reviewed-by: Heikki Linnakangas, Álvaro Herrera
Discussion: https://postgr.es/m/CAD21AoCU9YH%2Bb9Rr8YRw7UjmB%3D1zh8GKQkWNiuN9mVhMvkyrRg%40mail.gmail.com
Backpatch-through: 17
M src/include/lib/radixtree.h
Doc: fix incorrect EXPLAIN ANALYZE output for bloom indexes
commit : f36fe5d07bf94c97cfb6b3f3eefef47702a70e97
author : David Rowley <drowley@postgresql.org>
date : Tue, 10 Dec 2024 09:25:27 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 10 Dec 2024 09:25:27 +1300
It looks like the example case was once modified to increase the number
of rows but the EXPLAIN ANALYZE output wasn't updated to reflect that.
Also adjust the text which discusses the index sizes. With the example
table size, the bloom index isn't quite 8 times more space efficient
than the btree indexes.
Discussion: https://postgr.es/m/CAApHDvovx8kQ0=HTt85gFDAwmTJHpCgiSvRmQZ_6u_g-vQYM_w@mail.gmail.com
Backpatch-through: 13, all supported versions
M doc/src/sgml/bloom.sgml
Fix small memory leaks in GUC checks
commit : 9add1bbfa6659f45eb70f19961ce88973801b8ed
author : Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 9 Dec 2024 20:58:23 +0100
committer: Daniel Gustafsson <dgustafsson@postgresql.org>
date : Mon, 9 Dec 2024 20:58:23 +0100
Follow-up commit to a9d58bfe8a3a. Backpatch down to v16 where
this was added in order to keep the code consistent for future
backpatches.
Author: Tofig Aliev <t.aliev@postgrespro.ru>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/bba4313fdde9db46db279f96f3b748b1@postgrespro.ru
Backpatch-through: 16
M src/backend/commands/variable.c
Simplify executor's determination of whether to use parallelism.
commit : 556f7b7bc18d34ddec45392965c3b3038206bb62
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Dec 2024 14:38:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Dec 2024 14:38:19 -0500
Our parallel-mode code only works when we are executing a query
in full, so ExecutePlan must disable parallel mode when it is
asked to do partial execution. The previous logic for this
involved passing down a flag (variously named execute_once or
run_once) from callers of ExecutorRun or PortalRun. This is
overcomplicated, and unsurprisingly some of the callers didn't
get it right, since it requires keeping state that not all of
them have handy; not to mention that the requirements for it were
undocumented. That led to assertion failures in some corner
cases. The only state we really need for this is the existing
QueryDesc.already_executed flag, so let's just put all the
responsibility in ExecutePlan. (It could have been done in
ExecutorRun too, leading to a slightly shorter patch -- but if
there's ever more than one caller of ExecutePlan, it seems better
to have this logic in the subroutine than the callers.)
This makes those ExecutorRun/PortalRun parameters unnecessary.
In master it seems okay to just remove them, returning the
API for those functions to what it was before parallelism.
Such an API break is clearly not okay in stable branches,
but for them we can just leave the parameters in place after
documenting that they do nothing.
Per report from Yugo Nagata, who also reviewed and tested
this patch. Back-patch to all supported branches.
Discussion: https://postgr.es/m/20241206062549.710dc01cf91224809dd6c0e1@sraoss.co.jp
M src/backend/executor/execMain.c
M src/backend/tcop/postgres.c
M src/backend/tcop/pquery.c
M src/include/executor/execdesc.h
M src/include/utils/portal.h
Improve comment about dropped entries in pgstat.c
commit : bb93b33d7e39bb7a61ad6673193ab953fdc3f29c
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 14:35:44 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 14:35:44 +0900
pgstat_write_statsfile() discards any entries marked as dropped from
being written to the stats file at shutdown, and also included an
assertion based on the same condition.
The intention of the assertion is to track that no pgstats entries
should be left around as terminating backends should drop any entries
they still hold references on before the stats file is written by the
checkpointer, and it not worth taking down the server in this case if
there is a bug making that possible.
Let's improve the comment of this area to document clearly what's
intended.
Based on a discussion with Bertrand Drouvot and Anton A. Melnikov.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/a13e8cdf-b97a-4ecb-8f42-aaa367974e29@postgrespro.ru
Backpatch-through: 15
M src/backend/utils/activity/pgstat.c
Fix invalidation of local pgstats references for entry reinitialization
commit : dc5f9054186a0bbee494e0a6f72c2e9e6a15d3f0
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 10:46:03 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 9 Dec 2024 10:46:03 +0900
818119afccd3 has introduced the "generation" concept in pgstats entries,
incremented a counter when a pgstats entry is reinitialized, but it did
not count on the fact that backends still holding local references to
such entries need to be refreshed if the cache age is outdated. The
previous logic only updated local references when an entry was dropped,
but it needs also to consider entries that are reinitialized.
This matters for replication slot stats (as well as custom pgstats kinds
in 18~), where concurrent drops and creates of a slot could cause
incorrect stats to be locally referenced. This would lead to an
assertion failure at shutdown when writing out the stats file, as the
backend holding an outdated local reference would not be able to drop
during its shutdown sequence the stats entry that should be dropped, as
the last process holding a reference to the stats entry. The
checkpointer was then complaining about such an entry late in the
shutdown sequence, after the shutdown checkpoint is finished with the
control file updated, causing the stats file to not be generated. In
non-assert builds, the entry would just be skipped with the stats file
written.
Note that only logical replication slots use statistics.
A test case based on TAP is added to test_decoding, where a persistent
connection peeking at a slot's data is kept with concurrent drops and
creates of the same slot. This is based on the isolation test case that
Anton has sent. As it requires a node shutdown with a check to make
sure that the stats file is written with this specific sequence of
events, TAP is used instead.
Reported-by: Anton A. Melnikov
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/56bf8ff9-dd8c-47b2-872a-748ede82af99@postgrespro.ru
Backpatch-through: 15
M contrib/test_decoding/t/001_repl_stats.pl
M src/backend/utils/activity/pgstat_shmem.c
Fix possible crash during WindowAgg evaluation
commit : 9d5ce4f1a00aae95193b9737c4e8ced7f0aa4aaa
author : David Rowley <drowley@postgresql.org>
date : Mon, 9 Dec 2024 14:24:07 +1300
committer: David Rowley <drowley@postgresql.org>
date : Mon, 9 Dec 2024 14:24:07 +1300
When short-circuiting WindowAgg node evaluation on the top-level
WindowAgg node using quals on monotonic window functions, because the
WindowAgg run condition can mean there's no need to evaluate subsequent
window function results in the same partition once the run condition
becomes false, it was possible that the executor would use stale results
from the previous invocation of the window function in some cases.
A fix for this was partially done by a5832722, but that commit only
fixed the issue for non-top-level WindowAgg nodes. I mistakenly thought
that the top-level WindowAgg didn't have this issue, but Jayesh's example
case clearly shows that's incorrect. At the time, I also thought that
this only affected 32-bit systems as all window functions which then
supported run conditions returned BIGINT, however, that's wrong as
ExecProject is still called and that could cause evaluation of any other
window function belonging to the same WindowAgg node, one of which may
return a byref type.
The only queries affected by this are WindowAggs with a "Run Condition"
which contains at least one window function with a byref result type,
such as lead() or lag() on a byref column. The window clause must also
contain a PARTITION BY clause (without a PARTITION BY, execution of the
WindowAgg stops immediately when the run condition becomes false and
there's no risk of using the stale results).
Reported-by: Jayesh Dehankar
Discussion: https://postgr.es/m/193261e2c4d.3dd3cd7c1842.871636075166132237@zohocorp.com
Backpatch-through: 15, where WindowAgg run conditions were added
M src/backend/executor/nodeWindowAgg.c
M src/test/regress/expected/window.out
M src/test/regress/sql/window.sql
Ensure that pg_amop/amproc entries depend on their lefttype/righttype.
commit : ec7b89cc533aaa8356b018e557b252047330b471
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 15:56:28 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 15:56:28 -0500
Usually an entry in pg_amop or pg_amproc does not need a dependency on
its amoplefttype/amoprighttype/amproclefttype/amprocrighttype types,
because there is an indirect dependency via the argument types of its
referenced operator or procedure, or via the opclass it belongs to.
However, for some support procedures in some index AMs, the argument
types of the support procedure might not mention the column data type
at all. Also, the amop/amproc entry might be treated as "loose" in
the opfamily, in which case it lacks a dependency on any particular
opclass; or it might be a cross-type entry having a reference to a
datatype that is not its opclass' opcintype.
The upshot of all this is that there are cases where a datatype can
be dropped while leaving behind amop/amproc entries that mention it,
because there is no path in pg_depend showing that those entries
depend on that type. Such entries are harmless in normal activity,
because they won't get used, but they cause problems for maintenance
actions such as dropping the operator family. They also cause pg_dump
to produce bogus output. The previous commit put a band-aid on the
DROP OPERATOR FAMILY failure, but a real fix is needed.
To fix, add pg_depend entries showing that a pg_amop/pg_amproc entry
depends on its lefttype/righttype. To avoid bloating pg_depend too
much, skip this if the referenced operator or function has that type
as an input type. (I did not bother with considering the possible
indirect dependency via the opclass' opcintype; at least in the
reported case, that wouldn't help anyway.)
Probably, the reason this has escaped notice for so long is that
add-on datatypes and relevant opclasses/opfamilies are usually
packaged as extensions nowadays, so that there's no way to drop
a type without dropping the referencing opclasses/opfamilies too.
Still, in the absence of pg_depend entries there's nothing that
constrains DROP EXTENSION to drop the opfamily entries before the
datatype, so it seems possible for a DROP failure to occur anyway.
The specific case that was reported doesn't fail in v13, because
v13 prefers to attach the support procedure to the opclass not the
opfamily. But it's surely possible to construct other edge cases
that do fail in v13, so patch that too.
Per report from Yoran Heling. Back-patch to all supported branches.
Discussion: https://postgr.es/m/Z1MVCOh1hprjK5Sf@gmai021
M src/backend/commands/opclasscmds.c
M src/include/access/amapi.h
Make getObjectDescription robust against dangling amproc type links.
commit : 5b44a317ae031de76989be98871cd1e67c1669e4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 14:28:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 14:28:16 -0500
Yoran Heling reported a case where a data type could be dropped
while references to its OID remain behind in pg_amproc. This
causes getObjectDescription to fail, which blocks dropping the
operator family (since our DROP code likes to construct descriptions
of everything it's dropping). The proper fix for this requires
adding more pg_depend entries. But to allow DROP to go through with
already-corrupt catalogs, tweak getObjectDescription to print "???"
for the type instead of failing when it processes such an entry.
I changed the logic for pg_amop similarly, for consistency,
although it is not known that the problem can manifest in pg_amop.
Per report from Yoran Heling. Back-patch to all supported
branches (although the problem may be unreachable in v13).
Discussion: https://postgr.es/m/Z1MVCOh1hprjK5Sf@gmai021
M src/backend/catalog/objectaddress.c
Fix is_digit labeling of to_timestamp's FFn format codes.
commit : 765f76d8cd9911eaae12aa958811e28fe2040dd9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 13:12:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Dec 2024 13:12:32 -0500
These format codes produce or consume strings of digits, so they
should be labeled with is_digit = true, but they were not.
This has effect in only one place, where is_next_separator()
is checked to see if the preceding format code should slurp up
all the available digits. Thus, with a format such as '...SSFF3'
with remaining input '12345', the 'SS' code would consume all
five digits (and then complain about seconds being out of range)
when it should eat only two digits.
Per report from Nick Davies. This bug goes back to d589f9446
where the FFn codes were introduced, so back-patch to v13.
Discussion: https://postgr.es/m/AM8PR08MB6356AC979252CFEA78B56678B6312@AM8PR08MB6356.eurprd08.prod.outlook.com
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
doc: remove LC_COLLATE and LC_CTYPE from SHOW command
commit : c140c0ffb75b0130830b8ef307b0d3b21e18c5cf
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Dec 2024 12:55:55 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Dec 2024 12:55:55 +0100
The corresponding read-only server settings have been removed since
in PG16. See commit b0f6c437160db6.
Author: Pierre Giraud <pierre.giraud@dalibo.com>
Discussion: https://www.postgresql.org/message-id/flat/a75a2fb0-f4b3-4c0c-be3d-7a62d266d760%40dalibo.com
M doc/src/sgml/ref/show.sgml
Fix use-after-free in parallel_vacuum_reset_dead_items
commit : 83ce20d67184d79f7a1e65feed7c06ec03e4472d
author : John Naylor <john.naylor@postgresql.org>
date : Wed, 4 Dec 2024 16:51:55 +0700
committer: John Naylor <john.naylor@postgresql.org>
date : Wed, 4 Dec 2024 16:51:55 +0700
parallel_vacuum_reset_dead_items used a local variable to hold a
pointer from the passed vacrel, purely as a shorthand. This pointer
was later freed and a new allocation was made and stored to the
struct. Then the local pointer was mistakenly referenced again.
This apparently happened not to break anything since the freed chunk
would have been put on the context's freelist, so it was accidentally
the same pointer anyway, in which case the DSA handle was correctly
updated. The minimal fix is to change two places so they access
dead_items through the vacrel. This coding style is a maintenance
hazard, so while at it get rid of most other similar usages, which
were inconsistently used anyway.
Analysis and patch by Vallimaharajan G, with further defensive coding
by me
Backpath to v17, when TidStore came in
Discussion: https://postgr.es/m/1936493cc38.68cb2ef27266.7456585136086197135@zohocorp.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/vacuumparallel.c
Fix synchronized_standby_slots GUC check hook
commit : 9abdc1841e16ff4eaf2fa8e2e0472ee71a4a5a5c
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 17:50:57 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 17:50:57 +0100
The validate_sync_standby_slots subroutine requires an LWLock, so it
cannot run in processes without PGPROC; skip it there to avoid a crash.
This replaces the current test for ReplicationSlotCtl being not null,
which appears to be a solution for the same problem but less general.
I also rewrote a related comment that mentioned ReplicationSlotCtl in
StandbySlotsHaveCaughtup.
This code came in with commit bf279ddd1c28; backpatch to 17.
Reported-by: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Zhijie Hou <houzj.fnst@fujitsu.com>
Discussion: https://postgr.es/m/202411281216.sutbxtr6idnn@alvherre.pgsql
M src/backend/replication/slot.c
Drop "Lock" suffix from LWLock wait event names
commit : 5ffbbcfa160b2e689dfb0542dfdbc4e16fc0dec9
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 15:50:03 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 3 Dec 2024 15:50:03 +0100
Commit da952b415f44 unintentially reverted the SQL-visible part of
commit 14a910109126, which breaks queries joining pg_wait_events with
pg_stat_acivity. Remove the suffix again.
Backpatch to 17.
Reported-by: Christophe Courtois <christophe.courtois@dalibo.com>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/18728-450924477056a339%40postgresql.org
Discussion: https://postgr.es/m/Z01w1+LihtRiS0Te@ip-10-97-1-34.eu-west-3.compute.internal
M src/backend/storage/lmgr/generate-lwlocknames.pl
M src/backend/storage/lmgr/lwlock.c
doc: Clarify some terms for pg_createsubscriber
commit : 7a35052a2b6e32fdcbe60d668509c35b747f89a8
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Dec 2024 16:21:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 3 Dec 2024 16:21:10 +0900
The last section of pg_createsubscriber used the terms
"publication-name", "replication-slot-name", and "subscription-name".
These terms are not defined on the page, which was confusing, and the
intention is clearly to refer to the values one would give to the
options --publication, --subscription and --replication-slot. Let's
simplify the documentation by mentioning the option switches, instead of
these terms.
Reported-by: Christophe Courtois
Author: Shubham Khanna
Reviewed-by: Vignesh C, Peter Smith
Discussion: https://postgr.es/m/173288198026.714.15127074046508836738@wrigleys.postgresql.org
Backpatch-through: 17
M doc/src/sgml/ref/pg_createsubscriber.sgml
RelationTruncate() must set DELAY_CHKPT_START.
commit : d4ffbf47b2d43fe3319474970948f276c8e2f8c9
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 3 Dec 2024 09:27:05 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 3 Dec 2024 09:27:05 +1300
Previously, it set only DELAY_CHKPT_COMPLETE. That was important,
because it meant that if the XLOG_SMGR_TRUNCATE record preceded a
XLOG_CHECKPOINT_ONLINE record in the WAL, then the truncation would also
happen on disk before the XLOG_CHECKPOINT_ONLINE record was
written.
However, it didn't guarantee that the sync request for the truncation
was processed before the XLOG_CHECKPOINT_ONLINE record was written. By
setting DELAY_CHKPT_START, we guarantee that if an XLOG_SMGR_TRUNCATE
record is written to WAL before the redo pointer of a concurrent
checkpoint, the sync request queued by that operation must be processed
by that checkpoint, rather than being left for the following one.
This is a refinement of commit 412ad7a5563. Back-patch to all supported
releases, like that commit.
Author: Robert Haas <robertmhaas@gmail.com>
Reported-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2B-2rjGZC2kwqr2NMLBcEBp4uf59QT1advbWYF_uc%2B0Aw%40mail.gmail.com
M src/backend/catalog/storage.c
Fix broken list-munging in ecpg's remove_variables().
commit : a963abd549e9e2b518460ccf31ea3728214207c0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Dec 2024 14:15:37 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Dec 2024 14:15:37 -0500
The loops over cursor argument variables neglected to ever advance
"prevvar". The code would accidentally do the right thing anyway
when removing the first or second list entry, but if it had to
remove the third or later entry then it would also remove all
entries between there and the first entry. AFAICS this would
only matter for cursors that reference out-of-scope variables,
which is a weird Informix compatibility hack; between that and
the lack of impact for short lists, it's not so surprising that
nobody has complained. Nonetheless it's a pretty obvious bug.
It would have been more obvious if these loops used a more standard
coding style for chasing the linked lists --- this business with the
"prev" pointer sometimes pointing at the current list entry is
confusing and overcomplicated. So rather than just add a minimal
band-aid, I chose to rewrite the loops in the same style we use
elsewhere, where the "prev" pointer is NULL until we are dealing with
a non-first entry and we save the "next" pointer at the top of the
loop. (Two of the four loops touched here are not actually buggy,
but it seems better to make them all look alike.)
Coverity discovered this problem, but not until 2b41de4a5 added code
to free no-longer-needed arguments structs. With that, the incorrect
link updates are possibly touching freed memory, and it complained
about that. Nonetheless the list corruption hazard is ancient, so
back-patch to all supported branches.
M src/interfaces/ecpg/preproc/variable.c
Avoid mislabeling of lateral references, redux.
commit : 78883cd90511176bda3333a92b1cc3f65a66ad7f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 30 Nov 2024 12:42:20 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 30 Nov 2024 12:42:20 -0500
As I'd feared, commit 5c9d8636d was still a few bricks shy of a load.
We can't just leave pulled-up lateral-reference Vars with no new
nullingrels: we have to carefully compute what subset of the
to-be-replaced Var's nullingrels apply to them, else we still get
"wrong varnullingrels" errors. This is a bit tedious, but it looks
like we can use the nullingrel data this patch computes for other
purposes, enabling better optimization. We don't want to inject
unnecessary plan changes into stable branches though, so leave that
idea for a later HEAD-only patch.
Patch by me, but thanks to Richard Guo for devising a test case that
broke 5c9d8636d, and for preliminary investigation about how to fix
it. As before, back-patch to v16.
Discussion: https://postgr.es/m/E1tGn4j-0003zi-MP@gemulon.postgresql.org
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
M src/tools/pgindent/typedefs.list
Avoid mislabeling of lateral references when pulling up a subquery.
commit : 72822a99d44f040e9776c892a8ab877a06c7b7ad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Nov 2024 17:33:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 28 Nov 2024 17:33:16 -0500
If we are pulling up a subquery that's under an outer join, and
the subquery's target list contains a strict expression that uses
both a subquery variable and a lateral-reference variable, it's okay
to pull up the expression without wrapping it in a PlaceHolderVar.
That's safe because if the subquery variable is forced to NULL
by the outer join, the expression result will come out as NULL too,
so we don't have to force that outcome by evaluating the expression
below the outer join. It'd be correct to wrap in a PHV, but that can
lead to very significantly worse plans, since we'd then have to use
a nestloop plan to pass down the lateral reference to where the
expression will be evaluated.
However, when we do that, we should not mark the lateral reference
variable as being nulled by the outer join, because it isn't after
we pull up the expression in this way. So the marking logic added
by cb8e50a4a was incorrect in this detail, leading to "wrong
varnullingrels" errors from the consistency-checking logic in
setrefs.c. It seems to be sufficient to just not mark lateral
references at all in this case. (I have a nagging feeling that more
complexity may be needed in cases where there are several levels of
outer join, but some attempts to break it with that didn't succeed.)
Per report from Bertrand Mamasam. Back-patch to v16, as the previous
patch was.
Discussion: https://postgr.es/m/CACZ67_UA_EVrqiFXJu9XK50baEpH=ofEPJswa2kFxg6xuSw-ww@mail.gmail.com
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
psql: Add tab completion for COPY (MERGE ...
commit : 4527b9e26db8d4987df6f83fc580993d8d103a7e
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Nov 2024 09:14:41 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 28 Nov 2024 09:14:41 +0100
The underlying feature for this was added in PostgreSQL 17.
Author: Jian He <jian.universality@gmail.com>
Discussion: https://www.postgresql.org/message-id/CACJufxEmNjxvf1deR1zBrJbjAMeCooooLRzZ+yaaBuqDKh_6-Q@mail.gmail.com
M src/bin/psql/tab-complete.c
Revert "Handle better implicit transaction state of pipeline mode"
commit : 7668e85a4044d97e7d7e950310ca0b6467c0bd5d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Nov 2024 09:43:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 28 Nov 2024 09:43:21 +0900
This reverts commit d77f91214fb7 on all stable branches, due to concerns
regarding the compatility side effects this could create in a minor
release. The change still exists on HEAD.
Discussion: https://postgr.es/m/CA+TgmoZqRgeFTg4+Yf_CMRRXiHuNz1u6ZC4FvVk+rxw0RmOPnw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/protocol.sgml
M src/backend/access/transam/xact.c
M src/backend/tcop/postgres.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
ci: Fix cached MacPorts installation management
commit : c93dffd89ba58b1b22faae2d2a9ec95e536c6740
author : Andres Freund <andres@anarazel.de>
date : Wed, 27 Nov 2024 11:30:00 -0500
committer: Andres Freund <andres@anarazel.de>
date : Wed, 27 Nov 2024 11:30:00 -0500
1. The error reporting of "port setrequested list-of-packages..."
changed, hiding errors we were relying on to know if all packages in our
list were already installed. Use a loop instead.
2. The cached MacPorts installation was shared between PostgreSQL
major branches, though each branch wanted different packages. Add the
list of packages to cache key, so that different branches, when tested
in one github account/repo such as postgres/postgres, stop fighting with
each other, adding and removing packages.
Back-patch to 15 where CI began.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Suggested-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/au2uqfuy2nf43nwy2txmc5t2emhwij7kzupygto3d2ffgtrdgr%40ckvrlwyflnh2
M .cirrus.tasks.yml
M src/tools/ci/ci_macports_packages.sh
pgbench: Ensure previous progress message is fully cleared when updating.
commit : adb103fcacdde73d1446d72ce9c2da414c86bd74
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 27 Nov 2024 23:01:53 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 27 Nov 2024 23:01:53 +0900
During pgbench's table initialization, progress updates could display
leftover characters from the previous message if the new message
was shorter. This commit resolves the issue by appending spaces to
the current message to fully overwrite any remaining characters from
the previous line.
Back-patch to all the supported versions.
Author: Yushi Ogiwara, Tatsuo Ishii, Fujii Masao
Reviewed-by: Tatsuo Ishii, Fujii Masao
Discussion: https://postgr.es/m/9a9b8b95b6a709877ae48ad5b0c59bb9@oss.nttdata.com
M src/bin/pgbench/pgbench.c
Fix pg_get_constraintdef for NOT NULL constraints on domains
commit : 6e793582bc663fb7652834170be8902b982b7bcb
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 13:50:27 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 27 Nov 2024 13:50:27 +0100
We added pg_constraint rows for all not-null constraints, first for
tables and later for domains; but while the ones for tables were
reverted, the ones for domains were not. However, we did accidentally
revert ruleutils.c support for the ones on domains in 6f8bb7c1e961,
which breaks running pg_get_constraintdef() on them. Put that back.
This is only needed in branch 17, because we've reinstated this code in
branch master with commit 14e87ffa5c54. Add some new tests in both
branches.
I couldn't find anything else that needs de-reverting.
Reported-by: Erki Eessaar <erki.eessaar@taltech.ee>
Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/AS8PR01MB75110350415AAB8BBABBA1ECFE222@AS8PR01MB7511.eurprd01.prod.exchangelabs.com
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/domain.out
M src/test/regress/sql/domain.sql
Fix typo
commit : 7786717b9e5d0e7ad828a7c7a65c74732ec492a9
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:12:09 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:12:09 +0100
from commit 9044fc1d45a
M src/tools/pgindent/exclude_file_patterns
Exclude LLVM files from whitespace checks
commit : 3c1ab90068496a7d616497931c41cae4d54c97ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:08:12 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 27 Nov 2024 11:08:12 +0100
Commit 9044fc1d45a added some files from upstream LLVM. These files
have different whitespace rules, which make the git whitespace checks
powered by gitattributes fail. To fix, add those files to the exclude
list.
M .gitattributes
Handle better implicit transaction state of pipeline mode
commit : d77f91214fb70879f7013dfd10a7bcac2a8159cc
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Nov 2024 09:31:37 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 27 Nov 2024 09:31:37 +0900
When using a pipeline, a transaction starts from the first command and
is committed with a Sync message or when the pipeline ends.
Functions like IsInTransactionBlock() or PreventInTransactionBlock()
were already able to understand a pipeline as being in a transaction
block, but it was not the case of CheckTransactionBlock(). This
function is called for example to generate a WARNING for SET LOCAL,
complaining that it is used outside of a transaction block.
The current state of the code caused multiple problems, like:
- SET LOCAL executed at any stage of a pipeline issued a WARNING, even
if the command was at least second in line where the pipeline is in a
transaction state.
- LOCK TABLE failed when invoked at any step of a pipeline, even if it
should be able to work within a transaction block.
The pipeline protocol assumes that the first command of a pipeline is
not part of a transaction block, and that any follow-up commands is
considered as within a transaction block.
This commit changes the backend so as an implicit transaction block is
started each time the first Execute message of a pipeline has finished
processing, with this implicit transaction block ended once a sync is
processed. The checks based on XACT_FLAGS_PIPELINING in the routines
checking if we are in a transaction block are not necessary: it is
enough to rely on the existing ones.
Some tests are added to pgbench, that can be backpatched down to v17
when \syncpipeline is involved and down to v14 where \startpipeline and
\endpipeline are available. This is unfortunately limited regarding the
error patterns that can be checked, but it provides coverage for various
pipeline combinations to check if these succeed or fail. These tests
are able to capture the case of SET LOCAL's WARNING. The author has
proposed a different feature to improve the coverage by adding similar
meta-commands to psql where error messages could be checked, something
more useful for the cases where commands cannot be used in transaction
blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as
future work for v18~.
Author: Anthonin Bonnefoy
Reviewed-by: Jelte Fennema-Nio, Michael Paquier
Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/protocol.sgml
M src/backend/access/transam/xact.c
M src/backend/tcop/postgres.c
M src/bin/pgbench/t/001_pgbench_with_server.pl
meson: Build pgevent as shared_module rather than shared_library
commit : e00c1e249f9b5308fc47b09b8945c47d5146aa64
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 26 Nov 2024 18:06:08 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 26 Nov 2024 18:06:08 +0100
This matches the behavior of the makefiles and the old MSVC build
system. The main effect is that the build result gets installed into
pkglibdir rather than bindir. The documentation says to locate the
library in pkglibdir, so this makes the code match the documentation
again.
Reviewed-by: Ryohei Takahashi (Fujitsu) <r.takahashi_2@fujitsu.com>
Discussion: https://www.postgresql.org/message-id/flat/TY3PR01MB118912125614599641CA881B782522%40TY3PR01MB11891.jpnprd01.prod.outlook.com
M src/bin/pgevent/meson.build
Clean up newlines following left parentheses
commit : b0e572819d51bfd87d161e5e1d11178610f6fb3a
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 26 Nov 2024 17:10:07 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 26 Nov 2024 17:10:07 +0100
Most came in during the 17 cycle, so backpatch there. Some
(particularly reorderbuffer.h) are very old, but backpatching doesn't
seem useful.
Like commits c9d297751959, c4f113e8fef9.
M contrib/bloom/bloom.h
M src/backend/backup/basebackup_incremental.c
M src/backend/utils/init/postinit.c
M src/bin/pg_combinebackup/load_manifest.c
M src/common/parse_manifest.c
M src/common/unicode/category_test.c
M src/include/common/parse_manifest.h
M src/include/replication/reorderbuffer.h
Fix C23 compiler warning
commit : f3834c12afd8125c4e5e8774a4b941a58dab8f5b
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 22 Oct 2024 08:12:43 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 22 Oct 2024 08:12:43 +0200
The approach of declaring a function pointer with an empty argument
list and hoping that the compiler will not complain about casting it
to another type no longer works with C23, because foo() is now
equivalent to foo(void).
We don't need to do this here. With a few struct forward declarations
we can supply a correct argument list without having to pull in
another header file.
(This is the only new warning with C23. Together with the previous
fix a67a49648d9, this makes the whole code compile cleanly under C23.)
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/95c6a9bf-d306-43d8-b880-664ef08f2944%40eisentraut.org
Discussion: https://www.postgresql.org/message-id/flat/87o72eo9iu.fsf%40gentoo.org
M src/include/nodes/pathnodes.h
Rename C23 keyword
commit : ad89c8bda1f096a8592f1c27abec1e34e5f10408
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 13 Aug 2024 06:15:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 13 Aug 2024 06:15:28 +0200
constexpr is a keyword in C23. Rename a conflicting identifier for
future-proofing.
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/08abc832-1384-4aca-a535-1a79765b565e%40eisentraut.org
Discussion: https://www.postgresql.org/message-id/flat/87o72eo9iu.fsf%40gentoo.org
M src/backend/optimizer/util/predtest.c
Fix NULLIF()'s handling of read-write expanded objects.
commit : 97be02ad0015502fac0b788bc0a4739a398f41d5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 18:08:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 18:08:58 -0500
If passed a read-write expanded object pointer, the EEOP_NULLIF
code would hand that same pointer to the equality function
and then (unless equality was reported) also return the same
pointer as its value. This is no good, because a function that
receives a read-write expanded object pointer is fully entitled
to scribble on or even delete the object, thus corrupting the
NULLIF output. (This problem is likely unobservable with the
equality functions provided in core Postgres, but it's easy to
demonstrate with one coded in plpgsql.)
To fix, make sure the pointer passed to the equality function
is read-only. We can still return the original read-write
pointer as the NULLIF result, allowing optimization of later
operations.
Per bug #18722 from Alexander Lakhin. This has been wrong
since we invented expanded objects, so back-patch to all
supported branches.
Discussion: https://postgr.es/m/18722-fd9e645448cc78b4@postgresql.org
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/include/executor/execExpr.h
M src/test/regress/expected/case.out
M src/test/regress/sql/case.sql
Avoid "you don't own a lock of type ExclusiveLock" in GRANT TABLESPACE.
commit : 718af10dab446c2ae73fe027a9f65be69bcf2980
author : Noah Misch <noah@leadboat.com>
date : Mon, 25 Nov 2024 14:42:35 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 25 Nov 2024 14:42:35 -0800
This WARNING appeared because SearchSysCacheLocked1() read
cc_relisshared before catcache initialization, when the field is false
unconditionally. On the basis of reading false there, it constructed a
locktag as though pg_tablespace weren't relisshared. Only shared
catalogs could be affected, and only GRANT TABLESPACE was affected in
practice. SearchSysCacheLocked1() callers use one other shared-relation
syscache, DATABASEOID. DATABASEOID is initialized by the end of
CheckMyDatabase(), making the problem unreachable for pg_database.
Back-patch to v13 (all supported versions). This has no known impact
before v16, where ExecGrant_common() first appeared. Earlier branches
avoid trouble by having a separate ExecGrant_Tablespace() that doesn't
use LOCKTAG_TUPLE. However, leaving this unfixed in v15 could ensnare a
future back-patch of a SearchSysCacheLocked1() call.
Reported by Aya Iwata.
Discussion: https://postgr.es/m/OS7PR01MB11964507B5548245A7EE54E70EA212@OS7PR01MB11964.jpnprd01.prod.outlook.com
M src/backend/utils/cache/syscache.c
M src/test/regress/expected/tablespace.out
M src/test/regress/sql/tablespace.sql
Clean up <stdbool.h> reference in meson.build.
commit : aefbd6c29fa984e9ab1fd4b0b5d3e3fce4b18b03
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Nov 2024 11:29:31 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 26 Nov 2024 11:29:31 +1300
Commit bc5a4dfc accidentally left a check for <stdbool.h> in
meson.build's header_checks. Synchronize with configure, which no
longer defines HAVE_STDBOOL_H.
There is still a reference to <stdbool.h> in an earlier test to see if
we need -std=c99 to get C99 features, like autoconf 2.69's
AC_PROG_CC_C99. (Therefore the test remove by this commit was
tautological since day one: you'd have copped "C compiler does not
support C99" before making it this far.)
Back-patch to 16, where meson begins.
M meson.build
Update configure probes for CFLAGS needed for ARM CRC instructions.
commit : e266a0ed67d2fb34c41de11a7cd3f73c66765331
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 12:50:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 25 Nov 2024 12:50:17 -0500
On ARM platforms where the baseline CPU target lacks CRC instructions,
we need to supply a -march flag to persuade the compiler to compile
such instructions. It turns out that our existing choice of
"-march=armv8-a+crc" has not worked for some time, because recent gcc
will interpret that as selecting software floating point, and then
will spit up if the platform requires hard-float ABI, as most do
nowadays. The end result was to silently fall back to software CRC,
which isn't very desirable since in practice almost all currently
produced ARM chips do have hardware CRC.
We can fix this by using "-march=armv8-a+crc+simd" to enable the
correct ABI choice. (This has no impact on the code actually
generated, since neither of the files we compile with this flag
does any floating-point stuff, let alone SIMD.) Keep the test for
"-march=armv8-a+crc" since that's required for soft-float ABI,
but try that second since most platforms we're likely to build on
use hard-float.
Since this isn't working as-intended on the last several years'
worth of gcc releases, back-patch to all supported branches.
Discussion: https://postgr.es/m/4496616.iHFcN1HehY@portable-bastien
M configure
M configure.ac
M meson.build
Add support for Tcl 9
commit : f979197eba77fc07c316cdb93436b3ac135222df
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 25 Nov 2024 08:03:16 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 25 Nov 2024 08:03:16 +0100
Tcl 9 changed several API functions to take Tcl_Size, which is
ptrdiff_t, instead of int, for 64-bit enablement. We have to change a
few local variables to be compatible with that. We also provide a
fallback typedef of Tcl_Size for older Tcl versions.
The affected variables are used for quantities that will not approach
values beyond the range of int, so this doesn't change any
functionality.
Reviewed-by: Tristan Partin <tristan@partin.io>
Discussion: https://www.postgresql.org/message-id/flat/bce0fe54-75b4-438e-b42b-8e84bc7c0e9c%40eisentraut.org
M src/pl/tcl/pltcl.c
Assume that <stdbool.h> conforms to the C standard.
commit : 0813a5e69186cb6705ce5b390d4488d8b783c8f2
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 25 Nov 2024 13:11:28 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 25 Nov 2024 13:11:28 +1300
Previously we checked "for <stdbool.h> that conforms to C99" using
autoconf's AC_HEADER_STDBOOL macro. We've required C99 since PostgreSQL
12, so the test was redundant, and under C23 it was broken: autoconf
2.69's implementation doesn't understand C23's new empty header (the
macros it's looking for went away, replaced by language keywords).
Later autoconf versions fixed that, but let's just remove the
anachronistic test.
HAVE_STDBOOL_H and HAVE__BOOL will no longer be defined, but they
weren't directly tested in core or likely extensions (except in 11, see
below). PG_USE_STDBOOL (or USE_STDBOOL in 11 and 12) is still defined
when sizeof(bool) is 1, which should be true on all modern systems.
Otherwise we define our own bool type and values of size 1, which would
fail to compile under C23 as revealed by the broken test. (We'll
probably clean that dead code up in master, but here we want a minimal
back-patchable change.)
This came to our attention when GCC 15 recently started using using C23
by default and failed to compile the replacement code, as reported by
Sam James and build farm animal alligator.
Back-patch to all supported releases, and then two older versions that
also know about <stdbool.h>, per the recently-out-of-support policy[1].
12 requires C99 so it's much like the supported releases, but 11 only
assumes C89 so it now uses AC_CHECK_HEADERS instead of the overly picky
AC_HEADER_STDBOOL. (I could find no discussion of which historical
systems had <stdbool.h> but failed the conformance test; if they ever
existed, they surely aren't relevant to that policy's goals.)
[1] https://wiki.postgresql.org/wiki/Committing_checklist#Policies
Reported-by: Sam James <sam@gentoo.org>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org> (master version)
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (approach)
Discussion: https://www.postgresql.org/message-id/flat/87o72eo9iu.fsf%40gentoo.org
M configure
M configure.ac
M meson.build
M src/include/c.h
M src/include/pg_config.h.in
Doc: Clarify the `inactive_since` field description.
commit : 5f46439d5931c518b670058e66849d7994f07588
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 25 Nov 2024 10:58:06 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 25 Nov 2024 10:58:06 +0530
Updated to specify that it represents the exact time a slot became
inactive, rather than the period of inactivity.
Reported-by: Peter Smith
Author: Bruce Momjian, Nisha Moond
Reviewed-by: Amit Kapila, Peter Smith
Backpatch-through: 17
Discussion: https://postgr.es/m/CAHut+PuvsyA5v8y7rYoY9mkDQzUhwaESM05yCByTMaDoRh30tA@mail.gmail.com
M doc/src/sgml/system-views.sgml
M src/backend/replication/logical/slotsync.c
M src/include/replication/slot.h
doc: Fix example with __next__() in PL/Python function
commit : 1a52069914ebf81ab8591f5b451fc9099529c211
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 25 Nov 2024 09:15:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 25 Nov 2024 09:15:41 +0900
Per PEP 3114, iterator.next() has been renamed to iterator.__next__(),
and one example in the documentation still used next(). This caused the
example provided to fail the function creation since Python 2 is not
supported anymore since 19252e8ec93.
Author: Erik Wienhold
Discussion: https://postgr.es/m/173209043143.2092749.13692266486972491694@wrigleys.postgresql.org
Backpatch-through: 15
M doc/src/sgml/plpython.sgml
Make the memory layout of Port struct independent of USE_OPENSSL
commit : e3b249c6d8f33227a5ee58f9b2d187e7ce71b4c7
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 17:43:04 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 17:43:04 +0200
Commit d39a49c1e4 added new fields to the struct, but missed the "keep
these last" comment on the previous fields. Add placeholder variables
so that the offsets of the fields are the same whether you build with
USE_OPENSSL or not. This is a courtesy to extensions that might peek
at the fields, to make the ABI the same regardless of the options used
to build PostgreSQL.
In reality, I don't expect any extensions to look at the 'raw_buf'
fields. Firstly, they are new in v17, so no one's written such
extensions yet. Secondly, extensions should have no business poking at
those fields anyway. Nevertheless, fix this properly on 'master'. On
v17, we mustn't change the memory layout, so just fix the comments.
Author: Jacob Champion
Discussion: https://www.postgresql.org/message-id/raw/CAOYmi%2BmKVJNzn5_TD_MK%3DhqO64r_w8Gb0FHCLk0oAkW-PJv8jQ@mail.gmail.com
M src/include/libpq/libpq-be.h
Fix data loss when restarting the bulk_write facility
commit : 9695835538c2c8e9cd0048028b8c85e1bbf5c79c
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 16:28:24 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 22 Nov 2024 16:28:24 +0200
If a user started a bulk write operation on a fork with existing data
to append data in bulk, the bulk_write machinery would zero out all
previously written pages up to the last page written by the new
bulk_write operation.
This is not an issue for PostgreSQL itself, because we never use the
bulk_write facility on a non-empty fork. But there are use cases where
it makes sense. TimescaleDB extension is known to do that to merge
partitions, for example.
Backpatch to v17, where the bulk_write machinery was introduced.
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reported-By: Erik Nordström <erik@timescale.com>
Reviewed-by: Erik Nordström <erik@timescale.com>
Discussion: https://www.postgresql.org/message-id/CACAa4VJ%2BQY4pY7M0ECq29uGkrOygikYtao1UG9yCDFosxaps9g@mail.gmail.com
M src/backend/storage/smgr/bulk_write.c
psql: Include \pset xheader_width in --help=commands|variables
commit : e6d6f2e46b57c0b89ecc6b9d37b571165f7c02a1
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 12:17:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 22 Nov 2024 12:17:49 +0900
psql's --help was missed the description of the \pset variable
xheader_width, that should be listed when using \? or --help=commands,
and described for --help=variables.
Oversight in a45388d6e098.
Author: Pavel Luzanov
Discussion: https://postgr.es/m/1e3e06d6-0807-4e62-a9f6-c11481e6eb10@postgrespro.ru
Backpatch-through: 16
M src/bin/psql/help.c
jit: Use -mno-outline-atomics for bitcode on ARM.
commit : 8a9a515188c8cdac3193d85d975eeaff872dd036
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 22 Nov 2024 14:53:21 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 22 Nov 2024 14:53:21 +1300
If the executable's .o files were produced by a compiler (probably gcc)
not using -moutline-atomics, and the corresponding .bc files were
produced by clang using -moutline-atomics (probably by default), then
the generated bitcode functions would have the target attribute
"+outline-atomics", and could fail at runtime when inlined. If the
target ISA at bitcode generation time was armv8-a (the most conservative
aarch64 target, no LSE), then LLVM IR atomic instructions would generate
calls to functions in libgcc.a or libclang_rt.*.a that switch between
LL/SC and faster LSE instructions depending on a runtime AT_HWCAP check.
Since the corresponding .o files didn't need those functions, they
wouldn't have been included in the executable, and resolution would
fail.
At least Debian and Ubuntu are known to ship gcc and clang compilers
that target armv8-a but differ on the use of outline atomics by default.
Fix, by suppressing the outline atomics attribute in bitcode explicitly.
Inline LL/SC instructions will be generated for atomic operations in
bitcode built for armv8-a. Only configure scripts are adjusted for now,
because the meson build system doesn't generate bitcode yet.
This doesn't seem to be a new phenomenon, so real cases of functions
using atomics that are inlined by JIT must be rare in the wild given how
long it took for a bug report to arrive. The reported case could be
reduced to:
postgres=# set jit_inline_above_cost = 0;
SET
postgres=# set jit_above_cost = 0;
SET
postgres=# select pg_last_wal_receive_lsn();
WARNING: failed to resolve name __aarch64_swp4_acq_rel
FATAL: fatal llvm error: Program used external function
'__aarch64_swp4_acq_rel' which could not be resolved!
The change doesn't affect non-ARM systems or later target ISAs.
Back-patch to all supported releases.
Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru>
Discussion: https://postgr.es/m/18610-37bf303f904fede3%40postgresql.org
M configure
M configure.ac
Fix newly introduced 010_keep_recycled_wals.pl
commit : 2fb3919356fb94dfef37998bf0439cc6896c93d4
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 17:04:26 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 17:04:26 +0100
It failed to set the archive_command as it desired because of a syntax
problem. Oversight in commit 90bcc7c2db1d.
This bug doesn't cause the test to fail, because the test only checks
pg_rewind's output messages, not the actual outcome (and the outcome in
both cases is that the file is kept, not deleted). But in either case
the message about the file being kept is there, so it's hard to get
excited about doing much more.
Reported-by: Antonin Houska <ah@cybertec.at>
Author: Alexander Kukushkin <cyberdemn@gmail.com>
Discussion: https://postgr.es/m/7822.1732167825@antos
M src/bin/pg_rewind/t/010_keep_recycled_wals.pl
Fix outdated bit in README.tuplock
commit : e2b08a6295826a0dd8ee905e9f8d2c8940fca049
author : Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 16:54:36 +0100
committer: Álvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 21 Nov 2024 16:54:36 +0100
Apparently this information has been outdated since first committed,
because we adopted a different implementation during development per
reviews and this detail was not updated in the README.
This has been wrong since commit 0ac5ad5134f2 introduced the file in
2013. Backpatch to all live branches.
Reported-by: Will Mortensen <will@extrahop.com>
Discussion: https://postgr.es/m/CAMpnoC6yEQ=c0Rdq-J7uRedrP7Zo9UMp6VZyP23QMT68n06cvA@mail.gmail.com
M src/backend/access/heap/README.tuplock
Fix memory leak in pgoutput for the WAL sender
commit : afe9b0d9fee1b4b02593ee4563653198771558cf
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Nov 2024 15:14:11 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 21 Nov 2024 15:14:11 +0900
RelationSyncCache, the hash table in charge of tracking the relation
schemas sent through pgoutput, was forgetting to free the TupleDesc
associated to the two slots used to store the new and old tuples,
causing some memory to be leaked each time a relation is invalidated
when the slots of an existing relation entry are cleaned up.
This is rather hard to notice as the bloat is pretty minimal, but a
long-running WAL sender would be in trouble over time depending on the
workload. sysbench has proved to be pretty good at showing the problem,
coupled with some memory monitoring of the WAL sender.
Issue introduced in 52e4f0cd472d, that has added row filters for tables
logically replicated.
Author: Boyu Yang
Reviewed-by: Michael Paquier, Hou Zhijie
Discussion: https://postgr.es/m/DM3PR84MB3442E14B340E553313B5C816E3252@DM3PR84MB3442.NAMPRD84.PROD.OUTLOOK.COM
Backpatch-through: 15
M src/backend/replication/pgoutput/pgoutput.c
doc: clarify that jsonb_path_match() returns an SQL boolean
commit : 7eff3138fc86cbea556c63a902fd3fcbf04bdf1d
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 17:03:45 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 20 Nov 2024 17:03:45 -0500
Not a JSON boolean. Also clarify that other predicate check expressions
functions return a JSON boolean, not an SQL boolean.
Reported-by: jian he
Discussion: https://postgr.es/m/CACJufxH7tP1NXCHN1bUBXcEB=dv7-qE+ZjB3UxwK6Em+9Qzb9Q@mail.gmail.com
Backpatch-through: 17
M doc/src/sgml/func.sgml
Avoid assertion failure if a setop leaf query contains setops.
commit : fea81aee83b50c514c4ba6789e22c22c028e3df2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Nov 2024 12:03:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Nov 2024 12:03:47 -0500
Ordinarily transformSetOperationTree will collect all UNION/
INTERSECT/EXCEPT steps into the setOperations tree of the topmost
Query, so that leaf queries do not contain any setOperations.
However, it cannot thus flatten a subquery that also contains
WITH, ORDER BY, FOR UPDATE, or LIMIT. I (tgl) forgot that in
commit 07b4c48b6 and wrote an assertion in rule deparsing that
a leaf's setOperations would always be empty.
If it were nonempty then we would want to parenthesize the subquery
to ensure that the output represents the setop nesting correctly
(e.g. UNION below INTERSECT had better get parenthesized). So
rather than just removing the faulty Assert, let's change it into
an additional case to check to decide whether to add parens. We
don't expect that the additional case will ever fire, but it's
cheap insurance.
Man Zeng and Tom Lane
Discussion: https://postgr.es/m/tencent_7ABF9B1F23B0C77606FC5FE3@qq.com
M src/backend/utils/adt/ruleutils.c
doc: Fix section of functions age(xid) and mxid_age(xid)
commit : 6fc3c587b8556c27c90998bd98259279dc794f33
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Nov 2024 14:21:07 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Nov 2024 14:21:07 +0900
In 17~, age(xid) and mxid_age(xid) were listed as deprecated. Based on
the discussion that led to 48b5aa3143, this is not intentional as this
could break many existing monitoring queries. Note that vacuumdb also
uses both of them.
In 16, both functions were listed under "Control Data Functions", which
is incorrect, so let's move them to the list of functions related to
transaction IDs and snapshots.
Author: Bertrand Drouvot
Discussion: https://postgr.es/m/Zzr2zZFyeFKXWe8a@ip-10-97-1-34.eu-west-3.compute.internal
Discussion: https://postgr.es/m/20231114013224.4z6oxa6p6va33rxr@awork3.anarazel.de
Backpatch-through: 16
M doc/src/sgml/func.sgml
Compare collations before merging UNION operations.
commit : c1ebef3c10dbfd8c6ec4c9b9ab03b7d1fe00ac79
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2024 18:26:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Nov 2024 18:26:19 -0500
In the dim past we figured it was okay to ignore collations
when combining UNION set-operation nodes into a single N-way
UNION operation. I believe that was fine at the time, but
it stopped being fine when we added nondeterministic collations:
the semantics of distinct-ness are affected by those. v17 made
it even less fine by allowing per-child sorting operations to
be merged via MergeAppend, although I think we accidentally
avoided any live bug from that.
Add a check that collations match before deciding that two
UNION nodes are equivalent. I also failed to resist the
temptation to comment plan_union_children() a little better.
Back-patch to all supported branches (v13 now), since they
all have nondeterministic collations.
Discussion: https://postgr.es/m/3605568.1731970579@sss.pgh.pa.us
M src/backend/optimizer/prep/prepunion.c