Stamp 13.17.
commit : 64ecc00908b7557afa911c15bb342ff06845bb19
author : Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 17:48:40 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 17:48:40 -0500
M configure
M configure.in
Last-minute updates for release notes.
commit : b3db5fdb05b6ca499f75d0ecf290dbd057bfb075
author : Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 17:40:13 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 17:40:13 -0500
Security: CVE-2024-10976, CVE-2024-10977, CVE-2024-10978, CVE-2024-10979
M doc/src/sgml/release-13.sgml
Parallel workers use AuthenticatedUserId for connection privilege checks.
commit : dc7378793add3c3d9a40ec2118d92bd719acab97
author : Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 17:05:53 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 17:05:53 -0500
Commit 5a2fed911 had an unexpected side-effect: the parallel worker
launched for the new test case would fail if it couldn't use a
superuser-reserved connection slot. The reason that test failed
while all our pre-existing ones worked is that the connection
privilege tests in InitPostgres had been based on the superuserness
of the leader's AuthenticatedUserId, but after the rearrangements
of 5a2fed911 we were testing the superuserness of CurrentUserId,
which the new test case deliberately made to be a non-superuser.
This all seems very accidental and probably not the behavior we really
want, but a security patch is no time to be redesigning things.
Pending some discussion about desirable semantics, hack it so that
InitPostgres continues to pay attention to the superuserness of
AuthenticatedUserId when starting a parallel worker.
Nathan Bossart and Tom Lane, per buildfarm member sawshark.
Security: CVE-2024-10978
M src/backend/utils/init/postinit.c
Fix cross-version upgrade tests.
commit : 0bd9560d964abc09e446e4c5e264bb7a0886e5ea
author : Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 13:57:21 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 13:57:21 -0500
TestUpgradeXversion knows how to make the main regression database's
references to pg_regress.so be version-independent. But it doesn't
do that for plperl's database, so that the C function added by
commit b7e3a52a8 is causing cross-version upgrade test failures.
Path of least resistance is to just drop the function at the end
of the new test.
In <= v14, also take the opportunity to clean up the generated
test files.
Security: CVE-2024-10979
M src/pl/plperl/GNUmakefile
M src/pl/plperl/input/plperl_env.source
M src/pl/plperl/output/plperl_env.source
src/tools/msvc: Respect REGRESS_OPTS in plcheck.
commit : 9f97c31c26a9704a6b2b92778a6e0b1d3606ff5d
author : Noah Misch <[email protected]>
date : Mon, 11 Nov 2024 10:55:18 -0800
committer: Noah Misch <[email protected]>
date : Mon, 11 Nov 2024 10:55:18 -0800
v16 commit 8fe3e697a1a83a722b107c7cb9c31084e1f4d077 used REGRESS_OPTS in
a way needing this. That broke "vcregress plcheck". Back-patch
v16..v12; newer versions don't have this build system.
M src/tools/msvc/vcregress.pl
Add needed .gitignore files in back branches.
commit : 6bccd7b037d09b91ce272c68f43705e2fecd4cca
author : Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 10:42:32 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 10:42:32 -0500
v14 and earlier use generated test files, which require being
.gitignore'd to avoid git complaints when testing in-tree.
Security: CVE-2024-10979
A src/pl/plperl/expected/.gitignore
A src/pl/plperl/sql/.gitignore
Fix improper interactions between session_authorization and role.
commit : 76123ded6e9b3624e380ac326645bd026aacd2f5
author : Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 10:29:54 -0500
committer: Tom Lane <[email protected]>
date : Mon, 11 Nov 2024 10:29:54 -0500
The SQL spec mandates that SET SESSION AUTHORIZATION implies
SET ROLE NONE. We tried to implement that within the lowest-level
functions that manipulate these settings, but that was a bad idea.
In particular, guc.c assumes that it doesn't matter in what order
it applies GUC variable updates, but that was not the case for these
two variables. This problem, compounded by some hackish attempts to
work around it, led to some security-grade issues:
* Rolling back a transaction that had done SET SESSION AUTHORIZATION
would revert to SET ROLE NONE, even if that had not been the previous
state, so that the effective user ID might now be different from what
it had been.
* The same for SET SESSION AUTHORIZATION in a function SET clause.
* If a parallel worker inspected current_setting('role'), it saw
"none" even when it should see something else.
Also, although the parallel worker startup code intended to cope
with the current role's pg_authid row having disappeared, its
implementation of that was incomplete so it would still fail.
Fix by fully separating the miscinit.c functions that assign
session_authorization from those that assign role. To implement the
spec's requirement, teach set_config_option itself to perform "SET
ROLE NONE" when it sets session_authorization. (This is undoubtedly
ugly, but the alternatives seem worse. In particular, there's no way
to do it within assign_session_authorization without incompatible
changes in the API for GUC assign hooks.) Also, improve
ParallelWorkerMain to directly set all the relevant user-ID variables
instead of relying on some of them to get set indirectly. That
allows us to survive not finding the pg_authid row during worker
startup.
In v16 and earlier, this includes back-patching 9987a7bf3 which
fixed a violation of GUC coding rules: SetSessionAuthorization
is not an appropriate place to be throwing errors from.
Security: CVE-2024-10978
M src/backend/access/transam/parallel.c
M src/backend/commands/variable.c
M src/backend/utils/init/miscinit.c
M src/backend/utils/misc/guc.c
M src/include/miscadmin.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Ensure cached plans are correctly marked as dependent on role.
commit : 952ff31e2a89e8ca79ecb12d61fddbeac3d89176
author : Nathan Bossart <[email protected]>
date : Mon, 11 Nov 2024 09:00:00 -0600
committer: Nathan Bossart <[email protected]>
date : Mon, 11 Nov 2024 09:00:00 -0600
If a CTE, subquery, sublink, security invoker view, or coercion
projection references a table with row-level security policies, we
neglected to mark the plan as potentially dependent on which role
is executing it. This could lead to later executions in the same
session returning or hiding rows that should have been hidden or
returned instead.
Reported-by: Wolfgang Walther
Reviewed-by: Noah Misch
Security: CVE-2024-10976
Backpatch-through: 12
M src/backend/executor/functions.c
M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql
M src/tools/pgindent/typedefs.list
Block environment variable mutations from trusted PL/Perl.
commit : e428cd058f0bebb5782b0c263565b0ad088e9650
author : Noah Misch <[email protected]>
date : Mon, 11 Nov 2024 06:23:43 -0800
committer: Noah Misch <[email protected]>
date : Mon, 11 Nov 2024 06:23:43 -0800
Many process environment variables (e.g. PATH), bypass the containment
expected of a trusted PL. Hence, trusted PLs must not offer features
that achieve setenv(). Otherwise, an attacker having USAGE privilege on
the language often can achieve arbitrary code execution, even if the
attacker lacks a database server operating system user.
To fix PL/Perl, replace trusted PL/Perl %ENV with a tied hash that just
replaces each modification attempt with a warning. Sites that reach
these warnings should evaluate the application-specific implications of
proceeding without the environment modification:
Can the application reasonably proceed without the modification?
If no, switch to plperlu or another approach.
If yes, the application should change the code to stop attempting
environment modifications. If that's too difficult, add "untie
%main::ENV" in any code executed before the warning. For example,
one might add it to the start of the affected function or even to
the plperl.on_plperl_init setting.
In passing, link to Perl's guidance about the Perl features behind the
security posture of PL/Perl.
Back-patch to v12 (all supported versions).
Andrew Dunstan and Noah Misch
Security: CVE-2024-10979
M doc/src/sgml/plperl.sgml
M src/pl/plperl/GNUmakefile
A src/pl/plperl/input/plperl_env.source
A src/pl/plperl/output/plperl_env.source
M src/pl/plperl/plc_trusted.pl
M src/test/regress/regress.c
Translation updates
commit : 706a96c437a39eae6b14e5483cc3c32dd9988408
author : Peter Eisentraut <[email protected]>
date : Mon, 11 Nov 2024 13:58:30 +0100
committer: Peter Eisentraut <[email protected]>
date : Mon, 11 Nov 2024 13:58:30 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: be7f3c3a26b382c9d7c9d32c7a972e452b56f529
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/es.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ru.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/fr.po
M src/bin/pg_archivecleanup/po/ru.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/fr.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/fr.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/fr.po
M src/bin/pg_controldata/po/ru.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_resetwal/po/fr.po
M src/bin/pg_resetwal/po/ru.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/fr.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_test_timing/po/fr.po
M src/bin/pg_test_timing/po/ru.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/fr.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/fr.po
M src/bin/pg_waldump/po/ru.po
M src/bin/psql/po/de.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ja.po
M src/bin/psql/po/ru.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ru.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/fr.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/fr.po
M src/interfaces/libpq/po/ja.po
M src/interfaces/libpq/po/ru.po
M src/pl/plperl/po/es.po
M src/pl/plperl/po/fr.po
M src/pl/plperl/po/ru.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/fr.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/fr.po
M src/pl/tcl/po/es.po
M src/pl/tcl/po/fr.po
libpq: Bail out during SSL/GSS negotiation errors
commit : 7b49707b72612ef068ce9275b9b6da104f1960f3
author : Michael Paquier <[email protected]>
date : Mon, 11 Nov 2024 10:20:02 +0900
committer: Michael Paquier <[email protected]>
date : Mon, 11 Nov 2024 10:20:02 +0900
This commit changes libpq so that errors reported by the backend during
the protocol negotiation for SSL and GSS are discarded by the client, as
these may include bytes that could be consumed by the client and write
arbitrary bytes to a client's terminal.
A failure with the SSL negotiation now leads to an error immediately
reported, without a retry on any other methods allowed, like a fallback
to a plaintext connection.
A failure with GSS discards the error message received, and we allow a
fallback as it may be possible that the error is caused by a connection
attempt with a pre-11 server, GSS encryption having been introduced in
v12. This was a problem only with v17 and newer versions; older
versions discard the error message already in this case, assuming a
failure caused by a lack of support for GSS encryption.
Author: Jacob Champion
Reviewed-by: Peter Eisentraut, Heikki Linnakangas, Michael Paquier
Security: CVE-2024-10977
Backpatch-through: 12
M doc/src/sgml/protocol.sgml
M src/interfaces/libpq/fe-connect.c
Release notes for 17.1, 16.5, 15.9, 14.14, 13.17, 12.21.
commit : 088692b3cd3c56ddde25129f3a0087eee7ca3d94
author : Tom Lane <[email protected]>
date : Sun, 10 Nov 2024 13:40:41 -0500
committer: Tom Lane <[email protected]>
date : Sun, 10 Nov 2024 13:40:41 -0500
M doc/src/sgml/release-13.sgml
Improve fix for not entering parallel mode when holding interrupts.
commit : 62685876f7b52030a5e17e4f8e54b038b2a082f2
author : Tom Lane <[email protected]>
date : Fri, 8 Nov 2024 13:42:01 -0500
committer: Tom Lane <[email protected]>
date : Fri, 8 Nov 2024 13:42:01 -0500
Commit ac04aa84a put the shutoff for this into the planner, which is
not ideal because it doesn't prevent us from re-using a previously
made parallel plan. Revert the planner change and instead put the
shutoff into InitializeParallelDSM, modeling it on the existing code
there for recovering from failure to allocate a DSM segment.
However, that code path is mostly untested, and testing a bit harder
showed there's at least one bug: ExecHashJoinReInitializeDSM is not
prepared for us to have skipped doing parallel DSM setup. I also
thought the Assert in ReinitializeParallelWorkers is pretty
ill-advised, and replaced it with a silent Min() operation.
The existing test case added by ac04aa84a serves fine to test this
version of the fix, so no change needed there.
Patch by me, but thanks to Noah Misch for the core idea that we
could shut off worker creation when !INTERRUPTS_CAN_BE_PROCESSED.
Back-patch to v12, as ac04aa84a was.
Discussion: https://postgr.es/m/CAC-SaSzHUKT=vZJ8MPxYdC_URPfax+yoA1hKTcF4ROz_Q6z0_Q@mail.gmail.com
M src/backend/access/transam/parallel.c
M src/backend/executor/nodeHashjoin.c
M src/backend/optimizer/plan/planner.c
Disallow partitionwise join when collations don't match
commit : 054701a2b77a2d99d1d2c73bfdc970319c592d16
author : Amit Langote <[email protected]>
date : Fri, 8 Nov 2024 16:30:44 +0900
committer: Amit Langote <[email protected]>
date : Fri, 8 Nov 2024 16:30:44 +0900
If the collation of any join key column doesn’t match the collation of
the corresponding partition key, partitionwise joins can yield incorrect
results. For example, rows that would match under the join key collation
might be located in different partitions due to the partitioning
collation. In such cases, a partitionwise join would yield different
results from a non-partitionwise join, so disallow it in such cases.
Reported-by: Tender Wang <[email protected]>
Author: Jian He <[email protected]>
Reviewed-by: Tender Wang <[email protected]>
Reviewed-by: Junwang Zhao <[email protected]>
Discussion: https://postgr.es/m/CAHewXNno_HKiQ6PqyLYfuqDtwp7KKHZiH1J7Pqyz0nr+PS2Dwg@mail.gmail.com
Backpatch-through: 12
M src/backend/optimizer/util/relnode.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Disallow partitionwise grouping when collations don't match
commit : ff65f695c0d322365a7b6f2571e232197d962a91
author : Amit Langote <[email protected]>
date : Fri, 8 Nov 2024 16:06:12 +0900
committer: Amit Langote <[email protected]>
date : Fri, 8 Nov 2024 16:06:12 +0900
If the collation of any grouping column doesn’t match the collation of
the corresponding partition key, partitionwise grouping can yield
incorrect results. For example, rows that would be grouped under the
grouping collation may end up in different partitions under the
partitioning collation. In such cases, full partitionwise grouping
would produce results that differ from those without partitionwise
grouping, so disallowed that.
Partial partitionwise aggregation is still allowed, as the Finalize
step reconciles partition-level aggregates with grouping requirements
across all partitions, ensuring that the final output remains
consistent.
This commit also fixes group_by_has_partkey() by ensuring the
RelabelType node is stripped from grouping expressions when matching
them to partition key expressions to avoid false mismatches.
Bug: #18568
Reported-by: Webbo Han <[email protected]>
Author: Webbo Han <[email protected]>
Reviewed-by: Tender Wang <[email protected]>
Reviewed-by: Aleksander Alekseev <[email protected]>
Reviewed-by: Jian He <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/[email protected]
Discussion: https://postgr.es/m/CAHewXNno_HKiQ6PqyLYfuqDtwp7KKHZiH1J7Pqyz0nr+PS2Dwg@mail.gmail.com
Backpatch-through: 12
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Message style improvement
commit : ebbfa2ae34ce85d09fe9ea87b41f9186864c7075
author : Peter Eisentraut <[email protected]>
date : Fri, 8 Nov 2024 07:17:55 +0100
committer: Peter Eisentraut <[email protected]>
date : Fri, 8 Nov 2024 07:17:55 +0100
Backpatch the part of edee0c621de that applies to a90bdd7a44d, which
was also backpatched. That way, the message is consistent in all
branches.
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
Fix lstat() for broken junction points on Windows.
commit : bb509a464e3e59e13b8869665fe5eccc98f83b39
author : Thomas Munro <[email protected]>
date : Tue, 25 Oct 2022 15:20:00 +1300
committer: Andrew Dunstan <[email protected]>
date : Tue, 25 Oct 2022 15:20:00 +1300
When using junction points to emulate symlinks on Windows, one edge case
was not handled correctly by commit c5cb8f3b: if a junction point is
broken (pointing to a non-existent path), we'd report ENOENT. This
doesn't break any known use case, but was noticed while developing a
test suite for these functions and is fixed here for completeness.
Also add translation ERROR_CANT_RESOLVE_FILENAME -> ENOENT, as that is
one of the errors Windows can report for some kinds of broken paths.
Discussion: https://postgr.es/m/CA%2BhUKG%2BajSQ_8eu2AogTncOnZ5me2D-Cn66iN_-wZnRjLN%2Bicg%40mail.gmail.com
(cherry picked from commit 387803d81d6256fcb60b9192bb5b00042442b4e3)
Author: Thomas Munro <[email protected]>
Author: Alexandra Wang <[email protected]>
M src/port/win32error.c
M src/port/win32stat.c
Provide lstat() for Windows.
commit : ee219102d2e76d3e7277ac1a7ddda7757737b31b
author : Thomas Munro <[email protected]>
date : Sat, 6 Aug 2022 12:00:57 +1200
committer: Andrew Dunstan <[email protected]>
date : Sat, 6 Aug 2022 12:00:57 +1200
Junction points will be reported with S_ISLNK(x.st_mode), simulating
POSIX lstat(). stat() will follow pseudo-symlinks, like in POSIX (but
only one level before giving up, unlike in POSIX).
This completes a TODO left by commit bed90759fcb.
Tested-by: Andrew Dunstan <[email protected]> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKGLfOOeyZpm5ByVcAt7x5Pn-%3DxGRNCvgiUPVVzjFLtnY0w%40mail.gmail.com
(cherry picked from commit c5cb8f3b770c043509b61528664bcd805e1777e6)
Author: Thomas Munro <[email protected]>
Author: Alexandra Wang <[email protected]>
M src/include/port/win32_port.h
M src/port/win32stat.c
Make unlink() work for junction points on Windows.
commit : fa56aa23fad3c27f4553025206afa8ba2500347f
author : Thomas Munro <[email protected]>
date : Sat, 6 Aug 2022 12:01:42 +1200
committer: Andrew Dunstan <[email protected]>
date : Sat, 6 Aug 2022 12:01:42 +1200
To support harmonization of Windows and Unix code, teach our unlink()
wrapper that junction points need to be unlinked with rmdir() on
Windows.
Tested-by: Andrew Dunstan <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLfOOeyZpm5ByVcAt7x5Pn-%3DxGRNCvgiUPVVzjFLtnY0w%40mail.gmail.com
(cherry picked from commit f357233c9db8be2a015163da8e1ab0630f444340)
Author: Thomas Munro <[email protected]>
Author: Alexandra Wang <[email protected]>
M src/port/dirmod.c
Add missing include guard to win32ntdll.h.
commit : 5c0b7581ba12bb01ebebce60583e3db6c0511057
author : Thomas Munro <[email protected]>
date : Wed, 12 Jan 2022 10:11:50 +1300
committer: Andrew Dunstan <[email protected]>
date : Wed, 12 Jan 2022 10:11:50 +1300
Oversight in commit e2f0f8ed. Also add this file to the exclusion lists
in headerscheck and cpluscpluscheck, because Unix systems don't have a
header it includes.
Reported-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/2760528.1641929756%40sss.pgh.pa.us
(cherry picked from commit af9e6331aeba149c93052c3549140082a85a3cf9)
Author: Thomas Munro <[email protected]>
Author: Alexandra Wang <[email protected]>
M src/include/port/win32ntdll.h
M src/tools/pginclude/cpluspluscheck
M src/tools/pginclude/headerscheck
Check for STATUS_DELETE_PENDING on Windows.
commit : 56b39cce778f93cd95a01df0da083e937424662d
author : Thomas Munro <[email protected]>
date : Fri, 10 Dec 2021 16:13:14 +1300
committer: Andrew Dunstan <[email protected]>
date : Fri, 10 Dec 2021 16:13:14 +1300
1. Update our open() wrapper to check for NT's STATUS_DELETE_PENDING
and translate it to Unix-like errors. This is done with
RtlGetLastNtStatus(), which is dynamically loaded from ntdll. A new
file win32ntdll.c centralizes lookup of NT functions, in case we decide
to add more in the future.
2. Remove non-working code that was trying to do something similar for
stat(), and just reuse the open() wrapper code. As a side effect,
stat() also gains resilience against "sharing violation" errors.
3. Since stat() is used very early in process startup, remove the
requirement that the Win32 signal event has been created before
pgwin32_open_handle() is reached. Instead, teach pg_usleep() to fall
back to a non-interruptible sleep if reached before the signal event is
available.
This could be back-patched, but for now it's in master only. The
problem has apparently been with us for a long time and generated only a
few complaints. Proposed patches trigger it more often, which led to
this investigation and fix.
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Alexander Lakhin <[email protected]>
Reviewed-by: Juan José Santamaría Flecha <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGJz_pZTF9mckn6XgSv69%2BjGwdgLkxZ6b3NWGLBCVjqUZA%40mail.gmail.com
(cherry picked from commit e2f0f8ed251d02c1eda79e1ca3cb3db2681e7a86)
Author: Thomas Munro <[email protected]>
Author: Alexandra Wang <[email protected]>
M configure
M configure.in
M src/backend/port/win32/signal.c
M src/include/port.h
A src/include/port/win32ntdll.h
M src/port/open.c
A src/port/win32ntdll.c
M src/port/win32stat.c
M src/tools/msvc/Mkvcbuild.pm
Disable clang 16's -Wcast-function-type-strict.
commit : b4363fc66e642b70f88455004e5bc6d67c65cf71
author : Thomas Munro <[email protected]>
date : Tue, 13 Dec 2022 10:03:28 +1300
committer: Andrew Dunstan <[email protected]>
date : Tue, 13 Dec 2022 10:03:28 +1300
Clang 16 is still in development, but seawasp reveals that it has
started warning about many of our casts of function pointers (those
introduced by commit 1c27d16e, and some older ones). Disable the new
warning for now, since otherwise buildfarm animal seawasp fails, and we
have no current plans to change our strategy for these callback function
types.
May be back-patched with other Clang/LLVM 16 changes around release
time.
Discussion: https://postgr.es/m/CA%2BhUKGJvX%2BL3aMN84ksT-cGy08VHErRNip3nV-WmTx7f6Pqhyw%40mail.gmail.com
(cherry picked from commit 101c37cd342a3ae134bb3e5e0abb14ae46692b56)
Author: Thomas Munro <[email protected]>
Author: Alexandra Wang <[email protected]>
M configure
M configure.in
A meson.build
Fix -Wcast-function-type warnings
commit : a5abacecb46358a7b771841e2ee0acbb1c353b79
author : Peter Eisentraut <[email protected]>
date : Tue, 14 Jul 2020 19:36:30 +0200
committer: Andrew Dunstan <[email protected]>
date : Tue, 14 Jul 2020 19:36:30 +0200
Three groups of issues needed to be addressed:
load_external_function() and related functions returned PGFunction,
even though not necessarily all callers are looking for a function of
type PGFunction. Since these functions are really just wrappers
around dlsym(), change to return void * just like dlsym().
In dynahash.c, we are using strlcpy() where a function with a
signature like memcpy() is expected. This should be safe, as the new
comment there explains, but the cast needs to be augmented to avoid
the warning.
In PL/Python, methods all need to be cast to PyCFunction, per Python
API, but this now runs afoul of these warnings. (This issue also
exists in core CPython.)
To fix the second and third case, we add a new type pg_funcptr_t that
is defined specifically so that gcc accepts it as a special function
pointer that can be cast to any other function pointer without the
warning.
Also add -Wcast-function-type to the standard warning flags, subject
to configure check.
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/1e97628e-6447-b4fd-e230-d109cec2d584%402ndquadrant.com
(cherry picked from commit de8feb1f3a23465b5737e8a8c160e8ca62f61339)
Author: Peter Eisentraut <[email protected]>
Author: Alexandra Wang <[email protected]>
M configure
M configure.in
M src/backend/utils/fmgr/dfmgr.c
M src/backend/utils/hash/dynahash.c
M src/include/c.h
M src/include/fmgr.h
M src/pl/plpython/plpy_plpymodule.c
Fix issues with Windows' stat() for files pending on deletion
commit : f1cf64167fc9a09f54f69ec1d82865ba6aca5fe6
author : Michael Paquier <[email protected]>
date : Mon, 12 Jul 2021 13:02:31 +0900
committer: Andrew Dunstan <[email protected]>
date : Mon, 12 Jul 2021 13:02:31 +0900
The code introduced by bed9075 to enhance the stat() implementation on
Windows for file sizes larger than 4GB fails to properly detect files
pending for deletion with its method based on NtQueryInformationFile()
or GetFileInformationByHandleEx(), as proved by Alexander Lakhin in a
custom TAP test of his own.
The method used in the implementation of open() to sleep and loop when
when failing on ERROR_ACCESS_DENIED (EACCES) is showing much more
stability, so switch to this method. This could still lead to issues if
the permission problem stays around for much longer than the timeout of
1 second used, but that should (hopefully) never happen in
performance-critical paths. Still, there could be a point in increasing
the timeouts for the sake of machines that handle heavy loads.
Note that WIN32's open() now uses microsoft_native_stat() as it should
be similar to stat() when working around issues with concurrent file
deletions.
I have spent some time testing this patch with pgbench in combination
of the SQL functions from genfile.c, as well as running the TAP test
provided on the thread with MSVC builds, and this looks much more
stable than the previous method.
Author: Alexander Lakhin
Reviewed-by: Tom Lane, Michael Paquier, Justin Pryzby
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 14
(cherry picked from commit 54fb8c7ddf152629021cab3ac3596354217b7d81)
Author: Alexandra Wang <[email protected]>
M src/port/open.c
M src/port/win32stat.c
Fix our Windows stat() emulation to handle file sizes > 4GB.
commit : a9beed67670e680edeadd2a3cf7557a3c9808adf
author : Tom Lane <[email protected]>
date : Fri, 9 Oct 2020 16:20:12 -0400
committer: Andrew Dunstan <[email protected]>
date : Fri, 9 Oct 2020 16:20:12 -0400
Hack things so that our idea of "struct stat" is equivalent to Windows'
struct __stat64, allowing it to have a wide enough st_size field.
Instead of relying on native stat(), use GetFileInformationByHandle().
This avoids a number of issues with Microsoft's multiple and rather
slipshod emulations of stat(). We still need to jump through hoops
to deal with ERROR_DELETE_PENDING, though :-(
Pull the relevant support code out of dirmod.c and put it into
its own file, win32stat.c.
Still TODO: do we need to do something different with lstat(),
rather than treating it identically to stat()?
Juan José Santamaría Flecha, reviewed by Emil Iggland;
based on prior work by Michael Paquier, Sergey Zubkovsky, and others
Discussion: https://postgr.es/m/1803D792815FC24D871C00D17AE95905CF5099@g01jpexmbkw24
Discussion: https://postgr.es/m/[email protected]
(cherry picked from commit bed90759fcbcd72d4d06969eebab81e47326f9a2)
Author: Alexandra Wang <[email protected]>
M configure
M configure.in
M src/include/port/win32_port.h
M src/port/dirmod.c
A src/port/win32stat.c
M src/tools/msvc/Mkvcbuild.pm
doc: Reword ALTER TABLE ATTACH restriction on NO INHERIT constraints
commit : f534e38914eb89b86fca8294c03d21ff72fd0719
author : Álvaro Herrera <[email protected]>
date : Thu, 7 Nov 2024 14:06:24 +0100
committer: Álvaro Herrera <[email protected]>
date : Thu, 7 Nov 2024 14:06:24 +0100
The previous wording is easy to read incorrectly; this change makes it
simpler, less ambiguous, and less prominent.
Backpatch to all live branches.
Reviewed-by: Amit Langote <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_table.sgml
Monkey-patch LLVM code to fix ARM relocation bug.
commit : e88d824a41278af98c208df320051bc1834d3937
author : Thomas Munro <[email protected]>
date : Wed, 6 Nov 2024 22:04:44 +1300
committer: Thomas Munro <[email protected]>
date : Wed, 6 Nov 2024 22:04:44 +1300
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <[email protected]>
Co-authored-by: Anthonin Bonnefoy <[email protected]>
Reviewed-by: Anthonin Bonnefoy <[email protected]>
Reviewed-by: Daniel Gustafsson <[email protected]> (license aspects)
Reported-by: Anthonin Bonnefoy <[email protected]>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
M src/backend/jit/llvm/Makefile
A src/backend/jit/llvm/SectionMemoryManager.LICENSE
A src/backend/jit/llvm/SectionMemoryManager.cpp
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
A src/include/jit/SectionMemoryManager.h
M src/include/jit/llvmjit.h
A src/include/jit/llvmjit_backport.h
M src/tools/pginclude/headerscheck
Suppress new "may be used uninitialized" warning.
commit : 4dc0c933f30bf5db5aa07acee6ff5f1e2f16401e
author : Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 19:42:52 -0700
committer: Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 19:42:52 -0700
Buildfarm member mamba fails to deduce that the function never uses this
variable without initializing it. Back-patch to v12, like commit
b412f402d1e020c5dac94f3bf4a005db69519b99.
M src/backend/catalog/index.c
Move I/O before the index_update_stats() buffer lock region.
commit : 6b01cac0be6dd044fa3c2d08df7e893b96dfafe6
author : Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 09:04:55 -0700
committer: Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 09:04:55 -0700
Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 enlarged the work done
here under the pg_class heap buffer lock. Two preexisting actions are
best done before holding that lock. Both RelationGetNumberOfBlocks()
and visibilitymap_count() do I/O, and the latter might exclusive-lock a
visibility map buffer. Moving these reduces contention and risk of
undetected LWLock deadlock. Back-patch to v12, like that commit.
Discussion: https://postgr.es/m/[email protected]
M src/backend/catalog/index.c
Revert "For inplace update, send nontransactional invalidations."
commit : fe8091c9e39e65c8a0044349f0c0e7c3386ae921
author : Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 09:05:00 -0700
committer: Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 09:05:00 -0700
This reverts commit 95c5acb3fc261067ab65ddc0b2dca8e162f09442 (v17) and
counterparts in each other non-master branch. If released, that commit
would have caused a worst-in-years minor release regression, via
undetected LWLock self-deadlock. This commit and its self-deadlock fix
warrant more bake time in the master branch.
Reported by Alexander Lakhin.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/transam/xact.c
M src/backend/catalog/index.c
M src/backend/replication/logical/decode.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/syscache.c
M src/include/utils/catcache.h
M src/include/utils/inval.h
M src/test/isolation/expected/inplace-inval.out
M src/test/isolation/specs/inplace-inval.spec
Revert "WAL-log inplace update before revealing it to other sessions."
commit : be74b943c92054baa701f3aa5d789c09e3d914d8
author : Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 09:04:59 -0700
committer: Noah Misch <[email protected]>
date : Sat, 2 Nov 2024 09:04:59 -0700
This reverts commit bfd5c6e279c8e1702eea882439dc7ebdf4d4b3a5 (v17) and
counterparts in each other non-master branch. This unblocks reverting a
commit on which it depends.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/heapam.c
M src/backend/access/transam/xloginsert.c
doc: fix ALTER DOMAIN domain_constraint to spell out options
commit : f9b446428dca239cd591ce4b6393124e47d5c30c
author : Bruce Momjian <[email protected]>
date : Fri, 1 Nov 2024 13:54:27 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 1 Nov 2024 13:54:27 -0400
It used to refer to CREATE DOMAIN, but CREATE DOMAIN allows NULL, while
ALTER DOMAIN does not.
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/ref/alter_domain.sgml
doc: remove mention of ActiveState for Perl and Tcl on Windows
commit : 930d90a0c8d42434665243602a3596fe40895d44
author : Bruce Momjian <[email protected]>
date : Fri, 1 Nov 2024 11:30:53 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 1 Nov 2024 11:30:53 -0400
Replace with Strawberry Perl and Magicsplat Tcl.
Reported-by: Yasir Hussain
Discussion: https://postgr.es/m/CAA9OW9fAAM_WDYYpAquqF6j1hmfRMzHPsFkRfP5E6oSfkF=dMA@mail.gmail.com
Backpatch-through: 12
M doc/src/sgml/install-windows.sgml
Unpin buffer before inplace update waits for an XID to end.
commit : 2a912bc1abdbaa2f73555cf2c71bb4c401aa515b
author : Noah Misch <[email protected]>
date : Tue, 29 Oct 2024 09:39:55 -0700
committer: Noah Misch <[email protected]>
date : Tue, 29 Oct 2024 09:39:55 -0700
Commit a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 changed inplace updates
to wait for heap_update() commands like GRANT TABLE and GRANT DATABASE.
By keeping the pin during that wait, a sequence of autovacuum workers
and an uncommitted GRANT starved one foreground LockBufferForCleanup()
for six minutes, on buildfarm member sarus. Prevent, at the cost of a
bit of complexity. Back-patch to v12, like the earlier commit. That
commit and heap_inplace_lock() have not yet appeared in any release.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/index/genam.c
M src/include/access/heapam.h
Update time zone data files to tzdata release 2024b.
commit : 8a8486175042477c3ce17976ce384d430cd1530f
author : Tom Lane <[email protected]>
date : Tue, 29 Oct 2024 11:49:38 -0400
committer: Tom Lane <[email protected]>
date : Tue, 29 Oct 2024 11:49:38 -0400
Historical corrections for Mexico, Mongolia, and Portugal.
Notably, Asia/Choibalsan is now an alias for Asia/Ulaanbaatar
rather than being a separate zone, mainly because the differences
between those zones were found to be based on untrustworthy data.
M src/timezone/data/tzdata.zi
M src/timezone/known_abbrevs.txt
M src/timezone/tznames/Default
M src/timezone/tznames/Europe.txt
doc: Add better description for rewrite functions in event triggers
commit : 93a596c73517452002061cace9f6f4acea370041
author : Michael Paquier <[email protected]>
date : Tue, 29 Oct 2024 15:35:22 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 29 Oct 2024 15:35:22 +0900
There are two functions that can be used in event triggers to get more
details about a rewrite happening on a relation. Both had a limited
documentation:
- pg_event_trigger_table_rewrite_reason() and
pg_event_trigger_table_rewrite_oid() were not mentioned in the main
event trigger section in the paragraph dedicated to the event
table_rewrite.
- pg_event_trigger_table_rewrite_reason() returns an integer which is a
bitmap of the reasons why a rewrite happens. There was no explanation
about the meaning of these values, forcing the reader to look at the
code to find out that these are defined in event_trigger.h.
While on it, let's add a comment in event_trigger.h where the
AT_REWRITE_* are defined, telling to update the documentation when
these values are changed.
Backpatch down to 13 as a consequence of 1ad23335f36b, where this area
of the documentation has been heavily reworked.
Author: Greg Sabino Mullane
Discussion: https://postgr.es/m/CAKAnmmL+Z6j-C8dAx1tVrnBmZJu+BSoc68WSg3sR+CVNjBCqbw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/event-trigger.sgml
M doc/src/sgml/func.sgml
M src/include/commands/event_trigger.h
Doc: clarify enable_indexscan=off also disabled Index Only Scans
commit : ee8db41a94326e670255ad6c50b887c8c9c897f5
author : David Rowley <[email protected]>
date : Tue, 29 Oct 2024 16:26:18 +1300
committer: David Rowley <[email protected]>
date : Tue, 29 Oct 2024 16:26:18 +1300
Disabling enable_indexscan has always also disabled Index Only Scans.
Here we make that more clear in the documentation in an attempt to
prevent future complaints complaining about this expected behavior.
Reported-by: Melanie Plageman
Author: David G. Johnston, David Rowley
Backpatch-through: 12, oldest supported version
Discussion: https://postgr.es/m/CAAKRu_atV=kovgpaLREyG68PB5+ncKvJ2UNoeRetEgyC3Yb5Sw@mail.gmail.com
M doc/src/sgml/config.sgml
WAL-log inplace update before revealing it to other sessions.
commit : 1e74e31f3c93f01c2ddb1e2b25db23fb0533244c
author : Noah Misch <[email protected]>
date : Fri, 25 Oct 2024 06:51:03 -0700
committer: Noah Misch <[email protected]>
date : Fri, 25 Oct 2024 06:51:03 -0700
A buffer lock won't stop a reader having already checked tuple
visibility. If a vac_update_datfrozenid() and then a crash happened
during inplace update of a relfrozenxid value, datfrozenxid could
overtake relfrozenxid. That could lead to "could not access status of
transaction" errors. Back-patch to v12 (all supported versions). In
v14 and earlier, this also back-patches the assertion removal from
commit 7fcf2faf9c7dd473208fd6d5565f88d7f733782b.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/heapam.c
M src/backend/access/transam/xloginsert.c
For inplace update, send nontransactional invalidations.
commit : 0ea9d40a667960ad5cca5f537bb88d3bf6038379
author : Noah Misch <[email protected]>
date : Fri, 25 Oct 2024 06:51:02 -0700
committer: Noah Misch <[email protected]>
date : Fri, 25 Oct 2024 06:51:02 -0700
The inplace update survives ROLLBACK. The inval didn't, so another
backend's DDL could then update the row without incorporating the
inplace update. In the test this fixes, a mix of CREATE INDEX and ALTER
TABLE resulted in a table with an index, yet relhasindex=f. That is a
source of index corruption. Back-patch to v12 (all supported versions).
The back branch versions don't change WAL, because those branches just
added end-of-recovery SIResetAll(). All branches change the ABI of
extern function PrepareToInvalidateCacheTuple(). No PGXN extension
calls that, and there's no apparent use case in extensions.
Reviewed by Nitin Motiani and (in earlier versions) Andres Freund.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/backend/access/transam/xact.c
M src/backend/catalog/index.c
M src/backend/replication/logical/decode.c
M src/backend/utils/cache/catcache.c
M src/backend/utils/cache/inval.c
M src/backend/utils/cache/syscache.c
M src/include/utils/catcache.h
M src/include/utils/inval.h
M src/test/isolation/expected/inplace-inval.out
M src/test/isolation/specs/inplace-inval.spec
At end of recovery, reset all sinval-managed caches.
commit : 67f30c79a1c17417029b3c1cd1d94aa56d5fb691
author : Noah Misch <[email protected]>
date : Fri, 25 Oct 2024 06:51:06 -0700
committer: Noah Misch <[email protected]>
date : Fri, 25 Oct 2024 06:51:06 -0700
An inplace update's invalidation messages are part of its transaction's
commit record. However, the update survives even if its transaction
aborts or we stop recovery before replaying its transaction commit.
After recovery, a backend that started in recovery could update the row
without incorporating the inplace update. That could result in a table
with an index, yet relhasindex=f. That is a source of index corruption.
This bulk invalidation avoids the functional consequences. A future
change can fix the !RecoveryInProgress() scenario without changing the
WAL format. Back-patch to v17 - v12 (all supported versions). v18 will
instead add invalidations to WAL.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/transam/xlog.c
M src/backend/storage/ipc/sinvaladt.c
M src/include/storage/sinvaladt.h
Stop reading uninitialized memory in heap_inplace_lock().
commit : 3e5ea478d875c15e7ba47778568b2f7edb54a0f1
author : Noah Misch <[email protected]>
date : Thu, 24 Oct 2024 09:16:14 -0700
committer: Noah Misch <[email protected]>
date : Thu, 24 Oct 2024 09:16:14 -0700
Stop computing a never-used value. This removes the read; the read had
no functional implications. Back-patch to v12, like commit
a07e03fd8fa7daf4d1356f7cb501ffe784ea6257.
Reported by Alexander Lakhin.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/heapam.c
M src/test/isolation/expected/intra-grant-inplace.out
M src/test/isolation/specs/intra-grant-inplace.spec
Remove unnecessary word in a comment
commit : 884c1a4d17dc7e643940e928cca42ab8a325dd46
author : Amit Langote <[email protected]>
date : Wed, 23 Oct 2024 17:54:48 +0900
committer: Amit Langote <[email protected]>
date : Wed, 23 Oct 2024 17:54:48 +0900
Relations opened by the executor are only closed once in
ExecCloseRangeTableRelations(), so the word "again" in the comment
for ExecGetRangeTableRelation() is misleading and unnecessary.
Discussion: https://postgr.es/m/CA+HiwqHnw-zR+u060i3jp4ky5UR0CjByRFQz50oZ05de7wUg=Q@mail.gmail.com
Backpatch-through: 12
M src/backend/executor/execUtils.c
ecpg: Fix out-of-bound read in DecodeDateTime()
commit : fcafbaadf7e507708d7257282e1ddcee1cc4c5eb
author : Michael Paquier <[email protected]>
date : Wed, 23 Oct 2024 08:35:07 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 23 Oct 2024 08:35:07 +0900
It was possible for the code to read out-of-bound data from the
"day_tab" table with some crafted input data. Let's treat these as
invalid input as the month number is incorrect.
A test is added to test this case with a check on the errno returned by
the decoding routine. A test close to the new one added in this commit
was testing for a failure, but did not look at the errno generated, so
let's use this commit to also change it, adding a check on the errno
returned by DecodeDateTime().
Like the other test scripts, dt_test should likely be expanded to
include more checks based on the errnos generated in these code paths.
This is left as future work.
This issue exists since 2e6f97560a83, so backpatch all the way down.
Reported-by: Pavel Nekrasov
Author: Bruce Momjian, Pavel Nekrasov
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/interfaces/ecpg/pgtypeslib/dt_common.c
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.c
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.stderr
M src/interfaces/ecpg/test/expected/pgtypeslib-dt_test.stdout
M src/interfaces/ecpg/test/pgtypeslib/dt_test.pgc
Restructure foreign key handling code for ATTACH/DETACH
commit : d20194cead7576dc53fc19e54ea2ec8547bae6d2
author : Álvaro Herrera <[email protected]>
date : Tue, 22 Oct 2024 16:01:18 +0200
committer: Álvaro Herrera <[email protected]>
date : Tue, 22 Oct 2024 16:01:18 +0200
... to fix bugs when the referenced table is partitioned.
The catalog representation we chose for foreign keys connecting
partitioned tables (in commit f56f8f8da6af) is inconvenient, in the
sense that a standalone table has a different way to represent the
constraint when referencing a partitioned table, than when the same
table becomes a partition (and vice versa). Because of this, we need to
create additional catalog rows on detach (pg_constraint and pg_trigger),
and remove them on attach. We were doing some of those things, but not
all of them, leading to missing catalog rows in certain cases.
The worst problem seems to be that we are missing action triggers after
detaching a partition, which means that you could update/delete rows
from the referenced partitioned table that still had referencing rows on
that table, the server failing to throw the required errors.
!!!
Note that this means existing databases with FKs that reference
partitioned tables might have rows that break relational integrity, on
tables that were once partitions on the referencing side of the FK.
Another possible problem is that trying to reattach a table
that had been detached would fail indicating that internal triggers
cannot be found, which from the user's point of view is nonsensical.
In branches 15 and above, we fix this by creating a new helper function
addFkConstraint() which is in charge of creating a standalone
pg_constraint row, and repurposing addFkRecurseReferencing() and
addFkRecurseReferenced() so that they're only the recursive routine for
each side of the FK, and they call addFkConstraint() to create
pg_constraint at each partitioning level and add the necessary triggers.
These new routines can be used during partition creation, partition
attach and detach, and foreign key creation. This reduces redundant
code and simplifies the flow.
In branches 14 and 13, we have a much simpler fix that consists on
simply removing the constraint on detach. The reason is that those
branches are missing commit f4566345cf40, which reworked the way this
works in a way that we didn't consider back-patchable at the time.
We opted to leave branch 12 alone, because it's different from branch 13
enough that the fix doesn't apply; and because it is going in EOL mode
very soon, patching it now might be worse since there's no way to undo
the damage if it goes wrong.
Existing databases might need to be repaired.
In the future we might want to rethink the catalog representation to
avoid this problem, but for now the code seems to do what's required to
make the constraints operate correctly.
Co-authored-by: Jehan-Guillaume de Rorthais <[email protected]>
Co-authored-by: Tender Wang <[email protected]>
Co-authored-by: Alvaro Herrera <[email protected]>
Reported-by: Guillaume Lelarge <[email protected]>
Reported-by: Jehan-Guillaume de Rorthais <[email protected]>
Reported-by: Thomas Baehler (SBB CFF FFS) <[email protected]>
Discussion: https://postgr.es/m/20230420144344.40744130@karst
Discussion: https://postgr.es/m/20230705233028.2f554f73@karst
Discussion: https://postgr.es/m/GVAP278MB02787E7134FD691861635A8BC9032@GVAP278MB0278.CHEP278.PROD.OUTLOOK.COM
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Fix wrong assertion and poor error messages in "COPY (query) TO".
commit : beab395a42b24f2049ad09b1d90ee9f218743d04
author : Tom Lane <[email protected]>
date : Mon, 21 Oct 2024 15:08:22 -0400
committer: Tom Lane <[email protected]>
date : Mon, 21 Oct 2024 15:08:22 -0400
If the query is rewritten into a NOTIFY command by a DO INSTEAD
rule, we'd get an assertion failure, or in non-assert builds
issue a rather confusing error message. Improve that.
Also fix a longstanding grammar mistake in a nearby error message.
Per bug #18664 from Alexander Lakhin. Back-patch to all supported
branches.
Tender Wang and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/copy.c
M src/test/regress/expected/copydml.out
M src/test/regress/sql/copydml.sql
Fix race condition in committing a serializable transaction
commit : 8e607a5a4be54f2d2bdfb7a53584d4b7ca1d2be3
author : Heikki Linnakangas <[email protected]>
date : Mon, 21 Oct 2024 09:49:21 +0300
committer: Heikki Linnakangas <[email protected]>
date : Mon, 21 Oct 2024 09:49:21 +0300
The finished transaction list can contain XIDs that are older than the
serializable global xmin. It's a short-lived state;
ClearOldPredicateLocks() removes any such transactions from the list,
and it's called whenever the global xmin advances. But if another
backend calls SummarizeOldestCommittedSxact() in that window, it will
call SerialAdd() on an XID that's older than the global xmin, or if
there are no more transactions running, when global xmin is
invalid. That trips the assertion in SerialAdd().
Fixes bug #18658 reported by Andrew Bille. Thanks to Alexander Lakhin
for analysis. Backpatch to all versions.
Discussion: https://www.postgresql.org/message-id/18658-7dab125ec688c70b%40postgresql.org
M src/backend/storage/lmgr/predicate.c
Note that index_name in ALTER INDEX ATTACH PARTITION can be schema-qualified
commit : fde796eea922a488593ceb94b93a18dd17e60463
author : Álvaro Herrera <[email protected]>
date : Sun, 20 Oct 2024 15:36:20 +0200
committer: Álvaro Herrera <[email protected]>
date : Sun, 20 Oct 2024 15:36:20 +0200
Missed in 8b08f7d4820f; backpatch to all supported branches.
Reported-by: [email protected]
Reviewed-by: Tom Lane <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/alter_index.sgml
Fix extreme skew detection in Parallel Hash Join.
commit : 45329466fdc955b166de186281f191d09d3a1661
author : Thomas Munro <[email protected]>
date : Thu, 17 Oct 2024 15:52:24 +1300
committer: Thomas Munro <[email protected]>
date : Thu, 17 Oct 2024 15:52:24 +1300
After repartitioning the inner side of a hash join that would have
exceeded the allowed size, we check if all the tuples from a parent
partition moved to one child partition. That is evidence that it
contains duplicate keys and later attempts to repartition will also
fail, so we should give up trying to limit memory (for lack of a better
fallback strategy).
A thinko prevented the check from working correctly in partition 0 (the
one that is partially loaded into memory already). After
repartitioning, we should check for extreme skew if the *parent*
partition's space_exhausted flag was set, not the child partition's.
The consequence was repeated futile repartitioning until per-partition
data exceeded various limits including "ERROR: invalid DSA memory alloc
request size 1811939328", OS allocation failure, or temporary disk space
errors. (We could also do something about some of those symptoms, but
that's material for separate patches.)
This problem only became likely when PostgreSQL 16 introduced support
for Parallel Hash Right/Full Join, allowing NULL keys into the hash
table. Repartitioning always leaves NULL in partition 0, no matter how
many times you do it, because the hash value is all zero bits. That's
unlikely for other hashed values, but they might still have caused
wasted extra effort before giving up.
Back-patch to all supported releases.
Reported-by: Craig Milhiser <[email protected]>
Reviewed-by: Andrei Lepikhov <[email protected]>
Discussion: https://postgr.es/m/CA%2BwnhO1OfgXbmXgC4fv_uu%3DOxcDQuHvfoQ4k0DFeB0Qqd-X-rQ%40mail.gmail.com
M src/backend/executor/nodeHash.c
Further refine _SPI_execute_plan's rule for atomic execution.
commit : 0d83ced3c91b33839124825b3518c20b1e97df93
author : Tom Lane <[email protected]>
date : Wed, 16 Oct 2024 17:36:30 -0400
committer: Tom Lane <[email protected]>
date : Wed, 16 Oct 2024 17:36:30 -0400
Commit 2dc1deaea turns out to have been still a brick shy of a load,
because CALL statements executing within a plpgsql exception block
could still pass the wrong snapshot to stable functions within the
CALL's argument list. That happened because standard_ProcessUtility
forces isAtomicContext to true if IsTransactionBlock is true, which
it always will be inside a subtransaction. Then ExecuteCallStmt
would think it does not need to push a new snapshot --- but
_SPI_execute_plan didn't do so either, since it thought it was in
nonatomic mode.
The best fix for this seems to be for _SPI_execute_plan to operate
in atomic execution mode if IsSubTransaction() is true, even when the
SPI context as a whole is non-atomic. This makes _SPI_execute_plan
have the same rules about when non-atomic execution is allowed as
_SPI_commit/_SPI_rollback have about when COMMIT/ROLLBACK are allowed,
which seems appropriately symmetric. (If anyone ever tries to allow
COMMIT/ROLLBACK inside a subtransaction, this would all need to be
rethought ... but I'm unconvinced that such a thing could be logically
consistent at all.)
For further consistency, also check IsSubTransaction() in
SPI_inside_nonatomic_context. That does not matter for its
one present-day caller StartTransaction, which can't be reached
inside a subtransaction. But if any other callers ever arise,
they'd presumably want this definition.
Per bug #18656 from Alexander Alehin. Back-patch to all
supported branches, like previous fixes in this area.
Discussion: https://postgr.es/m/[email protected]
M src/backend/executor/spi.c
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Reduce memory block size for decoded tuple storage to 8kB.
commit : cb988b04d0637f5d55bfb5a01b23dd4c39beccfa
author : Masahiko Sawada <[email protected]>
date : Wed, 16 Oct 2024 12:07:52 -0700
committer: Masahiko Sawada <[email protected]>
date : Wed, 16 Oct 2024 12:07:52 -0700
Commit a4ccc1cef introduced the Generation Context and modified the
logical decoding process to use a Generation Context with a fixed
block size of 8MB for storing tuple data decoded during logical
decoding (i.e., rb->tup_context). Several reports have indicated that
the logical decoding process can be terminated due to
out-of-memory (OOM) situations caused by excessive memory usage in
rb->tup_context.
This issue can occur when decoding a workload involving several
concurrent transactions, including a long-running transaction that
modifies tuples. By design, the Generation Context does not free a
memory block until all chunks within that block are
released. Consequently, if tuples modified by the long-running
transaction are stored across multiple memory blocks, these blocks
remain allocated until the long-running transaction completes, leading
to substantial memory fragmentation. The memory usage during logical
decoding, tracked by rb->size, does not account for memory
fragmentation, resulting in potentially much higher memory consumption
than the value of the logical_decoding_work_mem parameter.
Various improvement strategies were discussed in the relevant
thread. This change reduces the block size of the Generation Context
used in rb->tup_context from 8MB to 8kB. This modification
significantly decreases the likelihood of substantial memory
fragmentation occurring and is relatively straightforward to
backport. Performance testing across multiple platforms has confirmed
that this change will not introduce any performance degradation that
would impact actual operation.
Backport to all supported branches.
Reported-by: Alex Richman, Michael Guissine, Avi Weinberg
Reviewed-by: Amit Kapila, Fujii Masao, David Rowley
Tested-by: Hayato Kuroda, Shlok Kyal
Discussion: https://postgr.es/m/CAD21AoBTY1LATZUmvSXEssvq07qDZufV4AF-OHh9VD2pC0VY2A%40mail.gmail.com
Backpatch-through: 12
M src/backend/replication/logical/reorderbuffer.c
Correctly identify which EC members are computable at a plan node.
commit : 76de4b182cad1e53e01a52c78cf24f4e13b36253
author : Tom Lane <[email protected]>
date : Sat, 12 Oct 2024 14:56:08 -0400
committer: Tom Lane <[email protected]>
date : Sat, 12 Oct 2024 14:56:08 -0400
find_computable_ec_member() had the wrong mental model of what
its primary caller prepare_sort_from_pathkeys() would do with
the selected EquivalenceClass member expression. We will not
compute the EC expression in a plan node atop the one returning
the passed-in targetlist; rather, the EC expression will be
computed as an additional column of that targetlist. So any
Var or quasi-Var used in the given tlist is also available to the
EC expression. In simple cases this makes no difference because
the given tlist is just a list of Vars or quasi-Vars --- but if
we are considering an appendrel member produced by flattening
a UNION ALL, the tlist may contain expressions, resulting in
failure to match and a "could not find pathkey item to sort"
error.
To fix, we can flatten both the tlist and the EC members with
pull_var_clause(), and then just check for subset-ness, so
that the code is actually shorter than before.
While this bug is quite old, the present patch only works back to
v13. We could possibly make it work in v12 by back-patching parts
of 375398244. On the whole though I don't like the risk/reward
ratio of that idea. v12's final release is next month, meaning
there would be no chance to correct matters if the patch causes a
regression. Since this failure has escaped notice for 14 years,
it's likely nobody will hit it in the field with v12.
Per bug #18652 from Alexander Lakhin.
Andrei Lepikhov and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/path/equivclass.c
M src/test/regress/expected/inherit.out
M src/test/regress/sql/inherit.sql
Remove incorrect function import from pgindent
commit : 79ca063de8dace1ae0bb9774ca1f0480f4bcc73f
author : Daniel Gustafsson <[email protected]>
date : Wed, 9 Oct 2024 09:34:34 +0200
committer: Daniel Gustafsson <[email protected]>
date : Wed, 9 Oct 2024 09:34:34 +0200
Commit 149ac7d4559 which re-implemented pgindent in Perl explicitly
imported the devnull function from File::Spec, but the module does
not export anything. In recent versions of Perl calling a missing
import function cause a warning, which combined with warnings being
fatal cause pgindent to error out.
Backpatch to all supported versions.
Author: Erik Wienhold <[email protected]>
Reviewed-by: Andrew Dunstan <[email protected]>
Reviewed-by: Daniel Gustafsson <[email protected]>
Discusson: https://postgr.es/m/[email protected]
Backpatch-through: v12
M src/tools/pgindent/pgindent
vacuumdb: Schema-qualify operator in catalog query's WHERE clause.
commit : d4ade0bafb7571c8268f5eae4124e71df45ca8b2
author : Nathan Bossart <[email protected]>
date : Mon, 7 Oct 2024 16:49:20 -0500
committer: Nathan Bossart <[email protected]>
date : Mon, 7 Oct 2024 16:49:20 -0500
Commit 1ab67c9dfa, which modified this catalog query so that it
doesn't return temporary relations, forgot to schema-qualify the
operator. A comment earlier in the function implores us to fully
qualify everything in the query:
* Since we execute the constructed query with the default search_path
* (which could be unsafe), everything in this query MUST be fully
* qualified.
This commit fixes that. While at it, add a newline for consistency
with surrounding code.
Reviewed-by: Noah Misch
Discussion: https://postgr.es/m/ZwQJYcuPPUsF0reU%40nathan
Backpatch-through: 12
M src/bin/scripts/vacuumdb.c
Fix Y2038 issues with MyStartTime.
commit : b255493aefe9eea0ba2592d485819aaad21193fb
author : Nathan Bossart <[email protected]>
date : Mon, 7 Oct 2024 13:51:03 -0500
committer: Nathan Bossart <[email protected]>
date : Mon, 7 Oct 2024 13:51:03 -0500
Several places treat MyStartTime as a "long", which is only 32 bits
wide on some platforms. In reality, MyStartTime is a pg_time_t,
i.e., a signed 64-bit integer. This will lead to interesting bugs
on the aforementioned systems in 2038 when signed 32-bit integers
are no longer sufficient to store Unix time (e.g., "pg_ctl start"
hanging). To fix, ensure that MyStartTime is handled as a 64-bit
value everywhere. (Of course, users will need to ensure that
time_t is 64 bits wide on their system, too.)
Co-authored-by: Max Johnson
Discussion: https://postgr.es/m/CO1PR07MB905262E8AC270FAAACED66008D682%40CO1PR07MB9052.namprd07.prod.outlook.com
Backpatch-through: 12
M src/backend/utils/error/elog.c
M src/backend/utils/init/miscinit.c
M src/bin/pg_ctl/pg_ctl.c
Ignore not-yet-defined Portals in pg_cursors view.
commit : 4a17acd0dbb6d173e776ba5f7d16804a77bff5bf
author : Tom Lane <[email protected]>
date : Sun, 6 Oct 2024 16:03:48 -0400
committer: Tom Lane <[email protected]>
date : Sun, 6 Oct 2024 16:03:48 -0400
pg_cursor() supposed that any Portal it finds in the hash table must
have sourceText set up, but there's an edge case where that is not so.
A newly-created Portal has sourceText = NULL, and that doesn't change
until PortalDefineQuery is called. In SPI_cursor_open_internal,
we perform GetCachedPlan between CreatePortal and PortalDefineQuery,
and it's possible for user-defined code to execute during that
planning and cause a fetch from the pg_cursors view, resulting in a
null-pointer-dereference crash. (It looks like the same could happen
in exec_bind_message, but I've not tried to provoke a failure there.)
I considered trying to fix this by setting sourceText sooner, but
there may be instances of this same calling pattern in extensions,
and we couldn't be sure they'd get the memo promptly. It seems
better to redefine pg_cursor as not showing Portals that have
not yet had PortalDefineQuery called on them, which we can do by
just skipping them if sourceText is still NULL.
(Before a1c692358, pg_cursor would instead return a row with NULL
in the statement column. We could revert to that behavior but it
doesn't really seem like a better definition, especially since our
documentation doesn't suggest that the column could be NULL.)
Per report from PetSerAl. Back-patch to all supported branches.
Discussion: https://postgr.es/m/CAKygsHTBXLXjwV43kpZa+Cs+XTiaeeJiZdL4cPBm9f4MTdw7wg@mail.gmail.com
M src/backend/utils/mmgr/portalmem.c
Parse libpq's "keepalives" option more like other integer options.
commit : 2120eda94420416abb0ce57a65ea33bc72fdf55f
author : Tom Lane <[email protected]>
date : Wed, 2 Oct 2024 17:30:36 -0400
committer: Tom Lane <[email protected]>
date : Wed, 2 Oct 2024 17:30:36 -0400
Use pqParseIntParam (nee parse_int_param) instead of using strtol
directly. This allows trailing whitespace, which the previous coding
didn't, and makes the spelling of the error message consistent with
other similar cases.
This seems to be an oversight in commit e7a221797, which introduced
parse_int_param. That fixed places that were using atoi(), but missed
this place which was randomly using strtol() instead.
Ordinarily I'd consider this minor cleanup not worth back-patching.
However, it seems that ecpg assumes it can add trailing whitespace
to URL parameters, so that use of the keepalives option fails in
that context. Perhaps that's worth improving as a separate matter.
In the meantime, back-patch this to all supported branches.
Yuto Sasaki (some further cleanup by me)
Discussion: https://postgr.es/m/TY2PR01MB36286A7B97B9A15793335D18C1772@TY2PR01MB3628.jpnprd01.prod.outlook.com
M src/interfaces/libpq/fe-connect.c
Fix race condition in COMMIT PREPARED causing orphaned 2PC files
commit : 7bfaa467193ccb9ef17ba41277dc8985c3c9016d
author : Michael Paquier <[email protected]>
date : Tue, 1 Oct 2024 15:44:14 +0900
committer: Michael Paquier <[email protected]>
date : Tue, 1 Oct 2024 15:44:14 +0900
COMMIT PREPARED removes on-disk 2PC files near its end, but the state
checked if a file is on-disk or not gets read from shared memory while
not holding the two-phase state lock.
Because of that, there was a small window where a second backend doing a
PREPARE TRANSACTION could reuse the GlobalTransaction put back into the
2PC free list by the COMMIT PREPARED, overwriting the "ondisk" flag read
afterwards by the COMMIT PREPARED to decide if its on-disk two-phase
state file should be removed, preventing the file deletion.
This commit fixes this issue so as the "ondisk" flag in the
GlobalTransaction is read while holding the two-phase state lock, not
from shared memory after its entry has been added to the free list.
Orphaned two-phase state files flushed to disk after a checkpoint are
discarded at the beginning of recovery. However, a truncation of
pg_xact/ would make the startup process issue a FATAL when it cannot
read the SLRU page holding the state of the transaction whose 2PC file
was orphaned, which is a necessary step to decide if the 2PC file should
be removed or not. Removing manually the file would be necessary in
this case.
Issue introduced by effe7d9552dd, so backpatch all the way down.
Mea culpa.
Author: wuchengwen
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/backend/access/transam/twophase.c
Doc: replace unnecessary non-breaking space with ordinal space.
commit : 4f6d43c6f8ec138f6e9bc9c9a8871361819d7a2e
author : Tatsuo Ishii <[email protected]>
date : Tue, 1 Oct 2024 11:38:47 +0900
committer: Tatsuo Ishii <[email protected]>
date : Tue, 1 Oct 2024 11:38:47 +0900
There were unnecessary non-breaking spaces (nbsp, U+00A0, 0xc2a0 in
UTF-8) in the docs. This commit replaces them with ASCII spaces
(0x20).
config.sgml is backpatched through 17.
ref/drop_extension.sgml is backpatched through 13.
Discussion: https://postgr.es/m/20240930.153404.202479334310259810.ishii%40postgresql.org
Reviewed-by: Yugo Nagata, Daniel Gustafsson
Backpatch-through: 17, 13
M doc/src/sgml/ref/drop_extension.sgml
reindexdb: Skip reindexing temporary tables and indexes.
commit : 9410f7cbf4ff7bb0b94238b8d5fee4aecf157374
author : Fujii Masao <[email protected]>
date : Mon, 30 Sep 2024 11:17:23 +0900
committer: Fujii Masao <[email protected]>
date : Mon, 30 Sep 2024 11:17:23 +0900
Reindexing temp tables or indexes of other sessions is not allowed.
However, reindexdb in parallel mode previously listed them as
the objects to process, leading to failures.
This commit ensures reindexdb in parallel mode skips temporary tables
and indexes by adding a condition based on the relpersistence column
in pg_class to the object listing queries, preventing these issues.
Note that this commit does not affect reindexdb when temporary tables
or indexes are explicitly specified using the -t or -j options;
reindexdb in that case still does not skip them and can cause an error.
Back-patch to v13 where parallel mode was introduced in reindexdb.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/[email protected]
M src/bin/scripts/reindexdb.c
Remove NULL dereference from RenameRelationInternal().
commit : db1992455569265be12efc65720cfe13ca64122c
author : Noah Misch <[email protected]>
date : Sun, 29 Sep 2024 15:54:25 -0700
committer: Noah Misch <[email protected]>
date : Sun, 29 Sep 2024 15:54:25 -0700
Defect in last week's commit aac2c9b4fde889d13f859c233c2523345e72d32b,
per Coverity. Reaching this would need catalog corruption. Back-patch
to v12, like that commit.
M src/backend/commands/tablecmds.c
Avoid 037_invalid_database.pl hang under debug_discard_caches.
commit : 118dfd12138fe1eb57c0720d1d67bd17500eca31
author : Noah Misch <[email protected]>
date : Fri, 27 Sep 2024 15:28:56 -0700
committer: Noah Misch <[email protected]>
date : Fri, 27 Sep 2024 15:28:56 -0700
Back-patch to v12 (all supported versions).
M src/test/recovery/t/037_invalid_database.pl
Fix incorrect memory access in VACUUM FULL with invalid toast indexes
commit : 911eda9f3cc3d4d87fc060519947f35aa92f0d07
author : Michael Paquier <[email protected]>
date : Fri, 27 Sep 2024 09:40:21 +0900
committer: Michael Paquier <[email protected]>
date : Fri, 27 Sep 2024 09:40:21 +0900
An invalid toast index is skipped in reindex_relation(). These would be
remnants of a failed REINDEX CONCURRENTLY and they should never been
rebuilt as there can only be one valid toast index at a time.
REINDEX_REL_SUPPRESS_INDEX_USE, used by CLUSTER and VACUUM FULL, needs
to maintain a list of the indexes being processed. The list of indexes
is retrieved from the relation cache, and includes invalid indexes. The
code has missed that invalid toast indexes are ignored in
reindex_relation() as this leads to a hard failure in reindex_index(),
and they were left in the reindex pending list, making the list
inconsistent when rechecked. The incorrect memory access was happening
when scanning pg_class for the refresh of pg_database.datfrozenxid, when
doing a scan of pg_class.
This issue exists since REINDEX CONCURRENTLY exists, where invalid toast
indexes can exist, so backpatch all the way down.
Reported-by: Alexander Lakhin
Author: Tender Wang
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/backend/catalog/index.c
tests: Restrict pg_locks queries in advisory_locks.sql to current database
commit : f232d7c6827c30e8f464dfa31cbe243ff6002ba9
author : Andres Freund <[email protected]>
date : Wed, 5 Oct 2022 10:44:38 -0700
committer: Michael Paquier <[email protected]>
date : Wed, 5 Oct 2022 10:44:38 -0700
Otherwise testing an existing installation can fail, if there are other locks,
e.g. from one of the isolation tests.
This was originally applied as c3315a7da57b in 16~, but it is possible
to see this test fail depending on the concurrent activity for older
active branches.
Reviewed-by: Michael Paquier <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/test/regress/expected/advisory_lock.out
M src/test/regress/sql/advisory_lock.sql
vacuumdb: Skip temporary tables in query to build list of relations
commit : 9db4598c9c984d41896d517ff53b7b0cfbb3dc2e
author : Michael Paquier <[email protected]>
date : Wed, 25 Sep 2024 14:44:59 +0900
committer: Michael Paquier <[email protected]>
date : Wed, 25 Sep 2024 14:44:59 +0900
Running vacuumdb with a non-superuser while another user has created a
temporary table would lead to a mid-flight permission failure,
interrupting the operation. vacuum_rel() skips temporary relations of
other backends, and it makes no sense for vacuumdb to know about these
relations, so let's switch it to ignore temporary relations entirely.
Adding a qual in the query based on relpersistence simplifies the
generation of its WHERE clause in vacuum_one_database(), per se the
removal of "has_where".
Author: VaibhaveS, Michael Paquier
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAM_eQjwfAR=y3G1fGyS1U9FTmc+FyJm9amNfY2QCZBnDDbNPZg@mail.gmail.com
Backpatch-through: 12
M src/bin/scripts/vacuumdb.c
For inplace update durability, make heap_update() callers wait.
commit : 14c57cb63907eb7af0f973022b919c0f777db0d9
author : Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:18 -0700
committer: Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:18 -0700
The previous commit fixed some ways of losing an inplace update. It
remained possible to lose one when a backend working toward a
heap_update() copied a tuple into memory just before inplace update of
that tuple. In catalogs eligible for inplace update, use LOCKTAG_TUPLE
to govern admission to the steps of copying an old tuple, modifying it,
and issuing heap_update(). This includes MERGE commands. To avoid
changing most of the pg_class DDL, don't require LOCKTAG_TUPLE when
holding a relation lock sufficient to exclude inplace updaters.
Back-patch to v12 (all supported versions). In v13 and v12, "UPDATE
pg_class" or "UPDATE pg_database" can still lose an inplace update. The
v14+ UPDATE fix needs commit 86dc90056dfdbd9d1b891718d2e5614e3e432f35,
and it wasn't worth reimplementing that fix without such infrastructure.
Reviewed by Nitin Motiani and (in earlier versions) Heikki Linnakangas.
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/heapam.c
M src/backend/access/index/genam.c
M src/backend/catalog/aclchk.c
M src/backend/catalog/catalog.c
M src/backend/commands/dbcommands.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/executor/execReplication.c
M src/backend/executor/nodeModifyTable.c
M src/backend/utils/cache/relcache.c
M src/backend/utils/cache/syscache.c
M src/include/storage/lockdefs.h
M src/include/utils/syscache.h
M src/test/isolation/expected/intra-grant-inplace.out
M src/test/isolation/specs/eval-plan-qual.spec
M src/test/isolation/specs/intra-grant-inplace.spec
Fix data loss at inplace update after heap_update().
commit : a8ad1929d2ec04a5e46dd51d2ef5768c7179ef0b
author : Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:18 -0700
committer: Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:18 -0700
As previously-added tests demonstrated, heap_inplace_update() could
instead update an unrelated tuple of the same catalog. It could lose
the update. Losing relhasindex=t was a source of index corruption.
Inplace-updating commands like VACUUM will now wait for heap_update()
commands like GRANT TABLE and GRANT DATABASE. That isn't ideal, but a
long-running GRANT already hurts VACUUM progress more just by keeping an
XID running. The VACUUM will behave like a DELETE or UPDATE waiting for
the uncommitted change.
For implementation details, start at the systable_inplace_update_begin()
header comment and README.tuplock. Back-patch to v12 (all supported
versions). In back branches, retain a deprecated heap_inplace_update(),
for extensions.
Reported by Smolkin Grigory. Reviewed by Nitin Motiani, (in earlier
versions) Heikki Linnakangas, and (in earlier versions) Alexander
Lakhin.
Discussion: https://postgr.es/m/CAMp+ueZQz3yDk7qg42hk6-9gxniYbp-=bG2mgqecErqR5gGGOA@mail.gmail.com
M src/backend/access/heap/README.tuplock
M src/backend/access/heap/heapam.c
M src/backend/access/index/genam.c
M src/backend/catalog/index.c
M src/backend/catalog/toasting.c
M src/backend/commands/dbcommands.c
M src/backend/commands/vacuum.c
M src/include/access/genam.h
M src/include/access/heapam.h
M src/test/isolation/expected/intra-grant-inplace-db.out
M src/test/isolation/expected/intra-grant-inplace.out
M src/test/isolation/specs/intra-grant-inplace-db.spec
M src/test/isolation/specs/intra-grant-inplace.spec
Warn if LOCKTAG_TUPLE is held at commit, under debug_assertions.
commit : 5ef9b4a2f79670a1cf9649e568fba7920a72e9a4
author : Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:18 -0700
committer: Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:18 -0700
The current use always releases this locktag. A planned use will
continue that intent. It will involve more areas of code, making unlock
omissions easier. Warn under debug_assertions, like we do for various
resource leaks. Back-patch to v12 (all supported versions), the plan
for the commit of the new use.
Reviewed by Heikki Linnakangas.
Discussion: https://postgr.es/m/[email protected]
M src/backend/storage/lmgr/lock.c
Back-patch "Refactor code in tablecmds.c to check and process tablespace moves"
commit : 1299564cb9c787de3068777eaf8440fd0fceca4a
author : Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:24 -0700
committer: Noah Misch <[email protected]>
date : Tue, 24 Sep 2024 15:25:24 -0700
Back-patch commits 4c9c359d38ff1e2de388eedd860785be6a49201c and
24843297a96d7be16cc3f4b090aacfc6e5e6839e to v13 and v12. Before those
commits, we held the modifiable copy of the relation's pg_class row
throughout a table_relation_copy_data(). That can last long enough to
copy MaxBlockNumber of data. A subsequent fix will hold LockTuple() for
the lifespan of that modifiable copy. By back-patching this first, we
avoid a needless long-duration LOCKTAG_TUPLE.
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/include/commands/tablecmds.h
Drop global objects after completed test
commit : e43e71b6cd2f440e84da8a4a245837afba28b84f
author : Daniel Gustafsson <[email protected]>
date : Wed, 3 Apr 2024 13:33:25 +0200
committer: Michael Paquier <[email protected]>
date : Wed, 3 Apr 2024 13:33:25 +0200
Project policy is to not leave global objects behind after a regress
test run. This was found as a result of the development of a patch
to make pg_regress detect such leftovers automatically, which in the
end was withdrawn due to issues with parallel runs.
This was originally committed as 936e3fa3787a, but the issue also exists
in the 12~16 range.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/test/modules/test_pg_dump/expected/test_pg_dump.out
M src/test/modules/test_pg_dump/sql/test_pg_dump.sql
Doc: explain how to test ADMIN privilege with pg_has_role().
commit : 77930c5e92d943205cab4d3a27f435849723fec9
author : Tom Lane <[email protected]>
date : Fri, 20 Sep 2024 15:56:34 -0400
committer: Tom Lane <[email protected]>
date : Fri, 20 Sep 2024 15:56:34 -0400
This has always been possible, but the syntax is a bit obscure,
and our user-facing docs were not very helpful. Spell it out
more clearly.
Per complaint from Dominique Devienne. Back-patch to
all supported branches.
Discussion: https://postgr.es/m/CAFCRh-8JNEy+dV4SXFOrWca50u+d=--TO4cq=+ac1oBtfJy4AA@mail.gmail.com
M doc/src/sgml/func.sgml
doc PG relnotes: remove warning about commit links in PDF build
commit : 92a6cc5acbe0d341f6e48333db2e8862014caf89
author : Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 18:05:21 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 18:05:21 -0400
Make paragraph empty instead of removing it.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/stylesheet-fo.xsl
doc PG relnotes: document "Unresolved ID reference found" cause
commit : a979f2894c3d436b17a9de2655e4237de1c7df20
author : Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 12:01:58 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 12:01:58 -0400
Backpatch-through: 12
M doc/src/sgml/stylesheet-fo.xsl
doc PG relnotes: rename commit link paragraph for clarity
commit : ececa27ff5b8702153cacbda51ae174e2bfec0f9
author : Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 09:47:22 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 09:47:22 -0400
FYI, during PDF builds, this link type generates a "Unresolved ID
reference found" warning because it is suppressed from the PDF output.
Backpatch-through: 12
M doc/src/sgml/release.sgml
M doc/src/sgml/stylesheet-fo.xsl
Improve Perl script which adds commit links to release notes
commit : 6ce8c2c292bbe09d8d592f41e3ec8b2b8664b2e2
author : Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 08:45:32 -0400
committer: Bruce Momjian <[email protected]>
date : Thu, 19 Sep 2024 08:45:32 -0400
Reported-by: Andrew Dunstan
Discussion: https://postgr.es/m/[email protected]
Author: Andrew Dunstan
Backpatch-through: 12
M src/tools/add_commit_links.pl
doc PG relnotes: add paragraph explaining the section symbol
commit : d1389c2939fb6c4ae653fa5a5d484a7792ecfdc5
author : Bruce Momjian <[email protected]>
date : Wed, 18 Sep 2024 17:13:19 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 18 Sep 2024 17:13:19 -0400
And suppress the symbol in print mode, where the section symbol does not
appear.
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/release.sgml
M doc/src/sgml/stylesheet-fo.xsl
doc PG relnotes: no relnote footnotes for commit links in PDF
commit : 97ce16c7da99e8e5bb81c8dac51934fe8d68499c
author : Bruce Momjian <[email protected]>
date : Wed, 18 Sep 2024 16:34:51 -0400
committer: Bruce Momjian <[email protected]>
date : Wed, 18 Sep 2024 16:34:51 -0400
In print output, there are too many commit links for footnotes in the
release notes to be useful.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/stylesheet-fo.xsl
Don't enter parallel mode when holding interrupts.
commit : 916b8ae475fa852483a7ef05e793a22c922bf999
author : Noah Misch <[email protected]>
date : Tue, 17 Sep 2024 19:53:11 -0700
committer: Noah Misch <[email protected]>
date : Tue, 17 Sep 2024 19:53:11 -0700
Doing so caused the leader to hang in wait_event=ParallelFinish, which
required an immediate shutdown to resolve. Back-patch to v12 (all
supported versions).
Francesco Degrassi
Discussion: https://postgr.es/m/CAC-SaSzHUKT=vZJ8MPxYdC_URPfax+yoA1hKTcF4ROz_Q6z0_Q@mail.gmail.com
M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
doc PG relnotes: fix SGML markup for new commit links
commit : a01dbdb1cb8c888c4ad97eaba0e3a391a86d3edb
author : Bruce Momjian <[email protected]>
date : Mon, 16 Sep 2024 14:23:39 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Sep 2024 14:23:39 -0400
Backpatch-through: 12
M doc/src/sgml/postgres.sgml
doc PG relnotes: add links to commits
commit : 3ce928fe58889e1004c686187963daba35ecd441
author : Bruce Momjian <[email protected]>
date : Mon, 16 Sep 2024 14:14:37 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Sep 2024 14:14:37 -0400
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/release-13.sgml
scripts: add Perl script to add links to release notes
commit : 9052475cd9f0e0a954167d940bc776186aff9ef9
author : Bruce Momjian <[email protected]>
date : Mon, 16 Sep 2024 13:26:36 -0400
committer: Bruce Momjian <[email protected]>
date : Mon, 16 Sep 2024 13:26:36 -0400
Reported-by: jian he
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M src/tools/RELEASE_CHANGES
A src/tools/add_commit_links.pl
Replace usages of xmlXPathCompile() with xmlXPathCtxtCompile().
commit : 4310dfa254e9de9b5fbecbc54ef512187a9e1d09
author : Tom Lane <[email protected]>
date : Sun, 15 Sep 2024 13:33:09 -0400
committer: Tom Lane <[email protected]>
date : Sun, 15 Sep 2024 13:33:09 -0400
In existing releases of libxml2, xmlXPathCompile can be driven
to stack overflow because it fails to protect itself against
too-deeply-nested input. While there is an upstream fix as of
yesterday, it will take years for that to propagate into all
shipping versions. In the meantime, we can protect our own
usages basically for free by calling xmlXPathCtxtCompile instead.
(The actual bug is that libxml2 keeps its nesting counter in the
xmlXPathContext, and its parsing code was willing to just skip
counting nesting levels if it didn't have a context. So if we supply
a context, all is well. It seems odd actually that it works at all
to not supply a context, because this means that XPath parsing does
not have access to XML namespace info. Apparently libxml2 never
checks namespaces until runtime? Anyway, this seems like good
future-proofing even if its only immediate effect is to dodge a bug.)
Sadly, this hack only offers protection with libxml2 2.9.11 and newer.
Before that there are multiple similar problems, so if you are
processing untrusted XML it behooves you to get a newer version.
But we have some pretty old libxml2 in the buildfarm, so it seems
impractical to add a regression test to verify this fix.
Per bug #18617 from Jingzhou Fu. Back-patch to all supported
versions.
Discussion: https://postgr.es/m/[email protected]
Discussion: https://gitlab.gnome.org/GNOME/libxml2/-/issues/799
M contrib/xml2/xpath.c
M src/backend/utils/adt/xml.c
doc PG relnotes: add attribution for time zone data files items
commit : ad3a47cfcdf6e75457538c70a2fbdf52ddbc3a93
author : Bruce Momjian <[email protected]>
date : Sat, 14 Sep 2024 19:51:54 -0400
committer: Bruce Momjian <[email protected]>
date : Sat, 14 Sep 2024 19:51:54 -0400
This is needed for a future script to add commit links; specifically we
need the closing parentheses of the attribution.
Backpatch-through: 12
M doc/src/sgml/release-13.sgml
Run regression tests with timezone America/Los_Angeles.
commit : b28b9b19bbe3410da4a805ef775e0383a66af314
author : Tom Lane <[email protected]>
date : Sat, 14 Sep 2024 17:55:03 -0400
committer: Tom Lane <[email protected]>
date : Sat, 14 Sep 2024 17:55:03 -0400
Historically we've used timezone "PST8PDT", but the recent release
2024b of tzdb changes the definition of that zone in a way that
breaks many test cases concerned with dates before 1970. Although
we've not yet adopted 2024b into our own tree, this is already
problematic for people using --with-system-tzdata if their platform
has already adopted 2024b. To work with both older and newer
versions of tzdb, switch to using "America/Los_Angeles", accepting
the ensuing changes in regression test results.
Back-patch to all supported branches.
Per report and patch from Wolfgang Walther.
Discussion: https://postgr.es/m/[email protected]
M doc/src/sgml/ref/set.sgml
M doc/src/sgml/regress.sgml
M src/test/regress/expected/date.out
M src/test/regress/expected/horology.out
M src/test/regress/expected/timestamptz.out
M src/test/regress/pg_regress.c
M src/test/regress/sql/horology.sql
M src/test/regress/sql/timestamptz.sql
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required
commit : f40d9e9f1b5892a340d5ed5d650c9dc3cf72e6e9
author : Andrew Dunstan <[email protected]>
date : Sat, 14 Sep 2024 08:37:08 -0400
committer: Andrew Dunstan <[email protected]>
date : Sat, 14 Sep 2024 08:37:08 -0400
Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.
Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.
Original report of the issue from Muralikrishna Bandaru.
M src/tools/msvc/Mkvcbuild.pm
Allow _h_indexbuild() to be interrupted.
commit : b27215dbb42b6376515e1ab1ff06e0ad9ab13450
author : Tom Lane <[email protected]>
date : Fri, 13 Sep 2024 16:16:47 -0400
committer: Tom Lane <[email protected]>
date : Fri, 13 Sep 2024 16:16:47 -0400
When we are building a hash index that is large enough to need
pre-sorting (larger than either maintenance_work_mem or NBuffers),
the initial sorting phase is interruptible, but the insertion
phase wasn't. Add the missing CHECK_FOR_INTERRUPTS().
Per bug #18616 from Alexander Lakhin. Back-patch to all
supported branches.
Pavel Borisov
Discussion: https://postgr.es/m/[email protected]
M src/backend/access/hash/hashsort.c
Fix contrib/pageinspect's test for sequences.
commit : ef46a73f6941021ad40b342ea637f19fb5144cc8
author : Nathan Bossart <[email protected]>
date : Fri, 13 Sep 2024 10:16:40 -0500
committer: Nathan Bossart <[email protected]>
date : Fri, 13 Sep 2024 10:16:40 -0500
I managed to break this test in two different ways in commit
05036a3155.
First, the output of the new call to tuple_data_split() on the test
sequence is dependent on endianness. This is fixed by setting a
special start value for the test sequence that produces the same
output regardless of the endianness of the machine.
Second, on versions older than v15, the new test case fails under
"force_parallel_mode = regress" with the following error:
ERROR: cannot access temporary tables during a parallel operation
This is because pageinspect's disk-accessing functions are
incorrectly marked PARALLEL SAFE on versions older than v15 (see
commit aeaaf520f4 for details). This one is fixed by changing the
test sequence to be permanent. The only reason it was previously
marked temporary was to avoid needing a DROP SEQUENCE command at
the end of the test. Unlike some other tests in this file, the use
of a permanent sequence here shouldn't result in any test
instability like what was fixed by commit e2933a6e11.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/ZuOKOut5hhDlf_bP%40nathan
Backpatch-through: 12
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/sql/page.sql
Reintroduce support for sequences in pgstattuple and pageinspect.
commit : ca902529cc0fea8a0ef660b519640da09a2680ef
author : Nathan Bossart <[email protected]>
date : Thu, 12 Sep 2024 16:31:29 -0500
committer: Nathan Bossart <[email protected]>
date : Thu, 12 Sep 2024 16:31:29 -0500
Commit 4b82664156 restricted a number of functions provided by
contrib modules to only relations that use the "heap" table access
method. Sequences always use this table access method, but they do
not advertise as such in the pg_class system catalog, so the
aforementioned commit also (presumably unintentionally) removed
support for sequences from some of these functions. This commit
reintroduces said support for sequences to these functions and adds
a couple of relevant tests.
Co-authored-by: Ayush Vatsa
Reviewed-by: Robert Haas, Michael Paquier, Matthias van de Meent
Discussion: https://postgr.es/m/CACX%2BKaP3i%2Bi9tdPLjF5JCHVv93xobEdcd_eB%2B638VDvZ3i%3DcQA%40mail.gmail.com
Backpatch-through: 12
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/heapfuncs.c
M contrib/pageinspect/sql/page.sql
M contrib/pgstattuple/expected/pgstattuple.out
M contrib/pgstattuple/pgstattuple.c
M contrib/pgstattuple/sql/pgstattuple.sql
Remove incorrect Assert.
commit : abed06fdc5b23ca2b11adac236d43b6328fcfae1
author : Tom Lane <[email protected]>
date : Wed, 11 Sep 2024 11:41:47 -0400
committer: Tom Lane <[email protected]>
date : Wed, 11 Sep 2024 11:41:47 -0400
check_agglevels_and_constraints() asserted that if we find an
aggregate function in an EXPR_KIND_FROM_SUBSELECT expression, the
expression must be in a LATERAL subquery. Alexander Lakhin found a
case where that's not so: because of the odd scoping rules for NEW/OLD
within a rule, a reference to NEW/OLD could cause an aggregate to be
considered top-level even though it's in an unmarked sub-select.
The error message that would be thrown seems sufficiently on-point,
so just remove the Assert. (Hence, this is not a bug for production
builds.)
This Assert was added by me in commit eaccfded9 (9.3 era). It looks
like I put it in to cross-check that the new logic for detecting
misplaced aggregates (using agglevelsup) caught the same cases that a
previous check on p_lateral_active did. So there might have been some
related misbehavior before eaccfded9 ... but that's very ancient
history by now, so I didn't dig any deeper.
Per bug #18608 from Alexander Lakhin. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/[email protected]
M src/backend/parser/parse_agg.c
Stabilize 039_end_of_wal test.
commit : d1d0fe1feb23e76daa6c3e0ea995b927dcf4cfe3
author : Thomas Munro <[email protected]>
date : Sat, 31 Aug 2024 14:32:08 +1200
committer: Thomas Munro <[email protected]>
date : Sat, 31 Aug 2024 14:32:08 +1200
The first test was sensitive to the insert LSN after setting up the
catalogs, which depended on environmental things like the locales on the
OS and usernames. Switch to a new WAL file before the first test, as a
simple way to put every computer into the same state.
Back-patch to all supported releases.
Reported-by: Anton Voloshin <[email protected]>
Reported-by: Nathan Bossart <[email protected]>
Reviewed-by: Tom Lane <[email protected]>
Reviewed-by: Nathan Bossart <[email protected]>
Discussion: https://postgr.es/m/b26aeac2-cb6d-4633-a7ea-945baae83dcf%40postgrespro.ru
M src/test/recovery/t/039_end_of_wal.pl
Clarify restrict_nonsystem_relation_kind description.
commit : 187d8bb1a3156f055fe1d362f4a0162d64aacd1d
author : Masahiko Sawada <[email protected]>
date : Fri, 30 Aug 2024 15:05:57 -0700
committer: Masahiko Sawada <[email protected]>
date : Fri, 30 Aug 2024 15:05:57 -0700
This change improves the description of the
restrict_nonsystem_relation_kind parameter in guc_table.c and the
documentation for better clarity.
Backpatch to 12, where this GUC parameter was introduced.
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/6a96f1af-22b4-4a80-8161-1f26606b9ee2%40eisentraut.org
Backpatch-through: 12
M doc/src/sgml/config.sgml
M src/backend/utils/misc/guc.c
Disallow USING clause when altering type of generated column
commit : 7589d5c5b98dee21541214950d8a303c6979dd78
author : Peter Eisentraut <[email protected]>
date : Thu, 29 Aug 2024 08:38:29 +0200
committer: Peter Eisentraut <[email protected]>
date : Thu, 29 Aug 2024 08:38:29 +0200
This does not make sense. It would write the output of the USING
clause into the converted column, which would violate the generation
expression. This adds a check to error out if this is specified.
There was a test for this, but that test errored out for a different
reason, so it was not effective.
Reported-by: Jian He <[email protected]>
Reviewed-by: Yugo NAGATA <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/c7083982-69f4-4b14-8315-f9ddb20b9834%40eisentraut.org
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated.out
Fix a couple of wait event descriptions.
commit : 18e3e290bc6f241a0b4461074255c0fd873d25fe
author : Nathan Bossart <[email protected]>
date : Tue, 20 Aug 2024 13:43:20 -0500
committer: Nathan Bossart <[email protected]>
date : Tue, 20 Aug 2024 13:43:20 -0500
The descriptions for ProcArrayGroupUpdate and XactGroupUpdate claim
that these events mean we are waiting for the group leader "at end
of a parallel operation," but neither pertains to parallel
operations. This commit reverts these descriptions to their
wording before commit 3048898e73, i.e., "end of a parallel
operation" is changed to "transaction end."
Author: Sameer Kumar
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAGPeHmh6UMrKQHKCmX%2B5vV5TH9P%3DKw9en3k68qEem6J%3DyrZPUA%40mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/monitoring.sgml
Document limit on the number of out-of-line values per table
commit : c9aa6f4f86fc244f56d4fcdf2ee0154bb3ecac84
author : John Naylor <[email protected]>
date : Tue, 20 Aug 2024 10:02:34 +0700
committer: John Naylor <[email protected]>
date : Tue, 20 Aug 2024 10:02:34 +0700
Document the hard limit stemming from the size of an OID, and also
mention the perfomance impact that occurs before the hard limit
is reached.
Jakub Wartak and Robert Haas
Backpatch to all supported versions
Discussion: https://postgr.es/m/CAKZiRmwWhp2yxjqJLwbBjHdfbJBcUmmKMNAZyBjjtpgM9AMatQ%40mail.gmail.com
M doc/src/sgml/limits.sgml
Explain dropdb can't use syscache because of TOAST
commit : 4e7531fda587c35a4ee0fa2e5fdf90b0914a1685
author : Tomas Vondra <[email protected]>
date : Mon, 19 Aug 2024 13:31:51 +0200
committer: Tomas Vondra <[email protected]>
date : Mon, 19 Aug 2024 13:31:51 +0200
Add a comment explaining dropdb() can't rely on syscache. The issue with
flattened rows was fixed by commit 0f92b230f88b, but better to have
a clear explanation why the systable scan is necessary. The other places
doing in-place updates on pg_database have the same comment.
Suggestion and patch by Yugo Nagata. Backpatch to 12, same as the fix.
Author: Yugo Nagata
Backpatch-through: 12
Discussion: https://postgr.es/m/CAJTYsWWNkCt+-UnMhg=BiCD3Mh8c2JdHLofPxsW3m2dkDFw8RA@mail.gmail.com
M src/backend/commands/dbcommands.c
Fix regression in TLS session ticket disabling
commit : f925b7f65d497fae87fad1b419e137a32cfcb504
author : Daniel Gustafsson <[email protected]>
date : Mon, 19 Aug 2024 12:55:11 +0200
committer: Daniel Gustafsson <[email protected]>
date : Mon, 19 Aug 2024 12:55:11 +0200
Commit 274bbced disabled session tickets for TLSv1.3 on top of the
already disabled TLSv1.2 session tickets, but accidentally caused
a regression where TLSv1.2 session tickets were incorrectly sent.
Fix by unconditionally disabling TLSv1.2 session tickets and only
disable TLSv1.3 tickets when the right version of OpenSSL is used.
Backpatch to all supported branches.
Reported-by: Cameron Vogt <[email protected]>
Reported-by: Fire Emerald <[email protected]>
Reviewed-by: Jacob Champion <[email protected]>
Discussion: https://postgr.es/m/DM6PR16MB3145CF62857226F350C710D1AB852@DM6PR16MB3145.namprd16.prod.outlook.com
Backpatch-through: v12
M src/backend/libpq/be-secure-openssl.c
Fix DROP DATABASE for databases with many ACLs
commit : 33c615f764c32abf20bf8a5a71fcb44115551867
author : Tomas Vondra <[email protected]>
date : Mon, 19 Aug 2024 00:04:41 +0200
committer: Tomas Vondra <[email protected]>
date : Mon, 19 Aug 2024 00:04:41 +0200
Commit c66a7d75e652 modified DROP DATABASE so that if interrupted, the
database is known to be in an invalid state and can only be dropped.
This is done by setting a flag using an in-place update, so that it's
not lost in case of rollback.
For databases with many ACLs, this may however fail like this:
ERROR: wrong tuple length
This happens because with many ACLs, the pg_database.datacl attribute
gets TOASTed. The dropdb() code reads the tuple from the syscache, which
means it's detoasted. But the in-place update expects the tuple length
to match the on-disk tuple.
Fixed by reading the tuple from the catalog directly, not from syscache.
Report and fix by Ayush Tiwari. Backpatch to 12. The DROP DATABASE fix
was backpatched to 11, but 11 is EOL at this point.
Reported-by: Ayush Tiwari
Author: Ayush Tiwari
Reviewed-by: Tomas Vondra
Backpatch-through: 12
Discussion: https://postgr.es/m/CAJTYsWWNkCt+-UnMhg=BiCD3Mh8c2JdHLofPxsW3m2dkDFw8RA@mail.gmail.com
M src/backend/commands/dbcommands.c
docs: fix incorrect plpgsql error message
commit : 881ca9cd5904e3eab1201bb5df864036961f59b1
author : Bruce Momjian <[email protected]>
date : Fri, 16 Aug 2024 22:50:54 -0400
committer: Bruce Momjian <[email protected]>
date : Fri, 16 Aug 2024 22:50:54 -0400
Change "$1" to "username".
Reported-by: [email protected]
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 12
M doc/src/sgml/plpgsql.sgml
Fix creation of partition descriptor during concurrent detach+drop
commit : 5236dcdb62ac9da00a993e0971fac4ba5e240070
author : Alvaro Herrera <[email protected]>
date : Mon, 12 Aug 2024 18:17:56 -0400
committer: Alvaro Herrera <[email protected]>
date : Mon, 12 Aug 2024 18:17:56 -0400
If a partition undergoes DETACH CONCURRENTLY immediately followed by
DROP, this could cause a problem for a concurrent transaction
recomputing the partition descriptor when running a prepared statement,
because it tries to dereference a pointer to a tuple that's not found in
a catalog scan.
The existing retry logic added in commit dbca3469ebf8 is sufficient to
cope with the overall problem, provided we don't try to dereference a
non-existant heap tuple.
Arguably, the code in RelationBuildPartitionDesc() has been wrong all
along, since no check was added in commit 898e5e3290a7 against receiving
a NULL tuple from the catalog scan; that bug has only become
user-visible with DETACH CONCURRENTLY which was added in branch 14.
Therefore, even though there's no known mechanism to cause a crash
because of this, backpatch the addition of such a check to all supported
branches. In branches prior to 14, this would cause the code to fail
with a "missing relpartbound for relation XYZ" error instead of
crashing; that's okay, because there are no reports of such behavior
anyway.
Author: Kuntal Ghosh <[email protected]>
Reviewed-by: Junwang Zhao <[email protected]>
Reviewed-by: Tender Wang <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/partitioning/partdesc.c
Suppress Coverity warnings about Asserts in get_name_for_var_field.
commit : 016f443648d2c92a3391d12545f06cf86cc06f5f
author : Tom Lane <[email protected]>
date : Sun, 11 Aug 2024 12:24:56 -0400
committer: Tom Lane <[email protected]>
date : Sun, 11 Aug 2024 12:24:56 -0400
Coverity thinks dpns->plan could be null at these points. That
shouldn't really be possible, but it's easy enough to modify the
Asserts so they'd not core-dump if it were true.
These are new in b919a97a6. Back-patch to v13; the v12 version
of the patch didn't have these Asserts.
M src/backend/utils/adt/ruleutils.c
Allow adjusting session_authorization and role in parallel workers.
commit : adf9808fa966460f822cb8ccea424cef43f27e2b
author : Tom Lane <[email protected]>
date : Sat, 10 Aug 2024 15:51:28 -0400
committer: Tom Lane <[email protected]>
date : Sat, 10 Aug 2024 15:51:28 -0400
The code intends to allow GUCs to be set within parallel workers
via function SET clauses, but not otherwise. However, doing so fails
for "session_authorization" and "role", because the assign hooks for
those attempt to set the subsidiary "is_superuser" GUC, and that call
falls foul of the "not otherwise" prohibition. We can't switch to
using GUC_ACTION_SAVE for this, so instead add a new GUC variable
flag GUC_ALLOW_IN_PARALLEL to mark is_superuser as being safe to set
anyway. (This is okay because is_superuser has context PGC_INTERNAL
and thus only hard-wired calls can change it. We'd need more thought
before applying the flag to other GUCs; but maybe there are other
use-cases.) This isn't the prettiest fix perhaps, but other
alternatives we thought of would be much more invasive.
While here, correct a thinko in commit 059de3ca4: when rejecting
a GUC setting within a parallel worker, we should return 0 not -1
if the ereport doesn't longjmp. (This seems to have no consequences
right now because no caller cares, but it's inconsistent.) Improve
the comments to try to forestall future confusion of the same kind.
Despite the lack of field complaints, this seems worth back-patching.
Thanks to Nathan Bossart for the idea to invent a new flag,
and for review.
Discussion: https://postgr.es/m/[email protected]
M src/backend/utils/misc/guc.c
M src/include/utils/guc.h
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
doc: Fix name of CRC algorithm in "Reliability" section.
commit : c17d2d685821afee034d327e68468dfd5754a64c
author : Nathan Bossart <[email protected]>
date : Fri, 9 Aug 2024 10:52:37 -0500
committer: Nathan Bossart <[email protected]>
date : Fri, 9 Aug 2024 10:52:37 -0500
This section claims we use CRC-32 for WAL records and two-phase
state files, but we've actually used CRC-32C since v9.5 (commit
5028f22f6e). Fix that.
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/ZrUFpLP-w2zTAHqq%40nathan
Backpatch-through: 12
M doc/src/sgml/wal.sgml
Fix "failed to find plan for subquery/CTE" errors in EXPLAIN.
commit : 3ad35d5022c9e5329906542150a19f9a38f9564d
author : Tom Lane <[email protected]>
date : Fri, 9 Aug 2024 11:21:39 -0400
committer: Tom Lane <[email protected]>
date : Fri, 9 Aug 2024 11:21:39 -0400
To deparse a reference to a field of a RECORD-type output of a
subquery, EXPLAIN normally digs down into the subquery's plan to try
to discover exactly which anonymous RECORD type is meant. However,
this can fail if the subquery has been optimized out of the plan
altogether on the grounds that no rows could pass the WHERE quals,
which has been possible at least since 3fc6e2d7f. There isn't
anything remaining in the plan tree that would help us, so fall back
to printing the field name as "fN" for the N'th column of the record.
(This will actually be the right thing some of the time, since it
matches the column names we assign to RowExprs.)
In passing, fix a comment typo in create_projection_plan, which
I noticed while experimenting with an alternative fix for this.
Per bug #18576 from Vasya B. Back-patch to all supported branches.
Richard Guo and Tom Lane
Discussion: https://postgr.es/m/[email protected]
M src/backend/optimizer/plan/createplan.c
M src/backend/utils/adt/ruleutils.c
M src/test/regress/expected/rowtypes.out
M src/test/regress/sql/rowtypes.sql
Refuse ATTACH of a table referenced by a foreign key
commit : 2ee02c98dd17f19ac5407e80c904d408931ab9ec
author : Alvaro Herrera <[email protected]>
date : Thu, 8 Aug 2024 19:35:13 -0400
committer: Alvaro Herrera <[email protected]>
date : Thu, 8 Aug 2024 19:35:13 -0400
Trying to attach a table as a partition which is already on the
referenced side of a foreign key on the partitioned table that it is
being attached to, leads to strange behavior: we try to clone the
foreign key from the parent to the partition, but this new FK points to
the partition itself, and the mix of pg_constraint rows and triggers
doesn't behave well.
Rather than trying to untangle the mess (which might be possible given
sufficient time), I opted to forbid the ATTACH. This doesn't seem a
problematic restriction, given that we already fail to create the
foreign key if you do it the other way around, that is, having the
partition first and the FK second.
Backpatch to all supported branches.
Reported-by: Alexander Lakhin <[email protected]>
Reviewed-by: Tender Wang <[email protected]>
Discussion: https://postgr.es/m/[email protected]
M src/backend/commands/tablecmds.c
M src/test/regress/expected/foreign_key.out
M src/test/regress/sql/foreign_key.sql
Fix pg_rewind debug output to print the source timeline history
commit : c943e2aae7c59c673d502bbb243b57469be4c75f
author : Heikki Linnakangas <[email protected]>
date : Thu, 8 Aug 2024 10:20:25 +0300
committer: Heikki Linnakangas <[email protected]>
date : Thu, 8 Aug 2024 10:20:25 +0300
getTimelineHistory() is called twice, to read the source and the
target timeline history files. However, the loop to print the file
with the --debug option used the wrong variable when dealing with the
source. As a result, the source's history was always printed as empty.
Spotted while debugging bug #18575, but this does not fix that bug,
just the debugging output. Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/[email protected]
M src/bin/pg_rewind/pg_rewind.c
Revert ECPG's use of pnstrdup()
commit : f0096ef13be263cfefb0b47e48c00a0e2fbfef06
author : Peter Eisentraut <[email protected]>
date : Wed, 7 Aug 2024 09:21:07 +0200
committer: Peter Eisentraut <[email protected]>
date : Wed, 7 Aug 2024 09:21:07 +0200
Commit 0b9466fce added a dependency on fe_memutils' pnstrdup() inside
informix.c. This adds an exit() path in a library, which we don't
want. (Unlike libpq, the ecpg libraries don't have an automated check
for that, but it makes sense to keep them to a similar standard.) The
ecpg code can already handle failure results from the *strdup() call
by itself.
Author: Jacob Champion <[email protected]>
Discussion: https://www.postgresql.org/message-id/CAOYmi+=pg=W5L1h=3MEP_EB24jaBu2FyATrLXqQHGe7cpuvwyg@mail.gmail.com
M src/interfaces/ecpg/compatlib/informix.c
Teach RPM the package name provided in Perl alias packages.
commit : 382909b635cb0eb164ef678b68b0cc4ecfb26d6b
author : Noah Misch <[email protected]>
date : Wed, 7 Aug 2024 11:43:34 -0700
committer: Noah Misch <[email protected]>
date : Wed, 7 Aug 2024 11:43:34 -0700
When commit 1185be355462d1dc7e2950a7e52eb7ca0cb6f3c8 introduced
installation of a file containing "use PostgreSQL::Test::Utils", the RPM
Package Manager said "nothing provides perl(PostgreSQL::Test::Utils)".
Discussed on pgsql-packagers. Back-patch to v12, v13, and v14 only;
newer versions don't have the alias packages.
Reviewed by Andrew Dunstan, Tom Lane, and John Harvey. Reported by John
Harvey.
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/perl/PostgreSQL/Test/Utils.pm