PostgreSQL 15.8 commit log

Stamp 15.8.

commit   : 3cc5e51ab4507b33acc82684b7d79ac43b8a6b5d    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 5 Aug 2024 16:07:10 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 5 Aug 2024 16:07:10 -0400    

Click here for diff

M configure
M configure.ac

Last-minute updates for release notes.

commit   : 6eca8a5673307ce52f892f083df0b958d35f09e1    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 5 Aug 2024 14:03:20 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 5 Aug 2024 14:03:20 -0400    

Click here for diff

Security: CVE-2024-7348  

M doc/src/sgml/release-15.sgml

Restrict accesses to non-system views and foreign tables during pg_dump.

commit   : e81e53a0c1faf2b8f5fe8822d7218c91aeebea52    
  
author   : Masahiko Sawada <[email protected]>    
date     : Mon, 5 Aug 2024 06:05:25 -0700    
  
committer: Masahiko Sawada <[email protected]>    
date     : Mon, 5 Aug 2024 06:05:25 -0700    

Click here for diff

When pg_dump retrieves the list of database objects and performs the  
data dump, there was possibility that objects are replaced with others  
of the same name, such as views, and access them. This vulnerability  
could result in code execution with superuser privileges during the  
pg_dump process.  
  
This issue can arise when dumping data of sequences, foreign  
tables (only 13 or later), or tables registered with a WHERE clause in  
the extension configuration table.  
  
To address this, pg_dump now utilizes the newly introduced  
restrict_nonsystem_relation_kind GUC parameter to restrict the  
accesses to non-system views and foreign tables during the dump  
process. This new GUC parameter is added to back branches too, but  
these changes do not require cluster recreation.  
  
Back-patch to all supported branches.  
  
Reviewed-by: Noah Misch  
Security: CVE-2024-7348  
Backpatch-through: 12  

M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/pg_dump.sgml
M src/backend/foreign/foreign.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/plancat.c
M src/backend/rewrite/rewriteHandler.c
M src/backend/tcop/postgres.c
M src/backend/utils/misc/guc.c
M src/bin/pg_dump/pg_dump.c
M src/include/tcop/tcopprot.h
M src/test/regress/expected/create_view.out
M src/test/regress/sql/create_view.sql

Translation updates

commit   : 8b57eb67e8428d93f42f603a917e610477a08821    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 5 Aug 2024 12:20:32 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 5 Aug 2024 12:20:32 +0200    

Click here for diff

Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 09a5036e19350293c332e686d66c636762f7a454  

M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/ja.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/es.po
M src/bin/pg_amcheck/po/es.po
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_config/po/es.po
M src/bin/pg_controldata/po/es.po
M src/bin/pg_ctl/po/es.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_resetwal/po/es.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/sv.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_waldump/po/es.po
M src/bin/psql/po/es.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/sv.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/libpq/po/es.po
M src/interfaces/libpq/po/ru.po
M src/pl/plperl/po/es.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/ru.po
M src/pl/tcl/po/de.po
M src/pl/tcl/po/es.po
M src/pl/tcl/po/fr.po
M src/pl/tcl/po/ja.po
M src/pl/tcl/po/ru.po
M src/pl/tcl/po/sv.po

Release notes for 16.4, 15.8, 14.13, 13.16, 12.20.

commit   : c26f2728ac5d3de10c3bcd8f3fc36894d6b36f1b    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 4 Aug 2024 13:38:59 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 4 Aug 2024 13:38:59 -0400    

Click here for diff

M doc/src/sgml/release-15.sgml

Skip citext_utf8 test on Windows.

commit   : e52a44b8a4c11bc10685162e8e638357c01b9dc4    
  
author   : Thomas Munro <[email protected]>    
date     : Mon, 13 May 2024 07:55:20 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Mon, 13 May 2024 07:55:20 +1200    

Click here for diff

Back-patch of commit cff4e5a3 to 15 and 16, per request from Oleg  
Tselebrovskiy.  Original commit message:  
  
On other Windows build farm animals it is already skipped because they  
don't use UTF-8 encoding.  On "hamerkop", UTF-8 is used, and then the  
test fails.  
  
It is not clear to me (a non-Windows person looking only at buildfarm  
evidence) whether Windows is less sophisticated than other OSes and  
doesn't know how to downcase Turkish İ with the standard Unicode  
database, or if it is more sophisticated than other systems and uses  
locale-specific behavior like ICU does.  
  
Whichever the reason, the result is the same: we need to skip the test  
on Windows, just as we already do for ICU, at least until a  
Windows-savvy developer comes up with a better idea.  The technique for  
detecting the OS is borrowed from collate.windows.win1252.sql.  
  
This was anticipated by commit c2e8bd27, but the problem only surfaced  
when Windows build farm animals started using Meson.  
  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/CA%2BhUKGJ1LeC3aE2qQYTK95rFVON3ZVoTQpTKJqxkHdtEyawH4A%40mail.gmail.com  

M contrib/citext/expected/citext_utf8.out
M contrib/citext/expected/citext_utf8_1.out
M contrib/citext/sql/citext_utf8.sql

Update comment in portal.h.

commit   : d7bc9c9d04ad6abe80006bfbe93cb359abd31dee    
  
author   : Etsuro Fujita <[email protected]>    
date     : Thu, 1 Aug 2024 17:45:04 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Thu, 1 Aug 2024 17:45:04 +0900    

Click here for diff

We store tuples into the portal's tuple store for a PORTAL_ONE_MOD_WITH  
query as well.  
  
Back-patch to all supported branches.  
  
Reviewed by Andy Fan.  
  
Discussion: https://postgr.es/m/CAPmGK14HVYBZYZtHabjeCd-e31VT%3Dwx6rQNq8QfehywLcpZ2Hw%40mail.gmail.com  

M src/include/utils/portal.h

Revert "Allow parallel workers to cope with a newly-created session user ID."

commit   : afbf32fa1aa87f710b623cbdf746c55b4799574e    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 31 Jul 2024 20:55:25 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 31 Jul 2024 20:55:25 -0400    

Click here for diff

This reverts commit 48536305370acd75c6264fcefec7fae7af8c5440.  
  
Some buildfarm animals are failing with "cannot change  
"client_encoding" during a parallel operation".  It looks like  
assign_client_encoding is unhappy at being asked to roll back a  
client_encoding setting after a parallel worker encounters a  
failure.  There must be more to it though: why didn't I see this  
during local testing?  In any case, it's clear that moving the  
RestoreGUCState() call is not as side-effect-free as I thought.  
Given that the bug f5f30c22e intended to fix has gone unreported  
for years, it's not something that's urgent to fix; I'm not  
willing to risk messing with it further with only days to our  
next release wrap.  

M src/backend/access/transam/parallel.c
M src/backend/commands/variable.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql

Allow parallel workers to cope with a newly-created session user ID.

commit   : 48536305370acd75c6264fcefec7fae7af8c5440    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 31 Jul 2024 18:54:10 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 31 Jul 2024 18:54:10 -0400    

Click here for diff

Parallel workers failed after a sequence like  
	BEGIN;  
	CREATE USER foo;  
	SET SESSION AUTHORIZATION foo;  
because check_session_authorization could not see the uncommitted  
pg_authid row for "foo".  This is because we ran RestoreGUCState()  
in a separate transaction using an ordinary just-created snapshot.  
The same disease afflicts any other GUC that requires catalog lookups  
and isn't forgiving about the lookups failing.  
  
To fix, postpone RestoreGUCState() into the worker's main transaction  
after we've set up a snapshot duplicating the leader's.  This affects  
check_transaction_isolation and check_transaction_deferrable, which  
think they should only run during transaction start.  Make them  
act like check_transaction_read_only, which already knows it should  
silently accept the value when InitializingParallelWorker.  
  
Per bug #18545 from Andrey Rachitskiy.  Back-patch to all  
supported branches, because this has been wrong for awhile.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/transam/parallel.c
M src/backend/commands/variable.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql

Doc: mention executor memory usage for enable_partitionwise* GUCs

commit   : 27146e79bf0301969945d73388f430fa8ff08870    
  
author   : David Rowley <[email protected]>    
date     : Thu, 1 Aug 2024 01:27:20 +1200    
  
committer: David Rowley <[email protected]>    
date     : Thu, 1 Aug 2024 01:27:20 +1200    

Click here for diff

Prior to this commit, the docs for enable_partitionwise_aggregate and  
enable_partitionwise_join mentioned the additional overheads enabling  
these causes for the query planner, but they mentioned nothing about the  
possible surge in work_mem-consuming executor nodes that could end up in  
the final plan.  Dimitrios reported the OOM killer intervened on his  
query as a result of using enable_partitionwise_aggregate=on.  
  
Here we adjust the docs to mention the possible increase in the number of  
work_mem-consuming executor nodes that can appear in the final plan as a  
result of enabling these GUCs.  
  
Reported-by: Dimitrios Apostolou  
Reviewed-by: Ashutosh Bapat  
Discussion: https://postgr.es/m/3603c380-d094-136e-e333-610914fb3e80%40gmx.net  
Discussion: https://postgr.es/m/CAApHDvoZ0_yqwPFEpb6h261L76BUpmh5GxBQq0LeRzQ5Jh3zzg@mail.gmail.com  
Backpatch-through: 12, oldest supported version  

M doc/src/sgml/config.sgml

Use DELETE instead of UPDATE to speed up vacuum test

commit   : 9744fe24118ba5738dda9008533ed00227f5d069    
  
author   : Melanie Plageman <[email protected]>    
date     : Mon, 29 Jul 2024 15:36:55 -0400    
  
committer: Melanie Plageman <[email protected]>    
date     : Mon, 29 Jul 2024 15:36:55 -0400    

Click here for diff

e2e992820f104def introduced a test which generated dead tuples for  
vacuum with an UPDATE. The test only required enough dead TIDs for two  
rounds of index vacuuming. This can be accomplished with a DELETE  
instead of an UPDATE -- which generates about 50% less WAL and makes the  
test 20% faster in many cases. The test takes several seconds (more on  
slow buildfarm animals) because we need quite a few tuples to trigger  
two rounds of index vacuuming; so it is worth a follow-on commit to  
speed it up.  
  
Suggested-by: Masahiko Sawada  
Discussion: https://postgr.es/m/CAAKRu_bWmMjmqL%2BOZ2duEQ80u7cRvpsExLNZNjzk-pXX5skwMQ%40mail.gmail.com  
Backpatch-through: 14, the first version containing this test.  

M src/test/recovery/t/043_vacuum_horizon_floor.pl

Fix incorrect return value for pg_size_pretty(bigint)

commit   : 0a80e88d902d3dfdeed03ede1880b8994a8d892b    
  
author   : David Rowley <[email protected]>    
date     : Sun, 28 Jul 2024 22:24:15 +1200    
  
committer: David Rowley <[email protected]>    
date     : Sun, 28 Jul 2024 22:24:15 +1200    

Click here for diff

pg_size_pretty(bigint) would return the value in bytes rather than PB  
for the smallest-most bigint value.  This happened due to an incorrect  
assumption that the absolute value of -9223372036854775808 could be  
stored inside a signed 64-bit type.  
  
Here we fix that by instead storing that value in an unsigned 64-bit type.  
  
This bug does exist in versions prior to 15 but the code there is  
sufficiently different and the bug seems sufficiently non-critical that  
it does not seem worth risking backpatching further.  
  
Author: Joseph Koshakow <[email protected]>  
Discussion: https://postgr.es/m/CAAvxfHdTsMZPWEHUrZ=h3cky9Ccc3Mtx2whUHygY+ABP-mCmUw@mail.gmail.com  
Backpatch-through: 15  

M src/backend/utils/adt/dbsize.c
M src/test/regress/expected/dbsize.out
M src/test/regress/sql/dbsize.sql

libpq: Use strerror_r instead of strerror

commit   : 6ddc8556ce4aff68446ebe00f14930bc0bbeccca    
  
author   : Peter Eisentraut <[email protected]>    
date     : Sun, 28 Jul 2024 09:12:00 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Sun, 28 Jul 2024 09:12:00 +0200    

Click here for diff

Commit 453c4687377 introduced a use of strerror() into libpq, but that  
is not thread-safe.  Fix by using strerror_r() instead.  
  
In passing, update some of the code comments added by 453c4687377, as  
we have learned more about the reason for the change in OpenSSL that  
started this.  
  
Reviewed-by: Daniel Gustafsson <[email protected]>  
Discussion: Discussion: https://postgr.es/m/[email protected]  

M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c

Fix building with MSVC for TLS session disabling

commit   : ce3045e9b0e5414638621a4941d8e7419fca9433    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Fri, 26 Jul 2024 19:09:27 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Fri, 26 Jul 2024 19:09:27 +0200    

Click here for diff

Commit 274bbced85 omitted the required changes for the MSVC build  
system in v16 through v12. Per buildfarm animal hamerkop.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/tools/msvc/Solution.pm

Fix macro placement in pg_config.h.in

commit   : 970cd5c62b728cffcd2aa100278b35ab8b31b7fe    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Fri, 26 Jul 2024 14:16:40 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Fri, 26 Jul 2024 14:16:40 +0200    

Click here for diff

Commit 274bbced85383e831dde accidentally placed the pg_config.h.in  
for SSL_CTX_set_num_tickets on the wrong line wrt where autoheader  
places it.  Fix by re-arranging and backpatch to the same level as  
the original commit.  
  
Reported-by: Marina Polyakova <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: v12  

M src/include/pg_config.h.in

Disable all TLS session tickets

commit   : 118ec331bfb7c781c03613de22de015229535a3b    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Fri, 26 Jul 2024 11:09:45 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Fri, 26 Jul 2024 11:09:45 +0200    

Click here for diff

OpenSSL supports two types of session tickets for TLSv1.3, stateless  
and stateful. The option we've used only turns off stateless tickets  
leaving stateful tickets active. Use the new API introduced in 1.1.1  
to disable all types of tickets.  
  
Backpatch to all supported versions.  
  
Reviewed-by: Heikki Linnakangas <[email protected]>  
Reported-by: Andres Freund <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: v12  

M configure
M configure.ac
M src/backend/libpq/be-secure-openssl.c
M src/include/pg_config.h.in

Doc: fix misleading syntax synopses for targetlists.

commit   : b7bc06f0cb5fb8add8ae4a79671f8a8611137531    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 25 Jul 2024 19:52:08 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 25 Jul 2024 19:52:08 -0400    

Click here for diff

In the syntax synopses for SELECT, INSERT, UPDATE, etc,  
SELECT ... and RETURNING ... targetlists were missing { ... }  
braces around an OR (|) operator.  That allows misinterpretation  
which could lead to confusion.  
  
David G. Johnston, per gripe from [email protected].  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/ref/delete.sgml
M doc/src/sgml/ref/insert.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/ref/select_into.sgml
M doc/src/sgml/ref/update.sgml

ci: Pin MacPorts version to 2.9.3.

commit   : 541f493dc1287a0038b8d235d61297f6fdbcb079    
  
author   : Thomas Munro <[email protected]>    
date     : Thu, 25 Jul 2024 14:46:01 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Thu, 25 Jul 2024 14:46:01 +1200    

Click here for diff

Commit d01ce180 invented a new way to find the latest MacPorts version.  
By bad luck, a new beta release has just been published, and it seems  
to lack some packages we need.  Go back to searching for this specific  
version for now.  We still search with a pattern so that we can find the  
package for the running version of macOS, but for now we always look for  
2.9.3.  The code to do that had been anticipated already in a commented  
out line, I just didn't expect to have to use it so soon...  
  
Also include the whole MacPorts installation script in the cache key, so  
that changes to the script cause a fresh installation.  This should make  
it a bit easier to reason about the effect of changes on cached state in  
github accounts using CI, when we make adjustments.  
  
Back-patch to 15, like d01ce180.  
  
Discussion: https://postgr.es/m/CA%2BhUKGLqJdv6RcwyZ_0H7khxtLTNJyuK%2BvDFzv3uwYbn8hKH6A%40mail.gmail.com  

M .cirrus.tasks.yml
M src/tools/ci/ci_macports_packages.sh

ci: Upgrade macOS version from 13 to 14.

commit   : 0151b4d6bdf0a35ba8cf6542d3d61aa3b3289270    
  
author   : Thomas Munro <[email protected]>    
date     : Thu, 25 Jul 2024 11:26:48 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Thu, 25 Jul 2024 11:26:48 +1200    

Click here for diff

1.  Previously we were using ghcr.io/cirruslabs/macos-XXX-base:latest  
images, but Cirrus has started ignoring that and using a particular  
image, currently ghcr.io/cirruslabs/macos-runner:sonoma, for github  
accounts using free CI resources (as opposed to dedicated runner  
machines, as cfbot uses).  Let's just ask for that image anyway, to stay  
in sync.  
  
2.  Instead of hard-coding a MacPorts installation URL, deduce it from  
the running macOS version and the available releases.  This removes the  
need to keep the ci_macports_packages.sh in sync with .cirrus.task.yml,  
and to advance the MacPorts version from time to time.  
  
3.  Change the cache key we use to cache the whole macports installation  
across builds to include the OS major version, to trigger a fresh  
installation when appropriate.  
  
Back-patch to 15 where CI began.  
  
Reviewed-by: Andres Freund <[email protected]>  
Discussion: https://postgr.es/m/CA%2BhUKGLqJdv6RcwyZ_0H7khxtLTNJyuK%2BvDFzv3uwYbn8hKH6A%40mail.gmail.com  

M .cirrus.tasks.yml
M src/tools/ci/ci_macports_packages.sh

Fix a missing article in the documentation

commit   : 350f4f96bb9b7188df7fd4f755fe94562436853b    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 24 Jul 2024 14:13:55 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 24 Jul 2024 14:13:55 +0200    

Click here for diff

Per complaint from Grant Gryczan.  
  
It's a very old typo; backpatch all the way back.  
  
Author: Laurenz Albe <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/parallel.sgml

Reset relhassubclass upon attaching table as a partition

commit   : f74fac06c6b134f574eee5145f22a95901d1fa28    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 24 Jul 2024 12:38:18 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 24 Jul 2024 12:38:18 +0200    

Click here for diff

We don't allow inheritance parents as partitions, and have checks to  
prevent this; but if a table _was_ in the past an inheritance parents  
and all their children are removed, the pg_class.relhassubclass flag  
may remain set, which confuses the partition pruning code (most  
obviously, it results in an assertion failure; in production builds it  
may be worse.)  
  
Fix by resetting relhassubclass on attach.  
  
Backpatch to all supported versions.  
  
Reported-by: Alexander Lakhin <[email protected]>  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/catalog/heap.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql

Detect integer overflow in array_set_slice().

commit   : 547dd2cbda12a7316c796999d24e6b3a4338e4a6    
  
author   : Nathan Bossart <[email protected]>    
date     : Tue, 23 Jul 2024 21:59:02 -0500    
  
committer: Nathan Bossart <[email protected]>    
date     : Tue, 23 Jul 2024 21:59:02 -0500    

Click here for diff

When provided an empty initial array, array_set_slice() fails to  
check for overflow when computing the new array's dimensions.  
While such overflows are ordinarily caught by ArrayGetNItems(),  
commands with the following form are accepted:  
  
	INSERT INTO t (i[-2147483648:2147483647]) VALUES ('{}');  
  
To fix, perform the hazardous computations using overflow-detecting  
arithmetic routines.  As with commit 18b585155a, the added test  
cases generate errors that include a platform-dependent value, so  
we again use psql's VERBOSITY parameter to suppress printing the  
message text.  
  
Reported-by: Alexander Lakhin  
Author: Joseph Koshakow  
Reviewed-by: Jian He  
Discussion: https://postgr.es/m/31ad2cd1-db94-bdb3-f91a-65ffdb4bef95%40gmail.com  
Backpatch-through: 12  

M src/backend/utils/adt/arrayfuncs.c
M src/test/regress/expected/arrays.out
M src/test/regress/sql/arrays.sql

Doc: improve description of plpgsql's FETCH and MOVE commands.

commit   : 9aa644f92959eca29680595e725a6e71b273f08d    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 22 Jul 2024 19:43:12 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 22 Jul 2024 19:43:12 -0400    

Click here for diff

We were not being clear about which variants of the "direction"  
clause are permitted in MOVE.  Also, the text seemed to be  
written with only the FETCH/MOVE NEXT case in mind, so it  
didn't apply very well to other variants.  
  
Also, document that "MOVE count IN cursor" only works if count  
is a constant.  This is not the whole truth, because some other  
cases such as a parenthesized expression will also work, but  
we want to push people to use "MOVE FORWARD count" instead.  
The constant case is enough to cover what we allow in plain SQL,  
and that seems sufficient to claim support for.  
  
Update a comment in pl_gram.y claiming that we don't document  
that point.  
  
Per gripe from Philipp Salvisberg.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/plpgsql.sgml
M src/pl/plpgsql/src/pl_gram.y

Correctly check updatability of columns targeted by INSERT...DEFAULT.

commit   : 96953052a12c9cc87700bdca5fa030626a80e707    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 20 Jul 2024 13:40:15 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 20 Jul 2024 13:40:15 -0400    

Click here for diff

If a view has some updatable and some non-updatable columns, we failed  
to verify updatability of any columns for which an INSERT or UPDATE  
on the view explicitly specifies a DEFAULT item (unless the view has  
a declared default for that column, which is rare anyway, and one  
would almost certainly not write one for a non-updatable column).  
This would lead to an unexpected "attribute number N not found in  
view targetlist" error rather than the intended error.  
  
Per bug #18546 from Alexander Lakhin.  This bug is old, so back-patch  
to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/updatable_views.sql

Add overflow checks to money type.

commit   : b82791c8fc2c6bc9d69f4362d7b7f46b9876fa46    
  
author   : Nathan Bossart <[email protected]>    
date     : Fri, 19 Jul 2024 11:52:32 -0500    
  
committer: Nathan Bossart <[email protected]>    
date     : Fri, 19 Jul 2024 11:52:32 -0500    

Click here for diff

None of the arithmetic functions for the the money type handle  
overflow.  This commit introduces several helper functions with  
overflow checking and makes use of them in the money type's  
arithmetic functions.  
  
Fixes bug #18240.  
  
Reported-by: Alexander Lakhin  
Author: Joseph Koshakow  
Discussion: https://postgr.es/m/18240-c5da758d7dc1ecf0%40postgresql.org  
Discussion: https://postgr.es/m/CAAvxfHdBPOyEGS7s%2Bxf4iaW0-cgiq25jpYdWBqQqvLtLe_t6tw%40mail.gmail.com  
Backpatch-through: 12  

M src/backend/utils/adt/cash.c
M src/test/regress/expected/money.out
M src/test/regress/sql/money.sql

Test that vacuum removes tuples older than OldestXmin

commit   : e2e992820f104defeea421fd0927b763421a0396    
  
author   : Melanie Plageman <[email protected]>    
date     : Fri, 19 Jul 2024 11:11:19 -0400    
  
committer: Melanie Plageman <[email protected]>    
date     : Fri, 19 Jul 2024 11:11:19 -0400    

Click here for diff

If vacuum fails to prune a tuple killed before OldestXmin, it will later  
find that tuple dead in lazy_scan_prune() and loop infinitely.  
  
Add a test reproducing this scenario to the recovery suite which creates  
a table on a primary, updates the table to generate dead tuples for  
vacuum, and then, during the vacuum, uses a replica to force  
GlobalVisState->maybe_needed on the primary to move backwards and  
precede the value of OldestXmin set at the beginning of vacuuming the  
table.  
  
This commit is separate from the fix in case there are test stability  
issues.  
  
Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com  
Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com  
  
Author: Melanie Plageman  
Reviewed-by: Peter Geoghegan  

A src/test/recovery/t/043_vacuum_horizon_floor.pl

Ensure vacuum removes all visibly dead tuples older than OldestXmin

commit   : dc6354c67017a3fe60744fcfe4936d14d72f2d7c    
  
author   : Melanie Plageman <[email protected]>    
date     : Fri, 19 Jul 2024 11:06:03 -0400    
  
committer: Melanie Plageman <[email protected]>    
date     : Fri, 19 Jul 2024 11:06:03 -0400    

Click here for diff

If vacuum fails to remove a tuple with xmax older than  
VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed,  
it will loop infinitely in lazy_scan_prune(), which compares tuples'  
visibility information to OldestXmin.  
  
Starting in version 14, which uses GlobalVisState for visibility testing  
during pruning, it is possible for GlobalVisState->maybe_needed to  
precede OldestXmin if maybe_needed is forced to go backward while vacuum  
is running. This can happen if a disconnected standby with a running  
transaction older than VacuumCutoffs->OldestXmin reconnects to the  
primary after vacuum initially calculates GlobalVisState and OldestXmin.  
  
Fix this by having vacuum always remove tuples older than OldestXmin  
during pruning. This is okay because the standby won't replay the tuple  
removal until the tuple is removable. Thus, the worst that can happen is  
a recovery conflict.  
  
Fixes BUG# 17257  
  
Back-patched in versions 14-17  
  
Author: Melanie Plageman  
Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas  
Discussion: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com  

M src/backend/access/heap/pruneheap.c
M src/backend/access/heap/vacuumlazy.c
M src/include/access/heapam.h

postgres_fdw: Avoid "cursor can only scan forward" error.

commit   : f39f3e0fbe9b323aba41cf8c258f17259ab69a85    
  
author   : Etsuro Fujita <[email protected]>    
date     : Fri, 19 Jul 2024 13:15:05 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Fri, 19 Jul 2024 13:15:05 +0900    

Click here for diff

Commit d844cd75a disallowed rewind in a non-scrollable cursor to resolve  
anomalies arising from such a cursor operation.  However, this failed to  
take into account the assumption in postgres_fdw that when rescanning a  
foreign relation, it can rewind the cursor created for scanning the  
foreign relation without specifying the SCROLL option, regardless of its  
scrollability, causing this error when it tried to do such a rewind in a  
non-scrollable cursor.  Fix by modifying postgres_fdw to instead  
recreate the cursor, regardless of its scrollability, when rescanning  
the foreign relation.  (If we had a way to check its scrollability, we  
could improve this by rewinding it if it is scrollable and recreating it  
if not, but we do not have it, so this commit modifies it to recreate it  
in any case.)  
  
Per bug #17889 from Eric Cyr.  Devrim Gunduz also reported this problem.  
Back-patch to v15 where that commit enforced the prohibition.  
  
Reviewed by Tom Lane.  
  
Discussion: https://postgr.es/m/17889-e8c39a251d258dda%40postgresql.org  
Discussion: https://postgr.es/m/b415ac3255f8352d1ea921cf3b7ba39e0587768a.camel%40gunduz.org  

M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql

Avoid error in recovery test if history file is not yet present

commit   : bc9d706a19629a03180887bfa061a9a0ec6f5d7e    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 17 Jul 2024 10:35:50 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 17 Jul 2024 10:35:50 -0400    

Click here for diff

Error was detected when testing use of libpq sessions instead of psql  
for polling queries.  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch to all live branches  

M src/test/recovery/t/002_archiving.pl

Fix bad indentation introduced in 43cd30bcd1c

commit   : 11441ad48d09740efd99edb654534cc4662b6ecc    
  
author   : Andres Freund <[email protected]>    
date     : Mon, 15 Jul 2024 15:17:28 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Mon, 15 Jul 2024 15:17:28 -0700    

Click here for diff

Oops.  
  
Reported-by: Nathan Bossart <[email protected]>  
Discussion: https://postgr.es/m/ZpVZB9rH5tHllO75@nathan  
Backpatch: 12-, like 43cd30bcd1c  

M src/backend/utils/misc/guc.c

ci: Upgrade to Debian Bookworm

commit   : 27004d671162283840972113dbc7a22960cbbb07    
  
author   : Andres Freund <[email protected]>    
date     : Mon, 15 Jul 2024 09:26:02 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Mon, 15 Jul 2024 09:26:02 -0700    

Click here for diff

Bullseye is getting long in the tooth, upgrade to the current stable version.  
  
Backpatch to all versions with CI support, we don't want to generate CI images  
for multiple Debian versions.  
  
Author: Nazir Bilal Yavuz <[email protected]>  
Discussion: https://postgr.es/m/CAN55FZ0fY5EFHXLKCO_%3Dp4pwFmHRoVom_qSE_7B48gpchfAqzw%40mail.gmail.com  
Backpatch: 15-, where CI was added  

M .cirrus.tasks.yml

Fix type confusion in guc_var_compare()

commit   : b9f3db23b7b5c7f522162e16482b096d9c930f80    
  
author   : Andres Freund <[email protected]>    
date     : Mon, 15 Jul 2024 09:26:02 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Mon, 15 Jul 2024 09:26:02 -0700    

Click here for diff

Before this change guc_var_compare() cast the input arguments to  
const struct config_generic *.  That's not quite right however, as the input  
on one side is often just a char * on one side.  
  
Instead just use char *, the first field in config_generic.  
  
This fixes a -Warray-bounds warning with some versions of gcc. While the  
warning is only known to be triggered for <= 15, the issue the warning points  
out seems real, so apply the fix everywhere.  
  
Author: Nazir Bilal Yavuz <[email protected]>  
Reported-by: Erik Rijkers <[email protected]>  
Suggested-by: Andres Freund <[email protected]>  
Discussion: https://postgr.es/m/a74a1a0d-0fd2-3649-5224-4f754e8f91aa%40xs4all.nl  

M src/backend/utils/misc/guc.c

Avoid unhelpful internal error for incorrect recursive-WITH queries.

commit   : e7f9f44e3bd95404f4221fab29cd295805ff410d    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 14 Jul 2024 13:49:46 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 14 Jul 2024 13:49:46 -0400    

Click here for diff

checkWellFormedRecursion would issue "missing recursive reference"  
if a WITH RECURSIVE query contained a single self-reference but  
that self-reference was inside a top-level WITH, ORDER BY, LIMIT,  
etc, rather than inside the second arm of the UNION as expected.  
We already intended to throw more-on-point errors for such cases,  
but those error checks must be done before examining the UNION arm  
in order to have the desired results.  So this patch need only  
move some code (and improve the comments).  
  
Per bug #18536 from Alexander Lakhin.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/parser/parse_cte.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql

Don't lose partitioned table reltuples=0 after relhassubclass=f.

commit   : 2b4a2a79edd258e13a44601d17b44e9929abab14    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 13 Jul 2024 08:09:33 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 13 Jul 2024 08:09:33 -0700    

Click here for diff

ANALYZE sets relhassubclass=f when a partitioned table no longer has  
partitions.  An ANALYZE doing that proceeded to apply the inplace update  
of pg_class.reltuples to the old pg_class tuple instead of the new  
tuple, losing that reltuples=0 change if the ANALYZE committed.  
Non-partitioning inheritance trees were unaffected.  Back-patch to v14,  
where commit 375aed36ad83f0e021e9bdd3a0034c0c992c66dc introduced  
maintenance of partitioned table pg_class.reltuples.  
  
Reported by Alexander Lakhin.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/analyze.c
M src/test/regress/expected/vacuum.out
M src/test/regress/sql/vacuum.sql

Make sure to run pg_isready on correct port

commit   : 6f55b4f112ab72ad392b0acfd5bf9b827e9bb458    
  
author   : Andrew Dunstan <[email protected]>    
date     : Fri, 12 Jul 2024 18:29:15 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Fri, 12 Jul 2024 18:29:15 -0400    

Click here for diff

The current code can have pg_isready unexpectedly succeed if there is a  
server running on the default port. To avoid this we delay running the  
test until after a node has been created but before it starts, and then  
use that node's port, so we are fairly sure there is nothing running on  
the port.  
  
Backpatch to all live branches.  

M src/bin/scripts/t/080_pg_isready.pl

Fix lost Windows socket EOF events.

commit   : 5546a834cc6fa49057356eb90ecbd9ee12d7d58a    
  
author   : Thomas Munro <[email protected]>    
date     : Sat, 13 Jul 2024 14:59:46 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Sat, 13 Jul 2024 14:59:46 +1200    

Click here for diff

Winsock only signals an FD_CLOSE event once if the other end of the  
socket shuts down gracefully.  Because each WaitLatchOrSocket() call  
constructs and destroys a new event handle every time, with unlucky  
timing we can lose it and hang.  We get away with this only if the other  
end disconnects non-gracefully, because FD_CLOSE is repeatedly signaled  
in that case.  
  
To fix this design flaw in our Windows socket support fundamentally,  
we'd probably need to rearchitect it so that a single event handle  
exists for the lifetime of a socket, or switch to completely different  
multiplexing or async I/O APIs.  That's going to be a bigger job  
and probably wouldn't be back-patchable.  
  
This brute force kludge closes the race by explicitly polling with  
MSG_PEEK before sleeping.  
  
Back-patch to all supported releases.  This should hopefully clear up  
some random build farm and CI hang failures reported over the years.  It  
might also allow us to try using graceful shutdown in more places again  
(reverted in commit 29992a6) to fix instability in the transmission of  
FATAL error messages, but that isn't done by this commit.  
  
Reported-by: Tom Lane <[email protected]>  
Tested-by: Alexander Lakhin <[email protected]>  
Discussion: https://postgr.es/m/176008.1715492071%40sss.pgh.pa.us  

M src/backend/storage/ipc/latch.c

Add ORDER BY to new test query

commit   : 9f0f72d89ff6cff8261fd793257147cbb268a6ae    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 12 Jul 2024 13:44:19 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 12 Jul 2024 13:44:19 +0200    

Click here for diff

Per buildfarm.  

M src/test/regress/expected/constraints.out
M src/test/regress/sql/constraints.sql

Fix ALTER TABLE DETACH for inconsistent indexes

commit   : 4ae09c59d6f563531c456eda3147c849773ceaa7    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 12 Jul 2024 12:54:01 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 12 Jul 2024 12:54:01 +0200    

Click here for diff

When a partitioned table has an index that doesn't support a constraint,  
but a partition has an equivalent index that does, then a DETACH  
operation would misbehave: a crash in assertion-enabled systems (because  
we fail to find the constraint in the parent that we expect to), or a  
broken coninhcount value (-1) in production systems (because we blindly  
believe that we've successfully detached the parent).  
  
While we should reject an ATTACH of a partition with such an index, we  
have failed to do so in existing releases, so adding an error in stable  
releases might break the (unlikely) existing applications that rely on  
this behavior.  At this point I don't even want to reject them in  
master, because it'd break pg_upgrade if such databases exist, and there  
would be no easy way to fix existing databases without expensive index  
rebuilds.  
  
(Later on we could add ALTER TABLE ... ADD CONSTRAINT USING INDEX to  
partitioned tables, which would allow the user to fix such patterns.  At  
that point we could add more restrictions to prevent the problem from  
its root.)  
  
Also, add a test case that leaves one table in this condition, so that  
we can verify that pg_upgrade continues to work if we later decide to  
change the policy on the master branch.  
  
Backpatch to all supported branches.  
  
Co-authored-by: Tender Wang <[email protected]>  
Reported-by: Alexander Lakhin <[email protected]>  
Reviewed-by: Tender Wang <[email protected]>  
Reviewed-by: Michael Paquier <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/tablecmds.c
M src/test/regress/expected/constraints.out
M src/test/regress/sql/constraints.sql

Disable clang 16's -Wcast-function-type-strict.

commit   : 6e2552d180a0a3a694c2f91ec01bec51d9fed671    
  
author   : Thomas Munro <[email protected]>    
date     : Tue, 13 Dec 2022 10:03:28 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Tue, 13 Dec 2022 10:03:28 +1300    

Click here for diff

This is a back-patch of commit 101c37cd into REL_14_STABLE and  
REL_15_STABLE.  Those branches had commit de8feb1f3, which turned on  
-Wcast-function-type, but did not disable -Wcast-function-type-strict.  
This silences warnings about function pointer types without prototypes  
based on new C23 rules, that we want to suppress for now.  
  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/CA%2BhUKGKnBvdrbH2LW%2B7-Lv599t9JFOHjx%3Dxw-VQmdoj%3D9585CQ%40mail.gmail.com  
Discussion: https://postgr.es/m/CA%2BhUKGJvX%2BL3aMN84ksT-cGy08VHErRNip3nV-WmTx7f6Pqhyw%40mail.gmail.com  

M configure
M configure.ac

Fix possibility of logical decoding partial transaction changes.

commit   : aee8c2b95492e1b0a228184cd9147b1c0749d673    
  
author   : Masahiko Sawada <[email protected]>    
date     : Thu, 11 Jul 2024 22:48:16 +0900    
  
committer: Masahiko Sawada <[email protected]>    
date     : Thu, 11 Jul 2024 22:48:16 +0900    

Click here for diff

When creating and initializing a logical slot, the restart_lsn is set  
to the latest WAL insertion point (or the latest replay point on  
standbys). Subsequently, WAL records are decoded from that point to  
find the start point for extracting changes in the  
DecodingContextFindStartpoint() function. Since the initial  
restart_lsn could be in the middle of a transaction, the start point  
must be a consistent point where we won't see the data for partial  
transactions.  
  
Previously, when not building a full snapshot, serialized snapshots  
were restored, and the SnapBuild jumps to the consistent state even  
while finding the start point. Consequently, the slot's restart_lsn  
and confirmed_flush could be set to the middle of a transaction. This  
could lead to various unexpected consequences. Specifically, there  
were reports of logical decoding decoding partial transactions, and  
assertion failures occurred because only subtransactions were decoded  
without decoding their top-level transaction until decoding the commit  
record.  
  
To resolve this issue, the changes prevent restoring the serialized  
snapshot and jumping to the consistent state while finding the start  
point.  
  
On v17 and HEAD, a flag indicating whether snapshot restores should be  
skipped has been added to the SnapBuild struct, and SNAPBUILD_VERSION  
has been bumpded.  
  
On backbranches, the flag is stored in the LogicalDecodingContext  
instead, preserving on-disk compatibility.  
  
Backpatch to all supported versions.  
  
Reported-by: Drew Callahan  
Reviewed-by: Amit Kapila, Hayato Kuroda  
Discussion: https://postgr.es/m/2444AA15-D21B-4CCE-8052-52C7C2DAFE5C%40amazon.com  
Backpatch-through: 12  

M contrib/test_decoding/Makefile
A contrib/test_decoding/expected/skip_snapshot_restore.out
A contrib/test_decoding/specs/skip_snapshot_restore.spec
M src/backend/replication/logical/logical.c
M src/backend/replication/logical/snapbuild.c
M src/include/replication/logical.h

Make our back branches compatible with libxml2 2.13.x.

commit   : f68d6aabb7e2c803818185b49a3d356bdb2b2974    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 10 Jul 2024 20:15:52 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 10 Jul 2024 20:15:52 -0400    

Click here for diff

This back-patches HEAD commits 066e8ac6e, 6082b3d5d, e7192486d,  
and 896cd266f into supported branches.  Changes:  
  
* Use xmlAddChildList not xmlAddChild in XMLSERIALIZE  
(affects v16 and up only).  This was a flat-out coding mistake  
that we got away with due to lax checking in previous versions  
of xmlAddChild.  
  
* Use xmlParseInNodeContext not xmlParseBalancedChunkMemory.  
This is to dodge a bug in xmlParseBalancedChunkMemory in libxm2  
releases 2.13.0-2.13.2.  While that bug is now fixed upstream and  
will probably never be seen in any production-oriented distro, it is  
currently a problem on some more-bleeding-edge-friendly platforms.  
  
* Suppress "chunk is not well balanced" errors from libxml2,  
unless it is the only error.  This eliminates an error-reporting  
discrepancy between 2.13 and older releases.  This error is  
almost always redundant with previous errors, if not flat-out  
inappropriate, which is why 2.13 changed the behavior and why  
nobody's likely to miss it.  
  
Erik Wienhold and Tom Lane, per report from Frank Streitzig.  
  
Discussion: https://postgr.es/m/trinity-b0161630-d230-4598-9ebc-7a23acdb37cb-1720186432160@3c-app-gmx-bap25  
Discussion: https://postgr.es/m/trinity-361ba18b-541a-4fe7-bc63-655ae3a7d599-1720259822452@3c-app-gmx-bs01  

M src/backend/utils/adt/xml.c
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_2.out

commit   : 468b2367d2fe29fe1a53bcba408b472dc8f5051b    
  
author   : Andrew Dunstan <[email protected]>    
date     : Mon, 8 Jul 2024 13:46:21 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Mon, 8 Jul 2024 13:46:21 -0400    

Click here for diff

This reverts commit e9f15bc9. Instead of a hacky solution that didn't  
work on Windows, we avoid trying to move the directory possibly across  
drives, and instead remove it and recreate it in the new location.  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch to release 14 like the previous patch.  

M src/bin/pg_basebackup/t/010_pg_basebackup.pl

Choose ports for test servers less likely to result in conflicts

commit   : 33470429f2bdb967d945b72a5b553fbc6d1d8f96    
  
author   : Andrew Dunstan <[email protected]>    
date     : Mon, 8 Jul 2024 11:18:06 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Mon, 8 Jul 2024 11:18:06 -0400    

Click here for diff

If we choose ports in the range typically used for ephemeral ports there  
is a danger of encountering a port conflict due to a race condition  
between the time we choose the port in a range below that typically used  
to allocate ephemeral ports, but higher than the range typically used by  
well known services.  
  
Author: Jelte Fenema-Nio, with some editing by me.  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch to all live branches (12 and up)  

M src/test/perl/PostgreSQL/Test/Cluster.pm

Force nodes for SSL tests to start in TCP mode

commit   : e28850111eebdbfa67cd07887adfe7f7629f3c39    
  
author   : Andrew Dunstan <[email protected]>    
date     : Mon, 8 Jul 2024 05:51:26 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Mon, 8 Jul 2024 05:51:26 -0400    

Click here for diff

Currently they are started in unix socket mode in ost cases, and then  
converted to run in TCP mode. This can result in port collisions, and  
there is no virtue in startng in unix socket mode, so start as we will  
be going on.  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch to all live branches (12 and up).  

M src/test/ssl/t/SSL/Server.pm

Fix scale clamping in numeric round() and trunc().

commit   : 47ca912de02975458fca657435d558725dbd6b64    
  
author   : Dean Rasheed <[email protected]>    
date     : Mon, 8 Jul 2024 17:54:22 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Mon, 8 Jul 2024 17:54:22 +0100    

Click here for diff

The numeric round() and trunc() functions clamp the scale argument to  
the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much  
smaller than the actual allowed range of type numeric. As a result,  
they return incorrect results when asked to round/truncate more than  
2000 digits before or after the decimal point.  
  
Fix by using the correct upper and lower scale limits based on the  
actual allowed (and documented) range of type numeric.  
  
While at it, use the new NUMERIC_WEIGHT_MAX constant instead of  
SHRT_MAX in all other overflow checks, and fix a comment thinko in  
power_var() introduced by e54a758d24 -- the minimum value of  
ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this  
doesn't affect the point being made in the comment, that the resulting  
local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000).  
  
Back-patch to all supported branches.  
  
Dean Rasheed, reviewed by Joel Jacobson.  
  
Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql

Cope with <regex.h> name clashes.

commit   : 467d77bb163416914b997823aa4dd7be64bebc11    
  
author   : Thomas Munro <[email protected]>    
date     : Sat, 6 Jul 2024 10:24:49 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Sat, 6 Jul 2024 10:24:49 +1200    

Click here for diff

macOS 15's SDK pulls in headers related to <regex.h> when we include  
<xlocale.h>.  This causes our own regex_t implementation to clash with  
the OS's regex_t implementation.  Luckily our function names already had  
pg_ prefixes, but the macros and typenames did not.  
  
Include <regex.h> explicitly on all POSIX systems, and fix everything  
that breaks.  Then we can prove that we are capable of fully hiding and  
replacing the system regex API with our own.  
  
1.  Deal with standard-clobbering macros by undefining them all first.  
POSIX says they are "symbolic constants".  If they are macros, this  
allows us to redefine them.  If they are enums or variables, our macros  
will hide them.  
  
2.  Deal with standard-clobbering types by giving our types pg_  
prefixes, and then using macros to redirect xxx_t -> pg_xxx_t.  
  
After including our "regex/regex.h", the system <regex.h> is hidden,  
because we've replaced all the standard names.  The PostgreSQL source  
tree and extensions can continue to use standard prefix-less type and  
macro names, but reach our implementation, if they included our  
"regex/regex.h" header.  
  
Back-patch to all supported branches, so that macOS 15's tool chain can  
build them.  
  
Reported-by: Stan Hu <[email protected]>  
Suggested-by: Tom Lane <[email protected]>  
Tested-by: Aleksander Alekseev <[email protected]>  
Discussion: https://postgr.es/m/CAMBWrQnEwEJtgOv7EUNsXmFw2Ub4p5P%2B5QTBEgYwiyjy7rAsEQ%40mail.gmail.com  

M src/include/regex/regex.h
M src/tools/pgindent/typedefs.list

Doc: small improvements in discussion of geometric data types.

commit   : 7961cd49e26b35db65516847b94ee15b8b0f3580    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 4 Jul 2024 13:23:32 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 4 Jul 2024 13:23:32 -0400    

Click here for diff

State explicitly that the coordinates in our geometric data types are  
float8.  Also explain that polygons store their bounding box.  
  
While here, fix the table of geometric data types to show type  
"line"'s size correctly: it's 24 bytes not 32.  This has somehow  
escaped notice since that table was made in 1998.  
  
Per suggestion from Sebastian Skałacki.  The size error seems  
important enough to justify back-patching.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/datatype.sgml

doc: Specify when ssl_prefer_server_ciphers was added

commit   : 0e3df0e3c96cb449e81c74699e658252bc5cc67a    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Thu, 4 Jul 2024 11:38:37 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Thu, 4 Jul 2024 11:38:37 +0200    

Click here for diff

The ssl_prefer_server_ciphers setting is quite important from a  
security point of view, so simply stating that older versions  
doesn't have it isn't very helpful.  This adds the version when  
the GUC was added to help readers.  
  
Backpatch to all supported versions since this setting has been  
around since 9.4.  
  
Reviewed-by: Peter Eisentraut <[email protected]>  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: v12  

M doc/src/sgml/config.sgml

Fix missing installation/uninstallation rules for BackgroundPsql.pm

commit   : 9d7506a2f7cb2b040fc78886fb2b2b6d0feb68a1    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Mon, 1 Jul 2024 19:22:36 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Mon, 1 Jul 2024 19:22:36 +0300    

Click here for diff

Commit d5fd7865 backported BackgroundPsql perl module module with  
helper functions for tests running interactive or background psql  
tasks to PG 12 to 15, but did not add installation/uninstallation  
rules of the build system, causing problems running TAP tests for the  
extensions.  
  
Author: Pavan Deolasee <[email protected]>  
Discussion: https://www.postgresql.org/message-id/CABOikdPmRuZrcf_gtgXmQzZ5Tbg9yUJmqXDCAZ2aW%3DWi-PbDyQ%40mail.gmail.com  

M src/test/perl/Makefile

Preserve CurrentMemoryContext across notify and sinval interrupts.

commit   : 4df767cf906d4406306f5853fea0e8133803c0f1    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 1 Jul 2024 12:21:07 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 1 Jul 2024 12:21:07 -0400    

Click here for diff

ProcessIncomingNotify is called from the main processing loop that  
normally runs in MessageContext.  That outer-loop code assumes that  
whatever it allocates will be cleaned up when we're done processing  
the current client message --- but if we service a notify interrupt,  
then whatever gets allocated before the next switch into  
MessageContext will be permanently leaked in TopMemoryContext,  
because CommitTransactionCommand sets CurrentMemoryContext to  
TopMemoryContext.  There are observable leaks associated with  
(at least) encoding conversion of incoming queries and parameters  
attached to Bind messages.  
  
sinval catchup interrupts have a similar problem.  There might be  
others, but I've not identified any other clear cases.  
  
To fix, take care to save and restore CurrentMemoryContext across  
the Start/CommitTransactionCommand calls in these functions.  
  
Per bug #18512 from wizardbrony.  Commit to back branches only;  
in HEAD, this was dealt with by the riskier but more thoroughgoing  
approach in commit 1afe31f03.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/async.c
M src/backend/storage/ipc/sinval.c

Remove configuration-dependent output from new inplace-inval test.

commit   : 458fada72b3288d648a559a17c5a69f1d17ae15f    
  
author   : Noah Misch <[email protected]>    
date     : Fri, 28 Jun 2024 09:33:40 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Fri, 28 Jun 2024 09:33:40 -0700    

Click here for diff

Per buildfarm members prion and trilobite.  Back-patch to v12 (all  
supported versions), like commit  
0844b3968985447ed0a6937cfc8639e379da2fe6.  
  
Strategy reviewed by Tom Lane.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/test/isolation/expected/inplace-inval.out
M src/test/isolation/specs/inplace-inval.spec

Remove comment about xl_heap_inplace "AT END OF STRUCT".

commit   : 2ca8ca4827af253efc2852f65c3f1366b90b5130    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:06 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:06 -0700    

Click here for diff

Commit 2c03216d831160bedd72d45f712601b6f7d03f1c moved the tuple data  
from there to the buffer-0 data.  Back-patch to v12 (all supported  
versions), the plan for the next change to this struct.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/include/access/heapam_xlog.h

Cope with inplace update making catcache stale during TOAST fetch.

commit   : b08a4b6163ebb10d0fcfabbccd49eeb23cef70d3    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:06 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:06 -0700    

Click here for diff

This extends ad98fb14226ae6456fbaed7990ee7591cbe5efd2 to invals of  
inplace updates.  Trouble requires an inplace update of a catalog having  
a TOAST table, so only pg_database was at risk.  (The other catalog on  
which core code performs inplace updates, pg_class, has no TOAST table.)  
Trouble would require something like the inplace-inval.spec test.  
Consider GRANT ... ON DATABASE fetching a stale row from cache and  
discarding a datfrozenxid update that vac_truncate_clog() has already  
relied upon.  Back-patch to v12 (all supported versions).  
  
Reviewed (in an earlier version) by Robert Haas.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/catalog/catalog.c
M src/backend/utils/cache/catcache.c
M src/include/catalog/catalog.h

AccessExclusiveLock new relations just after assigning the OID.

commit   : 0f2835ea6fc140869b062f61c62a87acf3295153    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    

Click here for diff

This has no user-visible, important consequences, since other sessions'  
catalog scans can't find the relation until we commit.  However, this  
unblocks introducing a rule about locks required to heap_update() a  
pg_class row.  CREATE TABLE has been acquiring this lock eventually, but  
it can heap_update() pg_class.relchecks earlier.  create_toast_table()  
has been acquiring only ShareLock.  Back-patch to v12 (all supported  
versions), the plan for the commit relying on the new rule.  
  
Reviewed (in an earlier version) by Robert Haas.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/catalog/heap.c

Lock before setting relhassubclass on RELKIND_PARTITIONED_INDEX.

commit   : a338e41374e4cf7efded6839ac58a60916b31ba7    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    

Click here for diff

Commit 5b562644fec696977df4a82790064e8287927891 added a comment that  
SetRelationHasSubclass() callers must hold this lock.  When commit  
17f206fbc824d2b4b14480199ca9ff7dea417eda extended use of this column to  
partitioned indexes, it didn't take the lock.  As the latter commit  
message mentioned, we currently never reset a partitioned index to  
relhassubclass=f.  That largely avoids harm from the lock omission.  The  
cause for fixing this now is to unblock introducing a rule about locks  
required to heap_update() a pg_class row.  This might cause more  
deadlocks.  It gives minor user-visible benefits:  
  
- If an ALTER INDEX SET TABLESPACE runs concurrently with ALTER TABLE  
  ATTACH PARTITION or CREATE PARTITION OF, one transaction blocks  
  instead of failing with "tuple concurrently updated".  (Many cases of  
  DDL concurrency still fail that way.)  
  
- Match ALTER INDEX ATTACH PARTITION in choosing to lock the index.  
  
While not user-visible today, we'll need this if we ever make something  
set the flag to false for a partitioned index, like ANALYZE does today  
for tables.  Back-patch to v12 (all supported versions), the plan for  
the commit relying on the new rule.  In back branches, add  
LockOrStrongerHeldByMe() instead of adding a LockHeldByMe() parameter.  
  
Reviewed (in an earlier version) by Robert Haas.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/catalog/index.c
M src/backend/commands/indexcmds.c
M src/backend/commands/tablecmds.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/include/storage/lmgr.h
M src/include/storage/lock.h

Lock owned sequences during ALTER TABLE SET { LOGGED | UNLOGGED }.

commit   : 24561b498f117fc7ec58baba27b0d89c9efab376    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    

Click here for diff

These commands already make the persistence of owned sequences follow  
owned table persistence changes.  They didn't lock those sequences.  
They lost the effect of nextval() calls that other sessions make after  
the ALTER TABLE command, before the ALTER TABLE transaction commits.  
Fix by acquiring the same lock that ALTER SEQUENCE SET { LOGGED |  
UNLOGGED } acquires.  This might cause more deadlocks.  Back-patch to  
v15, where commit 344d62fb9a978a72cf8347f0369b9ee643fd0b31 introduced  
unlogged sequences.  
  
Reviewed (in an earlier version) by Robert Haas.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/sequence.c

Expand comments and add an assertion in nodeModifyTable.c.

commit   : f0544432c3cc5e74817dac146196dabcf3de6c25    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    

Click here for diff

Most comments concern RELKIND_VIEW.  One addresses the ExecUpdate()  
"tupleid" parameter.  A later commit will rely on these facts, but they  
hold already.  Back-patch to v12 (all supported versions), the plan for  
that commit.  
  
Reviewed (in an earlier version) by Robert Haas.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/executor/nodeModifyTable.c

Improve test coverage for changes to inplace-updated catalogs.

commit   : 3c6becd3d18011c7cd018bad5294d6d60f878a5f    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 27 Jun 2024 19:21:05 -0700    

Click here for diff

This covers both regular and inplace changes, since bugs arise at their  
intersection.  Where marked, these witness extant bugs.  Back-patch to  
v12 (all supported versions).  
  
Reviewed (in an earlier version) by Robert Haas.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/test/isolation/expected/eval-plan-qual.out
A src/test/isolation/expected/inplace-inval.out
A src/test/isolation/expected/intra-grant-inplace-db.out
A src/test/isolation/expected/intra-grant-inplace.out
M src/test/isolation/isolation_schedule
M src/test/isolation/specs/eval-plan-qual.spec
A src/test/isolation/specs/inplace-inval.spec
A src/test/isolation/specs/intra-grant-inplace-db.spec
A src/test/isolation/specs/intra-grant-inplace.spec
M src/test/recovery/t/027_stream_regress.pl
A src/test/regress/expected/database.out
M src/test/regress/expected/merge.out
M src/test/regress/parallel_schedule
A src/test/regress/sql/database.sql
M src/test/regress/sql/merge.sql

Avoid crashing when a JIT-inlined backend function throws an error.

commit   : 5401e70e47aab30a24f35a6fc909c52a0ccbbe3f    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 27 Jun 2024 14:43:59 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 27 Jun 2024 14:43:59 -0400    

Click here for diff

errfinish() assumes that the __FUNC__ and __FILE__ arguments it's  
passed are compile-time constant strings that can just be pointed  
to rather than physically copied.  However, it's possible for LLVM  
to generate code in which those pointers point into a dynamically  
loaded code segment.  If that segment gets unloaded before we're  
done with the ErrorData struct, we have dangling pointers that  
will lead to SIGSEGV.  In simple cases that won't happen, because we  
won't unload LLVM code before end of transaction.  But it's possible  
to happen if the error is thrown within end-of-transaction code run by  
_SPI_commit or _SPI_rollback, because since commit 2e517818f those  
functions clean up by ending the transaction and starting a new one.  
  
Rather than fixing this by adding pstrdup() overhead to every  
elog/ereport sequence, let's fix it by copying the risky pointers  
in CopyErrorData().  That solves it for _SPI_commit/_SPI_rollback  
because they use that function to preserve the error data across  
the transaction end/restart sequence; and it seems likely that  
any other code doing something similar would need to do that too.  
  
I'm suspicious that this behavior amounts to an LLVM bug (or a  
bug in our use of it?), because it implies that string constant  
references that should be pointer-equal according to a naive  
understanding of C semantics will sometimes not be equal.  
However, even if it is a bug and someday gets fixed, we'll have  
to cope with the current behavior for a long time to come.  
  
Report and patch by me.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/error/elog.c

Fix MVCC bug with prepared xact with subxacts on standby

commit   : 0e2f3d78bf5ab869da0dbefeba629d74d4078655    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 27 Jun 2024 21:06:32 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 27 Jun 2024 21:06:32 +0300    

Click here for diff

We did not recover the subtransaction IDs of prepared transactions  
when starting a hot standby from a shutdown checkpoint. As a result,  
such subtransactions were considered as aborted, rather than  
in-progress. That would lead to hint bits being set incorrectly, and  
the subtransactions suddenly becoming visible to old snapshots when  
the prepared transaction was committed.  
  
To fix, update pg_subtrans with prepared transactions's subxids when  
starting hot standby from a shutdown checkpoint. The snapshots taken  
from that state need to be marked as "suboverflowed", so that we also  
check the pg_subtrans.  
  
Backport to all supported versions.  
  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M src/backend/access/transam/twophase.c
M src/backend/access/transam/xlog.c
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/include/storage/standby.h
M src/test/recovery/t/009_twophase.pl
M src/tools/pgindent/typedefs.list

tests: Trim newline from result returned by BackgroundPsql->query

commit   : 8b6a7723b5a4b5e9540a7a2f66200b67aae27dfb    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 27 Jun 2024 21:06:27 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 27 Jun 2024 21:06:27 +0300    

Click here for diff

This went unnoticed, because only a few existing callers of  
BackgroundPsql->query used the result, and the ones that did were not  
bothered by an extra newline. I noticed because I was about to add a  
new test that checks the result.  
  
Backport to all supported versions, since I just backported the  
BackgroundPsql facility to all supported versions too.  

M src/test/perl/PostgreSQL/Test/BackgroundPsql.pm

Fix thinkos in comments

commit   : 468ffee47bcb9c8b7b4f7b3168aef804a3ad802c    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 27 Jun 2024 19:51:47 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 27 Jun 2024 19:51:47 +0200    

Click here for diff

The first one was noticed by Tender Wang and introduced with  
8aba9322511f; the other one was newly introduced with dbca3469ebf8.  

M src/backend/executor/execPartition.c

Backport BackgroundPsql perl test module

commit   : d5fd7865f068209162130a1a392367bf874b9e4d    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 27 Jun 2024 19:00:59 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 27 Jun 2024 19:00:59 +0300    

Click here for diff

Backport the new BackgroundPsql modules and the constructor functions,  
background_psql() and interactive_psql, to all supported  
branches. That makes it easier to backpatch tests that use it.  
  
BackgroundPsql was introduced in version 16. On version 16, this  
commit backports just the new timeout argument from master (commit  
334f512f45). On older branches, the whole facility. This includes the  
change to `use warnings FATAL => 'all'`, which we haven't otherwise  
backported, but it seems good to keep the file identical across  
branches.  
  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M contrib/amcheck/t/003_cic_2pc.pl
M src/bin/psql/t/010_tab_completion.pl
A src/test/perl/PostgreSQL/Test/BackgroundPsql.pm
M src/test/perl/PostgreSQL/Test/Cluster.pm
M src/test/recovery/t/010_logical_decoding_timelines.pl
M src/test/recovery/t/031_recovery_conflict.pl
M src/test/recovery/t/037_invalid_database.pl
M src/test/subscription/t/015_stream.pl

Drop the temporary tuple slots allocated by pgoutput.

commit   : 76fda614027436ca030e1fac4fe33484e995970f    
  
author   : Amit Kapila <[email protected]>    
date     : Thu, 27 Jun 2024 10:43:52 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Thu, 27 Jun 2024 10:43:52 +0530    

Click here for diff

In pgoutput, when converting the child table's tuple format to match the  
parent table's, we temporarily create a new slot to store the converted  
tuple. However, we missed to drop such temporary slots, leading to  
resource leakage.  
  
Reported-by: Bowen Shi  
Author: Hou Zhijie  
Reviewed-by: Amit Kapila  
Backpatch-through: 15  
Discussion: https://postgr.es/m/CAM_vCudv8dc3sjWiPkXx5F2b27UV7_YRKRbtSCcE-pv=cVACGA@mail.gmail.com  

M src/backend/replication/pgoutput/pgoutput.c

Fix overflow with pgstats DSA reference count

commit   : eb144dfcafdd7fcce817c75db0c353667b819e11    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 27 Jun 2024 09:44:55 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 27 Jun 2024 09:44:55 +0900    

Click here for diff

When pgstats is initialized for a backend, it uses dsa_attach_in_place()  
without a "segment" provided.  Hence, no callback is registered to  
automatically release the DSA attached once a backend exits.  Not doing  
any cleanup causes the reference count of the pgstats DSA to  
continuously increment, at some point overflowing it (the more the  
number of connections, the faster it is to reach this state).  Once the  
reference count overflows and then gets back to 0, new backends are not  
able to attach to the pgstats DSA, failing startup.  
  
This issue is resolved by adding in the pgstats shutdown hook a call to  
dsa_release_in_place(), ensuring that the DSA attached at backend  
startup is correctly released, keeping the reference count at bay.  
  
The author of this patch has been able to see this issue on a server  
with a long uptime and a high connection turnover.  
  
Issue introduced by 5891c7a8ed8f, so backpatch down to 15.  
  
Author: Anthonin Bonnefoy  
Discussion: https://postgr.es/m/CAO6_XqqJbJBL=M7Ym13TcB4Xnq58vRa2jcC+gwEPBgbAda6B1Q@mail.gmail.com  
Backpatch-through: 15  

M src/backend/utils/activity/pgstat_shmem.c

Fix bugs in MultiXact truncation

commit   : c809a2b2d68c8df0d70d525d54c488d364078c8c    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Fri, 21 Jun 2024 18:31:15 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Fri, 21 Jun 2024 18:31:15 +0300    

Click here for diff

1. TruncateMultiXact() performs the SLRU truncations in a critical  
section. Deleting the SLRU segments calls ForwardSyncRequest(), which  
will try to compact the request queue if it's full  
(CompactCheckpointerRequestQueue()). That in turn allocates memory,  
which is not allowed in a critical section. Backtrace:  
  
    TRAP: failed Assert("CritSectionCount == 0 || (context)->allowInCritSection"), File: "../src/backend/utils/mmgr/mcxt.c", Line: 1353, PID: 920981  
    postgres: autovacuum worker template0(ExceptionalCondition+0x6e)[0x560a501e866e]  
    postgres: autovacuum worker template0(+0x5dce3d)[0x560a50217e3d]  
    postgres: autovacuum worker template0(ForwardSyncRequest+0x8e)[0x560a4ffec95e]  
    postgres: autovacuum worker template0(RegisterSyncRequest+0x2b)[0x560a50091eeb]  
    postgres: autovacuum worker template0(+0x187b0a)[0x560a4fdc2b0a]  
    postgres: autovacuum worker template0(SlruDeleteSegment+0x101)[0x560a4fdc2ab1]  
    postgres: autovacuum worker template0(TruncateMultiXact+0x2fb)[0x560a4fdbde1b]  
    postgres: autovacuum worker template0(vac_update_datfrozenxid+0x4b3)[0x560a4febd2f3]  
    postgres: autovacuum worker template0(+0x3adf66)[0x560a4ffe8f66]  
    postgres: autovacuum worker template0(AutoVacWorkerMain+0x3ed)[0x560a4ffe7c2d]  
    postgres: autovacuum worker template0(+0x3b1ead)[0x560a4ffecead]  
    postgres: autovacuum worker template0(+0x3b620e)[0x560a4fff120e]  
    postgres: autovacuum worker template0(+0x3b3fbb)[0x560a4ffeefbb]  
    postgres: autovacuum worker template0(+0x2f724e)[0x560a4ff3224e]  
    /lib/x86_64-linux-gnu/libc.so.6(+0x27c8a)[0x7f62cc642c8a]  
    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f62cc642d45]  
    postgres: autovacuum worker template0(_start+0x21)[0x560a4fd16f31]  
  
To fix, bail out in CompactCheckpointerRequestQueue() without doing  
anything, if it's called in a critical section. That covers the above  
call path, as well as any other similar cases where  
RegisterSyncRequest might be called in a critical section.  
  
2. After fixing that, another problem became apparent: Autovacuum  
process doing that truncation can deadlock with the checkpointer  
process. TruncateMultiXact() sets "MyProc->delayChkptFlags |=  
DELAY_CHKPT_START". If the sync request queue is full and cannot be  
compacted, the process will repeatedly sleep and retry, until there is  
room in the queue. However, if the checkpointer is trying to start a  
checkpoint at the same time, and is waiting for the DELAY_CHKPT_START  
processes to finish, the queue will never shrink.  
  
More concretely, the autovacuum process is stuck here:  
  
    #0  0x00007fc934926dc3 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6  
    #1  0x000056220b24348b in WaitEventSetWaitBlock (set=0x56220c2e4b50, occurred_events=0x7ffe7856d040, nevents=1, cur_timeout=<optimized out>) at ../src/backend/storage/ipc/latch.c:1570  
    #2  WaitEventSetWait (set=0x56220c2e4b50, timeout=timeout@entry=10, occurred_events=<optimized out>, occurred_events@entry=0x7ffe7856d040, nevents=nevents@entry=1,  
        wait_event_info=wait_event_info@entry=150994949) at ../src/backend/storage/ipc/latch.c:1516  
    #3  0x000056220b243224 in WaitLatch (latch=<optimized out>, latch@entry=0x0, wakeEvents=wakeEvents@entry=40, timeout=timeout@entry=10, wait_event_info=wait_event_info@entry=150994949)  
        at ../src/backend/storage/ipc/latch.c:538  
    #4  0x000056220b26cf46 in RegisterSyncRequest (ftag=ftag@entry=0x7ffe7856d0a0, type=type@entry=SYNC_FORGET_REQUEST, retryOnError=true) at ../src/backend/storage/sync/sync.c:614  
    #5  0x000056220af9db0a in SlruInternalDeleteSegment (ctl=ctl@entry=0x56220b7beb60 <MultiXactMemberCtlData>, segno=segno@entry=11350) at ../src/backend/access/transam/slru.c:1495  
    #6  0x000056220af9dab1 in SlruDeleteSegment (ctl=ctl@entry=0x56220b7beb60 <MultiXactMemberCtlData>, segno=segno@entry=11350) at ../src/backend/access/transam/slru.c:1566  
    #7  0x000056220af98e1b in PerformMembersTruncation (oldestOffset=<optimized out>, newOldestOffset=<optimized out>) at ../src/backend/access/transam/multixact.c:3006  
    #8  TruncateMultiXact (newOldestMulti=newOldestMulti@entry=3221225472, newOldestMultiDB=newOldestMultiDB@entry=4) at ../src/backend/access/transam/multixact.c:3201  
    #9  0x000056220b098303 in vac_truncate_clog (frozenXID=749, minMulti=<optimized out>, lastSaneFrozenXid=749, lastSaneMinMulti=3221225472) at ../src/backend/commands/vacuum.c:1917  
    #10 vac_update_datfrozenxid () at ../src/backend/commands/vacuum.c:1760  
    #11 0x000056220b1c3f76 in do_autovacuum () at ../src/backend/postmaster/autovacuum.c:2550  
    #12 0x000056220b1c2c3d in AutoVacWorkerMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at ../src/backend/postmaster/autovacuum.c:1569  
  
and the checkpointer is stuck here:  
  
    #0  0x00007fc9348ebf93 in clock_nanosleep () from /lib/x86_64-linux-gnu/libc.so.6  
    #1  0x00007fc9348fe353 in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6  
    #2  0x000056220b40ecb4 in pg_usleep (microsec=microsec@entry=10000) at ../src/port/pgsleep.c:50  
    #3  0x000056220afb43c3 in CreateCheckPoint (flags=flags@entry=108) at ../src/backend/access/transam/xlog.c:7098  
    #4  0x000056220b1c6e86 in CheckpointerMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at ../src/backend/postmaster/checkpointer.c:464  
  
To fix, add AbsorbSyncRequests() to the loops where the checkpointer  
waits for DELAY_CHKPT_START or DELAY_CHKPT_COMPLETE operations to  
finish.  
  
Backpatch to v14. Before that, SLRU deletion didn't call  
RegisterSyncRequest, which avoided this failure. I'm not sure if there  
are other similar scenarios on older versions, but we haven't had  
any such reports.  
  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M src/backend/access/transam/xlog.c
M src/backend/postmaster/checkpointer.c

Remove redundant perl version checks

commit   : f853e23bf8276d5509ebb96395e6dc95fac0ec0a    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 26 Jun 2024 07:01:47 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 26 Jun 2024 07:01:47 -0400    

Click here for diff

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl  
scripts, including in plperl's plc_perlboot.pl. Because it made other  
changes it wasn't backpatched. However, now this is causing a failure on  
back branches when built with bleeding edge perl. Therefore, backpatch  
just that part of it which removed those uses, from 15 all the way down  
to 9.2, which is the earliest version currently built in the buildfarm.  
  
per report from Alexander Lakhin  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plperl/plc_perlboot.pl
M src/tools/pgindent/pgindent

Fix partition pruning setup during DETACH CONCURRENTLY

commit   : fb0fb0740fb6b7f51db6ad7cd40d0459ae1a8b98    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 24 Jun 2024 15:56:32 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 24 Jun 2024 15:56:32 +0200    

Click here for diff

When detaching partition in concurrent mode, it's possible for partition  
descriptors to not match the set that was recently seen when the plan  
was made, causing an assertion failure or (in production builds) failure  
to construct a working plan.  The case that was reported involves  
prepared statements, but I think it may be possible to hit this bug  
without that too.  
  
The problem is that CreatePartitionPruneState is constructing a  
PartitionPruneState under the assumption that new partitions can be  
added, but never removed, but it turns out that this isn't true: a  
prepared statement gets replanned when the DETACH CONCURRENTLY session  
sends out its invalidation message, but if the invalidation message  
arrives after ExecInitAppend started, we would build a partition  
descriptor without the partition, and then CreatePartitionPruneState  
would refuse to work with it.  
  
CreatePartitionPruneState already contains code to deal with the new  
descriptor having more partitions than before (and behaving for the  
extra partitions as if they had been pruned), but doesn't have code to  
deal with less partitions than before, and it is naïve about the case  
where the number of partitions is the same.  We could simply add that a  
new stanza for less partitions than before, and in simple testing it  
works to do that; but it's possible to press the test scripts even  
further and hit the case where one partition is added and a partition is  
removed quickly enough that we see the same number of partitions, but  
they don't actually match, causing hangs during execution.  
  
To cope with both these problems, we now memcmp() the arrays of  
partition OIDs, and do a more elaborate mapping (relying on the fact  
that both OID arrays are in partition-bounds order) if they're not  
identical.  
  
Backpatch to 14, where DETACH CONCURRENTLY appeared.  
  
Reported-by: yajun Hu <[email protected]>  
Reviewed-by: Tender Wang <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/executor/execPartition.c

Doc: Generated columns are skipped for logical replication.

commit   : a99b2ccd567937629cf4dc01391c59af66e3384d    
  
author   : Amit Kapila <[email protected]>    
date     : Fri, 21 Jun 2024 09:41:13 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Fri, 21 Jun 2024 09:41:13 +0530    

Click here for diff

Add a note in docs that generated columns are skipped for logical  
replication.  
  
Author: Peter Smith  
Reviewed-by: Peter Eisentraut  
Backpatch-through: 12  
Discussion: https://postgr.es/m/CAHut+PuXb1GLQztQkoWzYjSwkAZZ0dgCJaAHyJtZF3kmtcL=kA@mail.gmail.com  

M doc/src/sgml/ddl.sgml

Don't throw an error if a queued AFTER trigger no longer exists.

commit   : 1424c7abc44fcb61f6a2a63f7d163670ba3250be    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 20 Jun 2024 14:21:36 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 20 Jun 2024 14:21:36 -0400    

Click here for diff

afterTriggerInvokeEvents and AfterTriggerExecute have always  
treated it as an error if the trigger OID mentioned in a queued  
after-trigger event can't be found.  However, that fails to  
account for the edge case where the trigger's been dropped in  
the current transaction since queueing the event.  There seems  
no very good reason to disallow that case, so instead silently  
do nothing if the trigger OID can't be found.  
  
This does give up a little bit of bug-detection ability, but I don't  
recall that these error messages have ever actually revealed a bug,  
so it seems mostly theoretical.  Alternatives such as marking  
pending events DONE at the time of dropping a trigger would be  
complicated and perhaps introduce bugs of their own.  
  
Per bug #18517 from Alexander Lakhin.  Back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql

Fix possible Assert failure in cost_memoize_rescan

commit   : 27c6242a0484d58852c9e3c390ce643d2e41bd4b    
  
author   : David Rowley <[email protected]>    
date     : Wed, 19 Jun 2024 10:21:26 +1200    
  
committer: David Rowley <[email protected]>    
date     : Wed, 19 Jun 2024 10:21:26 +1200    

Click here for diff

In cost_memoize_rescan(), when calculating the hit_ratio using the calls  
and ndistinct estimations, if the value that was set in  
MemoizePath.calls had not been processed through clamp_row_est(), then it  
was possible that it was set to some non-integer value which could result  
in ndistinct being 1 higher than calls due to estimate_num_groups()  
performing clamp_row_est() on its input_rows.  This could result in  
hit_ratio values slightly below 0.0, which would cause an Assert failure.  
  
The value of MemoizePath.calls comes from the final parameter in the  
create_memoize_path() function, of which we only have one true caller of.  
That caller passes outer_path->rows.  All the core code I looked at  
always seems to call clamp_row_est() on the Path.rows, so there might  
have been no issues with any core Paths causing troubles here.  The bug  
report was about a CustomPath with a non-clamped row estimated.  
  
The misbehavior as a result of this seems to be mostly limited to the  
Assert() failing.  Aside from that, it seems the Memoize costs would  
just come out slightly higher than they should have, which is likely  
fairly harmless.  
  
Reported-by: Kohei KaiGai <[email protected]>  
Discussion: https://postgr.es/m/CAOP8fzZnTU+N64UYJYogb1hN-5hFP+PwTb3m_cnGAD7EsQwrKw@mail.gmail.com  
Reviewed-by: Richard Guo  
Backpatch-through: 14, where Memoize was introduced  

M src/backend/optimizer/util/pathnode.c

Fix insertion of SP-GiST REDIRECT tuples during REINDEX CONCURRENTLY.

commit   : f55083319323a752f63306c89c1884f0fcb6a85e    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 17 Jun 2024 14:30:59 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 17 Jun 2024 14:30:59 -0400    

Click here for diff

Reconstruction of an SP-GiST index by REINDEX CONCURRENTLY may  
insert some REDIRECT tuples.  This will typically happen in  
a transaction that lacks an XID, which leads either to assertion  
failure in spgFormDeadTuple or to insertion of a REDIRECT tuple  
with zero xid.  The latter's not good either, since eventually  
VACUUM will apply GlobalVisTestIsRemovableXid() to the zero xid,  
resulting in either an assertion failure or a garbage answer.  
  
In practice, since REINDEX CONCURRENTLY locks out index scans  
till it's done, it doesn't matter whether it inserts REDIRECTs  
or PLACEHOLDERs; and likewise it doesn't matter how soon VACUUM  
reduces such a REDIRECT to a PLACEHOLDER.  So in non-assert builds  
there's no observable problem here, other than perhaps a little  
index bloat.  But it's not behaving as intended.  
  
To fix, remove the failing Assert in spgFormDeadTuple, acknowledging  
that we might sometimes insert a zero XID; and guard VACUUM's  
GlobalVisTestIsRemovableXid() call with a test for valid XID,  
ensuring that we'll reduce such a REDIRECT the first time VACUUM  
sees it.  (Versions before v14 use TransactionIdPrecedes here,  
which won't fail on zero xid, so they really have no bug at all  
in non-assert builds.)  
  
Another solution could be to not create REDIRECTs at all during  
REINDEX CONCURRENTLY, making the relevant code paths treat that  
case like index build (which likewise knows that no concurrent  
index scans can be happening).  That would allow restoring the  
Assert in spgFormDeadTuple, but we'd still need the VACUUM change  
because redirection tuples with zero xid may be out there already.  
But there doesn't seem to be a nice way for spginsert() to tell that  
it's being called in REINDEX CONCURRENTLY without some API changes,  
so we'll leave that as a possible future improvement.  
  
In HEAD, also rename the SpGistState.myXid field to redirectXid,  
which seems less misleading (since it might not in fact be our  
transaction's XID) and is certainly less uninformatively generic.  
  
Per bug #18499 from Alexander Lakhin.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/spgist/spgutils.c
M src/backend/access/spgist/spgvacuum.c
M src/include/access/spgist_private.h

Clean out column-level pg_init_privs entries when dropping tables.

commit   : 1f1eedd3fa696f41ba426ebcd5de4fe08b5005cf    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 14 Jun 2024 16:20:35 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 14 Jun 2024 16:20:35 -0400    

Click here for diff

DeleteInitPrivs did not get the memo about how, when dropping a  
whole object (with subid == 0), you should drop entries relating  
to its sub-objects too.  This is visible in the test_pg_dump test  
case if one drops the extension at the end: the entry for  
	GRANT SELECT(col1) ON regress_pg_dump_table TO public;  
was still present in pg_init_privs afterwards, although it was  
pointing to a dangling table OID.  
  
Noted while fooling with a fix for REASSIGN OWNED for pg_init_privs  
entries.  This bug is aboriginal in the pg_init_privs feature  
though, and there seems no reason not to back-patch the fix.  

M src/backend/catalog/dependency.c
M src/test/modules/test_pg_dump/expected/test_pg_dump.out
M src/test/modules/test_pg_dump/sql/test_pg_dump.sql

Fix parsing of ignored operators in websearch_to_tsquery().

commit   : df95c1ec0e988f4eb7d67fd7541501c894dfe082    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 13 Jun 2024 20:34:43 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 13 Jun 2024 20:34:43 -0400    

Click here for diff

The manual says clearly that punctuation in the input of  
websearch_to_tsquery() is ignored, except for the special cases  
of dashes and quotes.  However, this failed for cases like  
"(foo bar) or something", or in general an ISOPERATOR character  
in front of the "or".  We'd switch back to WAITOPERAND state,  
then ignore the operator character while remaining in that state,  
and then reach the "or" in WAITOPERAND state which (intentionally)  
makes us treat it as data.  
  
The fix is simple enough: if we see an ISOPERATOR character while in  
WAITOPERATOR state, we have to skip it while staying in that state.  
(We don't need to worry about other punctuation characters: those will  
be consumed as though they were words, but then rejected by lexizing.)  
  
In v14 and up (since commit eb086056f) we can simplify the code a bit  
more too, because there is no longer a reason for the WAITOPERAND  
state to distinguish between quoted and unquoted operands.  
  
Per bug #18479 from Manos Emmanouilidis.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/adt/tsquery.c
M src/test/regress/expected/tsearch.out
M src/test/regress/sql/tsearch.sql

doc: Fix description WAL writer in glossary

commit   : 37c5e5f4d45c66b5b43d976d50fbd04488f2c8ef    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 14 Jun 2024 09:26:39 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 14 Jun 2024 09:26:39 +0900    

Click here for diff

The WAL writer is an auxiliary process, but its description in the  
glossary did not match that.  
  
This is inexact since d3014fff4cd4.  
  
Author: Masahiro Ikeda  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 15  

M doc/src/sgml/glossary.sgml

When replanning a plpgsql "simple expression", check it's still simple.

commit   : bf552b1b2d8bc777e7831b7d2dbbd68bdd824c67    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 13 Jun 2024 13:37:46 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 13 Jun 2024 13:37:46 -0400    

Click here for diff

The previous coding here assumed that we didn't need to recheck any  
of the querytree tests made in exec_simple_check_plan().  I think  
we supposed that those properties were fully determined by the  
syntax of the source text and hence couldn't change.  That is true  
for most of them, but at least hasTargetSRFs and hasAggs can change  
by dint of forcibly dropping an originally-referenced function and  
recreating it with new properties.  That leads to "unexpected plan  
node type" or similar failures.  
  
These tests are pretty cheap compared to the cost of replanning, so  
rather than sweat over exactly which properties need to be rechecked,  
let's just recheck them all.  Hence, factor out those tests into a new  
function exec_is_simple_query(), and rearrange callers as needed.  
  
A second problem in the same area was that if we failed during  
replanning or during exec_save_simple_expr(), we'd potentially  
leave behind now-dangling pointers to the old simple expression,  
potentially resulting in crashes later.  To fix, clear those pointers  
before replanning.  
  
The v12 code looks quite different in this area but still has the  
bug about needing to recheck query simplicity.  I chose to back-patch  
all of the plpgsql_simple.sql test script, which formerly didn't exist  
in this branch.  
  
Per bug #18497 from Nikita Kalinin.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plpgsql/src/expected/plpgsql_simple.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_simple.sql

Clamp result of MultiXactMemberFreezeThreshold

commit   : 7768ac1c17f0ec7883825e8768ab480caa7842ec    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 13 Jun 2024 19:01:30 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 13 Jun 2024 19:01:30 +0300    

Click here for diff

The purpose of the function is to reduce the effective  
autovacuum_multixact_freeze_max_age if the multixact members SLRU is  
approaching wraparound, to make multixid freezing more aggressive.  
The returned value should therefore never be greater than plain  
autovacuum_multixact_freeze_max_age.  
  
Reviewed-by: Robert Haas  
Discussion: https://www.postgresql.org/message-id/[email protected]  
Backpatch-through: 12, all supported versions  

M src/backend/access/transam/multixact.c

Skip some permissions checks on Cygwin

commit   : c38518fa978075d08d64a34e1b4504da13a81e02    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 13 Jun 2024 07:38:48 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 13 Jun 2024 07:38:48 -0400    

Click here for diff

These are checks that are already skipped on other Windows systems.  
  
Backpatch to all live branches, as appropriate.  

M src/bin/initdb/t/001_initdb.pl
M src/bin/pg_basebackup/t/010_pg_basebackup.pl
M src/bin/pg_ctl/t/001_start_stop.pl
M src/bin/pg_rewind/t/002_databases.pl
M src/bin/pg_verifybackup/t/003_corruption.pl

Fix infer_arbiter_indexes() to not assume resultRelation is 1.

commit   : 1d0399b54068586db003691c218c3d68bee055c8    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 11 Jun 2024 17:57:46 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 11 Jun 2024 17:57:46 -0400    

Click here for diff

infer_arbiter_indexes failed to renumber varnos in index expressions  
or predicates that it got from the catalogs.  This escaped detection  
up to now because the stored varnos in such trees will be 1, and an  
INSERT's result relation is usually the first rangetable entry,  
so that that was fine.  However, in cases such as inserting through  
an updatable view, it's not fine, leading to failure to match the  
expressions to the query with ensuing "there is no unique or exclusion  
constraint matching the ON CONFLICT specification" errors.  
  
Fix by copy-and-paste from get_relation_info().  
  
Per bug #18502 from Michael Wang.  Back-patch to all supported  
versions.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/optimizer/util/plancat.c
M src/test/regress/expected/insert_conflict.out
M src/test/regress/sql/insert_conflict.sql

Fix creation of partition descriptor during concurrent detach

commit   : 03c8cdbb7e75669322d3084dfc918b721c77f076    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 11 Jun 2024 11:38:45 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 11 Jun 2024 11:38:45 +0200    

Click here for diff

When a partition is being detached in concurrent mode, it is possible  
for find_inheritance_children_extended() to return that partition in the  
list, and immediately after that receive an invalidation message that  
sets its relpartbound to NULL just before we read it.  (This can happen  
because table_open() reads invalidation messages.)  Currently we raise  
an error  
  ERROR:  missing relpartbound for relation %u  
about the situation, but that's bogus because the table is no longer a  
partition, so we shouldn't be complaining about it.  A better reaction  
is to retry the find_inheritance_children_extended call to get a new  
list, which will no longer have the partition being detached.  
  
Noticed while investigating bug #18377.  
  
Backpatch to 14, where DETACH CONCURRENTLY appeared.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/partitioning/partdesc.c

Doc: Fix ambuiguity in column lists.

commit   : a8d747771fe59cd3cf7bc16029a3e4d01407a864    
  
author   : Amit Kapila <[email protected]>    
date     : Tue, 11 Jun 2024 09:15:14 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Tue, 11 Jun 2024 09:15:14 +0530    

Click here for diff

The behavior for columns added later to the table for publications with no  
specified column lists was not clear.  
  
Reported-by: Koen De Groote  
Author: Peter Smith  
Reviewed-by: Vignesh C, Laurenz Albe  
Backpatch-through: 15  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/logical-replication.sgml

Tighten test_predtest's input checks, and improve error messages.

commit   : 2eba27571edcce56ac04838792e79cb23fe179d4    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 7 Jun 2024 16:45:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 7 Jun 2024 16:45:56 -0400    

Click here for diff

test_predtest() neglected to consider the possibility that  
SPI_plan_get_cached_plan would return NULL.  This led to a core  
dump if the input (incorrectly) contains more than one SQL  
command.  
  
While here, let's expend more than zero effort on the error  
message for this case and nearby ones.  
  
Per (half of) bug #18483 from Alexander Kozhemyakin.  
Back-patch to all supported branches, not because this is  
very significant (it's merely test scaffolding) but to make  
our world a bit safer for fuzz testing.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/test/modules/test_predtest/test_predtest.c

Reject modifying a temp table of another session with ALTER TABLE.

commit   : 3c71cb497b0233f8b1e99c0110ca4111900ee08d    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 7 Jun 2024 14:50:09 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 7 Jun 2024 14:50:09 -0400    

Click here for diff

Normally this case isn't even reachable by non-superusers, since  
permissions checks prevent naming such a table.  However, it is  
possible to make it happen by altering a parent table whose child  
is another session's temp table.  
  
We definitely can't support any such ALTER that requires modifying  
the contents of such a table, since we lack access to the other  
session's temporary-buffer pool.  But there seems no good reason  
to allow it even if it'd only require changing catalog contents.  
One reason not to allow it is that we'd rather not expose the  
implementation-dependent behavior of whether a specific ALTER  
requires touching the table contents.  Another is that there may  
be (in future, even if not today) optimizations that assume that  
a session's own temp tables won't be modified by other sessions.  
  
Hence, add a RELATION_IS_OTHER_TEMP() check to all the places  
where ALTER TABLE currently does CheckTableNotInUse().  (I looked  
through all other callers of CheckTableNotInUse(), and they seem  
OK already.)  
  
Per bug #18492 from Alexander Lakhin.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/tablecmds.c

Fix behavior of stable functions called from a CALL's argument list.

commit   : a160e92779afb1146474585a943454eab8432ea0    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 7 Jun 2024 13:27:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 7 Jun 2024 13:27:26 -0400    

Click here for diff

If the CALL is within an atomic context (e.g. there's an outer  
transaction block), _SPI_execute_plan should acquire a fresh snapshot  
to execute any such functions with.  We failed to do that and instead  
passed them the Portal snapshot, which had been acquired at the start  
of the current SQL command.  This'd lead to seeing stale values of  
rows modified since the start of the command.  
  
This is arguably a bug in 84f5c2908: I failed to see that "are we in  
non-atomic mode" needs to be defined the same way as it is further  
down in _SPI_execute_plan, i.e. check !_SPI_current->atomic not just  
options->allow_nonatomic.  Alternatively the blame could be laid on  
plpgsql, which is unconditionally passing allow_nonatomic = true  
for CALL/DO even when it knows it's in an atomic context.  However,  
fixing it in spi.c seems like a better idea since that will also fix  
the problem for any extensions that may have copied plpgsql's coding  
pattern.  
  
While here, update an obsolete comment about _SPI_execute_plan's  
snapshot management.  
  
Per report from Victor Yegorov.  Back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/CAGnEboiRe+fG2QxuBO2390F7P8e2MQ6UyBjZSL_w1Cej+E4=Vw@mail.gmail.com  

M doc/src/sgml/spi.sgml
M src/backend/executor/spi.c
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/sql/plpgsql_call.sql

Add more debugging information when dropping twice pgstats entry

commit   : 2de059de2c2dbc9324deb304b625d1619af03a6c    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 7 Jun 2024 18:46:34 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 7 Jun 2024 18:46:34 +0900    

Click here for diff

Floris Van Nee has reported a bug in the pgstats facility where a stats  
entry already dropped would get again dropped.  This case should not  
happen, still the error generated did not offer any details about the  
stats entry getting dropped.  
  
This commit improves the error message generated to inform about the  
stats entry kind, database OID, object OID and refcount, which should  
help to debug more the problem reported.  Bertrand Drouvot has been  
independently able to reach this error path while writing a new feature,  
and more details about the failure would have been helpful for  
debugging.  
  
Author: Andres Freund, Bertrand Drouvot  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/ZkM30paAD8Cr/[email protected]  
Backpatch-through: 15  

M src/backend/utils/activity/pgstat_shmem.c

postgres_fdw: Refuse to send FETCH FIRST WITH TIES to remote servers.

commit   : b33c141cc52b5dfdf06dea1c36874a5842bada8f    
  
author   : Etsuro Fujita <[email protected]>    
date     : Fri, 7 Jun 2024 17:45:04 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Fri, 7 Jun 2024 17:45:04 +0900    

Click here for diff

Previously, when considering LIMIT pushdown, postgres_fdw failed to  
check whether the query has this clause, which led to pushing false  
LIMIT clauses, causing incorrect results.  
  
This clause has been supported since v13, so we need to do a  
remote-version check before deciding that it will be safe to push such a  
clause, but we do not currently have a way to do the check (without  
accessing the remote server); disable pushing such a clause for now.  
  
Oversight in commit 357889eb1.  Back-patch to v13, where that commit  
added the support.  
  
Per bug #18467 from Onder Kalaci.  
  
Patch by Japin Li, per a suggestion from Tom Lane, with some changes to  
the comments by me.  Review by Onder Kalaci, Alvaro Herrera, and me.  
  
Discussion: https://postgr.es/m/18467-7bb89084ff03a08d%40postgresql.org  

M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql

doc: Fix copy-and-paste mistake

commit   : 00071ef04ce08b8ac4168085c9d0fc3f685c825c    
  
author   : Peter Eisentraut <[email protected]>    
date     : Fri, 7 Jun 2024 08:02:15 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Fri, 7 Jun 2024 08:02:15 +0200    

Click here for diff

The wording from the "columns" view was copied to the "attributes"  
view without the required adjustments.  

M doc/src/sgml/information_schema.sgml

Fix failure with SQL-procedure polymorphic output arguments in v12.

commit   : 5fe43d41db599bbe27e8cc75970c093e9c74fa37    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 6 Jun 2024 15:16:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 6 Jun 2024 15:16:56 -0400    

Click here for diff

Before the v13-era commit 913bbd88d, check_sql_fn_retval fails to  
resolve polymorphic output types and then just throws up its hands and  
assumes the check will be made at runtime.  I think that's true for  
ordinary functions returning RECORD, but it doesn't happen in CALL,  
potentially resulting in crashes if the actual output of the SQL  
procedure's SELECT doesn't match the type inferred from polymorphism.  
With a little bit of rearrangement, we can use get_call_result_type  
instead of get_func_result_type and thereby infer the correct types.  
  
I'm still unwilling to back-patch all of 913bbd88d, so if the types  
don't match you'll get an error rather than perhaps silently inserting  
a cast as v13 and later can.  That's consistent with prior behavior  
though, so it seems fine.  
  
Prior to 70ffb27b2, you'd typically get other errors due to other  
shortcomings of CALL's management of polymorphism.  Nonetheless,  
this is an independent bug.  
  
Although there is no bug in v13 and up, it seems prudent to add  
the test case for this to the newer branches too.  It's clearly  
an under-tested area.  
  
Per report from Andrew Bille.  
  
Discussion: https://postgr.es/m/CAJnzarw9EeWHAQRm76dXd=7j+rgw6ERqC=nCay8jeFqTwKwhqQ@mail.gmail.com  

M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_procedure.sql

Prevent inconsistent use of stats entry for replication slots

commit   : bfc44da2478954330233a8df8a146ee920400947    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 6 Jun 2024 08:48:21 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 6 Jun 2024 08:48:21 +0900    

Click here for diff

Concurrent activity around replication slot creation and drop could  
cause a replication slot to use a stats entry it should not have used  
when created, triggering an assertion failure when retrieving this  
inconsistent entry from the dshash table used by the stats facility.  
  
The issue is that pgstat_drop_replslot() calls pgstat_drop_entry()  
without checking the result.  If pgstat_drop_entry() cannot free the  
entry related to the object dropped, pgstat_request_entry_refs_gc()  
should be called.  AtEOXact_PgStat_DroppedStats() and surrounding  
routines dropping stats entries already do that.  
  
This is documented in pgstat_internal.h, but let's add a comment at the  
top of pgstat_drop_entry() as that can be easy to miss.  
  
Reported-by: Alexander Lakhin  
Author: Floris Van Nee  
Analyzed-by: Andres Freund  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 15  

M src/backend/utils/activity/pgstat_replslot.c
M src/backend/utils/activity/pgstat_shmem.c

Fix documentation for POSIX semaphores.

commit   : bb8425491cb7d5c760913761bf13644a7e86f58f    
  
author   : Nathan Bossart <[email protected]>    
date     : Wed, 5 Jun 2024 15:32:47 -0500    
  
committer: Nathan Bossart <[email protected]>    
date     : Wed, 5 Jun 2024 15:32:47 -0500    

Click here for diff

The documentation for POSIX semaphores is missing a reference to  
max_wal_senders.  This commit fixes that in the same way that  
commit 4ebe51a5fb fixed the same issue in the documentation for  
System V semaphores.  
  
Discussion: https://postgr.es/m/20240517164452.GA1914161%40nathanxps13  
Backpatch-through: 12  

M doc/src/sgml/runtime.sgml

Fix pl/tcl's handling of errors from Tcl_ListObjGetElements().

commit   : 89ef2aedae448cc51cd7bf7c81d56a08d1da7467    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 4 Jun 2024 18:02:13 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 4 Jun 2024 18:02:13 -0400    

Click here for diff

In a procedure or function returning tuple, we use that function to  
parse the Tcl script's result, which is supposed to be a Tcl list.  
If it isn't, you get an error.  Commit 26abb50c4 incautiously  
supposed that we could use throw_tcl_error() to report such an error.  
That doesn't actually work, because low-level functions like  
Tcl_ListObjGetElements() don't fill Tcl's errorInfo variable.  
The result is either a null-pointer-dereference crash or emission  
of misleading context information describing the previous Tcl error.  
  
Back off to just reporting the interpreter's result string, and  
improve throw_tcl_error()'s comment to explain when to use it.  
  
Also, although the similar code in pltcl_trigger_handler() avoided  
this mistake, it was using a fairly confusing wording of the  
error message.  Improve that while we're here.  
  
Per report from A. Kozhemyakin.  Back-patch to all supported  
branches.  
  
Erik Wienhold and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/tcl/expected/pltcl_call.out
M src/pl/tcl/pltcl.c
M src/pl/tcl/sql/pltcl_call.sql

ci: windows: Use the same image for VS and MinGW tasks

commit   : 6b52e2298d8529eab777eaca339da3b04c862f9c    
  
author   : Andres Freund <[email protected]>    
date     : Mon, 3 Jun 2024 19:06:50 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Mon, 3 Jun 2024 19:06:50 -0700    

Click here for diff

The VS and MinGW Windows images have been merged, to reduce the space needed  
for images. Before 98811323c8e the split helped boot performance, but now that  
we are using VMs that doesn't appear to be the case anymore.  
  
Author: Nazir Bilal Yavuz <[email protected]>  
Discussion: https://postgr.es/m/CAN55FZ2kWYjPd7uUC5QswrB3tfVJDiURqC%2BMGM6a3oeev%3DVgOA%40mail.gmail.com  
Backpatch: 15-, where CI was added  

M .cirrus.tasks.yml

Fix documentation for System V semaphores.

commit   : f1884f57570ca73005d7a5b55eddd7af5c7067f7    
  
author   : Nathan Bossart <[email protected]>    
date     : Mon, 3 Jun 2024 12:10:43 -0500    
  
committer: Nathan Bossart <[email protected]>    
date     : Mon, 3 Jun 2024 12:10:43 -0500    

Click here for diff

The formulas for SEMMNI and SEMMNS do not include the archiver  
process, which was converted to an auxiliary process in v14, and  
the WAL summarizer process, which was introduced in v17.  This  
commit corrects these formulas and adds a missing reference to  
max_wal_senders nearby.  Since this section of the documentation  
tends to be incorrect quite often, we should likely give up on  
documenting the exact formulas in favor of something less fragile,  
but that is left as a future exercise.  
  
Reported-by: Sami Imseih  
Reviewed-by: Sami Imseih  
Discussion: https://postgr.es/m/20240517164452.GA1914161%40nathanxps13  
Backpatch-through: 12  

M doc/src/sgml/runtime.sgml

Improve stability of subscription/029_on_error.pl

commit   : 8e16f81f3db4eb03ac2195e9e5527852047c14a0    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 24 May 2024 11:21:31 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 24 May 2024 11:21:31 +0900    

Click here for diff

This test was failing when using wal_debug=on and -DWAL_DEBUG because of  
additional log entries that made the test grab an LSN not mapping with  
the error expected in the test.  
  
Previously the test would look for the first matching line to get the  
LSN to skip up to.  This is improved by having the test scan the logs  
with a regexp that checks for the expected ERROR string, ensuring that  
the wanted LSN comes from the correct context.  
  
Backpatch down to 15 where this test has been introduced.  
  
Author: Ian Ilyasov  
Discussion: https://postgr.es/m/GV1P251MB100415F17E6B2FDD7188777ECDE32@GV1P251MB1004.EURP251.PROD.OUTLOOK.COM  
Backpatch-through: 15  

M src/test/subscription/t/029_on_error.pl

Remove race conditions between ECPGdebug() and ecpg_log().

commit   : e892e72b3c8ce9221003cddca6b8a88e2c951cad    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 23 May 2024 15:52:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 23 May 2024 15:52:06 -0400    

Click here for diff

Coverity complains that ECPGdebug is accessing debugstream without  
holding debug_mutex, which is a fair complaint: we should take  
debug_mutex while changing the settings ecpg_log looks at.  
  
In some branches it also complains about unlocked use of simple_debug.  
I think it's intentional and safe to have a quick unlocked check of  
simple_debug at the start of ecpg_log, since that early exit will  
always be taken in non-debug cases.  But we should recheck  
simple_debug after acquiring the mutex.  In the worst case, calling  
ECPGdebug concurrently with ecpg_log in another thread could result  
in a null-pointer dereference due to debugstream transiently being  
NULL while simple_debug isn't 0.  
  
This is largely hypothetical, since it's unlikely anybody uses  
ECPGdebug() at all in the field, and our own regression tests  
don't seem to be hitting the theoretical race conditions either.  
Still, if we're going to the trouble of having mutexes here, we ought  
to be using them in a way that's actually safe not just almost safe.  
Hence, back-patch to all supported branches.  

M src/interfaces/ecpg/ecpglib/misc.c

doc: Fix column_name parameter in ALTER MATERIALIZED VIEW

commit   : c0df15ac7e9ef3870da50fde49c658290f7a0da6    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 23 May 2024 13:03:16 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 23 May 2024 13:03:16 +0900    

Click here for diff

Parameter column_name must be an existing column because ALTER  
MATERIALIZED VIEW cannot add new columns.  The old description was  
likely copied from ALTER TABLE.  
  
Author: Erik Wienhold  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

M doc/src/sgml/ref/alter_materialized_view.sgml

Fix handling of extended expression statistics in CREATE TABLE LIKE.

commit   : 2f3cfcf7677ff11ca0ca18b67bd8fc1ea6ae6dd5    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 22 May 2024 17:54:17 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 22 May 2024 17:54:17 -0400    

Click here for diff

transformTableLikeClause believed that it could process extended  
statistics immediately because "the representation of CreateStatsStmt  
doesn't depend on column numbers".  That was true when extended stats  
were first introduced, but it was falsified by the addition of  
extended stats on expressions: the parsed expression tree is fed  
forward by the LIKE option, and that will contain Vars.  So if the  
new table doesn't have attnums identical to the old one's (typically  
because there are some dropped columns in the old one), that doesn't  
work.  The CREATE goes through, but it emits invalid statistics  
objects that will cause problems later.  
  
Fortunately, we already have logic that can adapt expression trees  
to the possibly-new column numbering.  To use it, we have to delay  
processing of CREATE_TABLE_LIKE_STATISTICS into expandTableLikeClause,  
just as for other LIKE options that involve expressions.  
  
Per bug #18468 from Alexander Lakhin.  Back-patch to v14 where  
extended statistics on expressions were added.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql

Account for optimized MinMax aggregates during SS_finalize_plan.

commit   : 4ac385adc5b7ae6474fc8e8fb418131cf2e63b4b    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 18 May 2024 14:31:35 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 18 May 2024 14:31:35 -0400    

Click here for diff

We are capable of optimizing MIN() and MAX() aggregates on indexed  
columns into subqueries that exploit the index, rather than the normal  
thing of scanning the whole table.  When we do this, we replace the  
Aggref node(s) with Params referencing subquery outputs.  Such Params  
really ought to be included in the per-plan-node extParam/allParam  
sets computed by SS_finalize_plan.  However, we've never done so  
up to now because of an ancient implementation choice to perform  
that substitution during set_plan_references, which runs after  
SS_finalize_plan, so that SS_finalize_plan never sees these Params.  
  
The cleanest fix would be to perform a separate tree walk to do  
these substitutions before SS_finalize_plan runs.  That seems  
unattractive, first because a whole-tree mutation pass is expensive,  
and second because we lack infrastructure for visiting expression  
subtrees in a Plan tree, so that we'd need a new function knowing  
as much as SS_finalize_plan knows about that.  I also considered  
swapping the order of SS_finalize_plan and set_plan_references,  
but that fell foul of various assumptions that seem tricky to fix.  
So the approach adopted here is to teach SS_finalize_plan itself  
to check for such Aggrefs.  I refactored things a bit in setrefs.c  
to avoid having three copies of the code that does that.  
  
Back-patch of v17 commits d0d44049d and 779ac2c74.  When d0d44049d  
went in, there was no evidence that it was fixing a reachable bug,  
so I refrained from back-patching.  Now we have such evidence.  
  
Per bug #18465 from Hal Takahara.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/optimizer/plan/setrefs.c
M src/backend/optimizer/plan/subselect.c
M src/include/optimizer/planmain.h
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql

Fix documentation about DROP DATABASE FORCE process termination rights.

commit   : 484b9587370ecb0325bfc30ca435697f9f52acc6    
  
author   : Noah Misch <[email protected]>    
date     : Thu, 16 May 2024 14:11:00 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Thu, 16 May 2024 14:11:00 -0700    

Click here for diff

Specifically, it terminates a background worker even if the caller  
couldn't terminate the background worker with pg_terminate_backend().  
Commit 3a9b18b3095366cd0c4305441d426d04572d88c1 neglected to update  
this.  Back-patch to v13, which introduced DROP DATABASE FORCE.  
  
Reviewed by Amit Kapila.  Reported by Kirill Reshke.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/ref/drop_database.sgml
M src/backend/storage/ipc/procarray.c

Fix query result leak during binary upgrade

commit   : e6fc3b70df906868df36517c2c7c5a38725e1451    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Wed, 15 May 2024 22:48:51 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Wed, 15 May 2024 22:48:51 +0200    

Click here for diff

9a974cbcba00 moved the query in binary_upgrade_set_pg_class_oids to the  
outer level, but left the PQclear and query buffer destruction in the  
is_index conditional.  353708e1fb2d fixed the leak of the query buffer  
but left the PGresult leak. This moves clearing the result to the outer  
level ensuring that it will be called.  
  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: v15  

M src/bin/pg_dump/pg_dump.c

doc: Remove claims that initdb and pg_ctl use libpq environment variables

commit   : a826021e562450abb3b9682087111029f2319330    
  
author   : Peter Eisentraut <[email protected]>    
date     : Wed, 15 May 2024 13:05:30 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Wed, 15 May 2024 13:05:30 +0200    

Click here for diff

Erroneously introduced by 571df93cff8.  
  
Reviewed-by: Daniel Gustafsson <[email protected]>  
Discussion: https://www.postgresql.org/message-id/flat/8458c9c5-18f1-46d7-94c4-1c30e4f44908%40eisentraut.org  

M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_ctl-ref.sgml

Fix handling of polymorphic output arguments for procedures.

commit   : c40e78d239a8f7b8a3b6fd180b6e642e269dec5f    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 14 May 2024 20:19:20 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 14 May 2024 20:19:20 -0400    

Click here for diff

Most of the infrastructure for procedure arguments was already  
okay with polymorphic output arguments, but it turns out that  
CallStmtResultDesc() was a few bricks shy of a load here.  It thought  
all it needed to do was call build_function_result_tupdesc_t, but  
that function specifically disclaims responsibility for resolving  
polymorphic arguments.  Failing to handle that doesn't seem to be  
a problem for CALL in plpgsql, but CALL from plain SQL would get  
errors like "cannot display a value of type anyelement", or even  
crash outright.  
  
In v14 and later we can simply examine the exposed types of the  
CallStmt.outargs nodes to get the right type OIDs.  But it's a lot  
more complicated to fix in v12/v13, because those versions don't  
have CallStmt.outargs, nor do they do expand_function_arguments  
until ExecuteCallStmt runs.  We have to duplicatively run  
expand_function_arguments, and then re-determine which elements  
of the args list are output arguments.  
  
Per bug #18463 from Drew Kimball.  Back-patch to all supported  
versions, since it's busted in all of them.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/functioncmds.c
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/sql/plpgsql_call.sql
M src/test/regress/expected/create_procedure.out
M src/test/regress/sql/create_procedure.sql

Fix pg_sequence_last_value() for unlogged sequences on standbys.

commit   : 857d280c652876e0bbf0e1b5c237574514b50ebc    
  
author   : Nathan Bossart <[email protected]>    
date     : Mon, 13 May 2024 15:53:50 -0500    
  
committer: Nathan Bossart <[email protected]>    
date     : Mon, 13 May 2024 15:53:50 -0500    

Click here for diff

Presently, when this function is called for an unlogged sequence on  
a standby server, it will error out with a message like  
  
	ERROR:  could not open file "base/5/16388": No such file or directory  
  
Since the pg_sequences system view uses pg_sequence_last_value(),  
it can error similarly.  To fix, modify the function to return NULL  
for unlogged sequences on standby servers.  Since this bug is  
present on all versions since v15, this approach is preferable to  
making the ERROR nicer because we need to repair the pg_sequences  
view without modifying its definition on released versions.  For  
consistency, this commit also modifies the function to return NULL  
for other sessions' temporary sequences.  The pg_sequences view  
already appropriately filters out such sequences, so there's no bug  
there, but we might as well offer some defense in case someone  
invokes this function directly.  
  
Unlogged sequences were first introduced in v15, but temporary  
sequences are much older, so while the fix for unlogged sequences  
is only back-patched to v15, the temporary sequence portion is  
back-patched to all supported versions.  
  
We could also remove the privilege check in the pg_sequences view  
definition in v18 if we modify this function to return NULL for  
sequences for which the current user lacks privileges, but that is  
left as a future exercise for when v18 development begins.  
  
Reviewed-by: Tom Lane, Michael Paquier  
Discussion: https://postgr.es/m/20240501005730.GA594666%40nathanxps13  
Backpatch-through: 12  

M doc/src/sgml/system-views.sgml
M src/backend/commands/sequence.c
M src/test/recovery/t/001_stream_rep.pl

Fix recursive RECORD-returning plpython functions.

commit   : 6e29963edd55cfc038304322049d8fe697580f2c    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 9 May 2024 13:16:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 9 May 2024 13:16:21 -0400    

Click here for diff

If we recursed to a new call of the same function, with a different  
coldeflist (AS clause), it would fail because the inner call would  
overwrite the outer call's idea of what to return.  This is vaguely  
like 1d2fe56e4 and c5bec5426, but it's not due to any API decisions:  
it's just that we computed the actual output rowtype at the start of  
the call, and saved it in the per-procedure data structure.  We can  
fix it at basically zero cost by doing the computation at the end  
of each call instead of the start.  
  
It's not clear that there's any real-world use-case for such a  
function, but given that it doesn't cost anything to fix,  
it'd be silly not to.  
  
Per report from Andreas Karlsson.  Back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plpython/expected/plpython_composite.out
M src/pl/plpython/plpy_exec.c
M src/pl/plpython/sql/plpython_composite.sql

Fix overread in JSON parsing errors for incomplete byte sequences

commit   : 8c3f30e675e972e3c2750ea82781f2c4ba77a9f8    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 9 May 2024 12:45:45 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 9 May 2024 12:45:45 +0900    

Click here for diff

json_lex_string() relies on pg_encoding_mblen_bounded() to point to the  
end of a JSON string when generating an error message, and the input it  
uses is not guaranteed to be null-terminated.  
  
It was possible to walk off the end of the input buffer by a few bytes  
when the last bytes consist of an incomplete multi-byte sequence, as  
token_terminator would point to a location defined by  
pg_encoding_mblen_bounded() rather than the end of the input.  This  
commit switches token_terminator so as the error uses data up to the  
end of the JSON input.  
  
More work should be done so as this code could rely on an equivalent of  
report_invalid_encoding() so as incorrect byte sequences can show in  
error messages in a readable form.  This requires work for at least two  
cases in the JSON parsing API: an incomplete token and an invalid escape  
sequence.  A more complete solution may be too invasive for a backpatch,  
so this is left as a future improvement, taking care of the overread  
first.  
  
A test is added on HEAD as test_json_parser makes this issue  
straight-forward to check.  
  
Note that pg_encoding_mblen_bounded() no longer has any callers.  This  
will be removed on HEAD with a separate commit, as this is proving to  
encourage unsafe coding.  
  
Author: Jacob Champion  
Discussion: https://postgr.es/m/CAOYmi+ncM7pwLS3AnKCSmoqqtpjvA8wmCdoBtKA3ZrB2hZG6zA@mail.gmail.com  
Backpatch-through: 13  

M src/common/jsonapi.c

Ensure that "pg_restore -l" reports dependent TOC entries correctly.

commit   : 6a458d93ba5f8d4b12db6a78d31ba2623e9cb594    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 7 May 2024 18:22:52 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 7 May 2024 18:22:52 -0400    

Click here for diff

If -l was specified together with selective-restore options such as -n  
or -N, dependent TOC entries such as comments would be omitted from  
the listing, even when an actual restore would have selected them.  
This happened because PrintTOCSummary neglected to update the te->reqs  
marking of the entry they depended on.  
  
Per report from Justin Pryzby.  This has been wrong since 0d4e6ed30  
taught _tocEntryRequired to sometimes look at the "reqs" marking of  
other TOC entries, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/ZjoeirG7yxODdC4P@pryzbyj2023  

M src/bin/pg_dump/pg_backup_archiver.c

Don't corrupt plpython's "TD" dictionary in a recursive trigger call.

commit   : 363e8c2f98b80e039873902b3b3fd003ae09bd9e    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 7 May 2024 18:15:00 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 7 May 2024 18:15:00 -0400    

Click here for diff

If a plpython-language trigger caused another one to be invoked,  
the "TD" dictionary created for the inner one would overwrite the  
outer one's "TD" dictionary.  This is more or less the same problem  
that 1d2fe56e4 fixed for ordinary functions in plpython, so fix it  
the same way, by saving and restoring "TD" during a recursive  
invocation.  
  
This fix makes an ABI-incompatible change in struct PLySavedArgs.  
I'm not too worried about that because it seems highly unlikely that  
any extension is messing with those structs.  We could imagine doing  
something weird to preserve nominal ABI compatibility in the back  
branches, like keeping the saved TD object in an extra element of  
namedargs[].  However, that would only be very nominal compatibility:  
if anything *is* touching PLySavedArgs, it would likely do the wrong  
thing due to not knowing about the additional value.  So I judge it  
not worth the ugliness to do something different there.  
  
(I also changed struct PLyProcedure, but its added field fits  
into formerly-padding space, so that should be safe.)  
  
Per bug #18456 from Jacques Combrink.  This bug is very ancient,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plpython/expected/plpython_trigger.out
M src/pl/plpython/plpy_exec.c
M src/pl/plpython/plpy_procedure.c
M src/pl/plpython/plpy_procedure.h
M src/pl/plpython/sql/plpython_trigger.sql