PostgreSQL 16.6 commit log

commit   : 1f47e7b59b92c58eee6840325d1aaa6acee84194    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 18 Nov 2024 15:33:54 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 18 Nov 2024 15:33:54 -0500    

commit   : 7a0ecaa497d85b0445d2f06abf670ec329b59a5c    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 16 Nov 2024 17:09:53 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 16 Nov 2024 17:09:53 -0500    

commit   : ee33d58471de0f6e7ac7fca2f918b171efff6bb1    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 16 Nov 2024 12:58:26 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 16 Nov 2024 12:58:26 -0500    

Commits aac2c9b4f et al. added a bool field to struct ResultRelInfo.  
That's no problem in the master branch, but in released branches  
care must be taken when modifying publicly-visible structs to avoid  
an ABI break for extensions.  Frequently we solve that by adding the  
new field at the end of the struct, and that's what was done here.  
But ResultRelInfo has stricter constraints than just about any other  
node type in Postgres.  Some executor APIs require extensions to index  
into arrays of ResultRelInfo, which means that any change whatever in  
sizeof(ResultRelInfo) causes a fatal ABI break.  
  
Fortunately, this is easy to fix, because the new field can be  
squeezed into available padding space instead --- indeed, that's where  
it was put in master, so this fix also removes a cross-branch coding  
variation.  
  
Per report from Pavan Deolasee.  Patch v14-v17 only; earlier versions  
did not gain the extra field, nor is there any problem in master.  
  
Discussion: https://postgr.es/m/CABOikdNmVBC1LL6pY26dyxAS2f+gLZvTsNt=2XbcyG7WxXVBBQ@mail.gmail.com  

commit   : b0918c1286d316f6ffa93995452270afd4fc4335    
  
author   : Noah Misch <[email protected]>    
date     : Fri, 15 Nov 2024 20:39:56 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Fri, 15 Nov 2024 20:39:56 -0800    

After commit 5a2fed911a85ed6d8a015a6bafe3a0d9a69334ae, the catalog state  
resulting from these commands ceased to affect sessions.  Restore the  
longstanding behavior, which is like beginning the session with a SET  
ROLE command.  If cherry-picking the CVE-2024-10978 fixes, default to  
including this, too.  (This fixes an unintended side effect of fixing  
CVE-2024-10978.)  Back-patch to v12, like that commit.  The release team  
decided to include v12, despite the original intent to halt v12 commits  
earlier this week.  
  
Tom Lane and Noah Misch.  Reported by Etienne LAFARGE.  
  
Discussion: https://postgr.es/m/CADOZwSb0UsEr4_UTFXC5k7=fyyK8uKXekucd+-uuGjJsGBfxgw@mail.gmail.com  

commit   : f353911337cf0c291527528a9b21941a1a83fead    
  
author   : Masahiko Sawada <[email protected]>    
date     : Fri, 15 Nov 2024 17:06:05 -0800    
  
committer: Masahiko Sawada <[email protected]>    
date     : Fri, 15 Nov 2024 17:06:05 -0800    

Previously LogicalIncreaseRestartDecodingForSlot() accidentally  
accepted any LSN as the candidate_lsn and candidate_valid after the  
restart_lsn of the replication slot was updated, so it potentially  
caused the restart_lsn to move backwards.  
  
A scenario where this could happen in logical replication is: after a  
logical replication restart, based on previous candidate_lsn and  
candidate_valid values in memory, the restart_lsn advances upon  
receiving a subscriber acknowledgment. Then, logical decoding restarts  
from an older point, setting candidate_lsn and candidate_valid based  
on an old RUNNING_XACTS record. Subsequent subscriber acknowledgments  
then update the restart_lsn to an LSN older than the current value.  
  
In the reported case, after WAL files were removed by a checkpoint,  
the retreated restart_lsn prevented logical replication from  
restarting due to missing WAL segments.  
  
This change essentially modifies the 'if' condition to 'else if'  
condition within the function. The previous code had an asymmetry in  
this regard compared to LogicalIncreaseXminForSlot(), which does  
almost the same thing for different fields.  
  
The WAL removal issue was reported by Hubert Depesz Lubaczewski.  
  
Backpatch to all supported versions, since the bug exists since 9.4  
where logical decoding was introduced.  
  
Reviewed-by: Tomas Vondra, Ashutosh Bapat, Amit Kapila  
Discussion: https://postgr.es/m/Yz2hivgyjS1RfMKs%40depesz.com  
Discussion: https://postgr.es/m/85fff40e-148b-4e86-b921-b4b846289132%40vondra.me  
Backpatch-through: 13  

commit   : b6312becc81949381046159c488809f8baefd2f2    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 15 Nov 2024 18:23:38 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 15 Nov 2024 18:23:38 -0500    

In commit 08c0d6ad6 which introduced "rainbow" arcs in regex NFAs,  
I didn't think terribly hard about what to do when creating the color  
complement of a rainbow arc.  Clearly, the complement cannot match any  
characters, and I took the easy way out by just not building any arcs  
at all in the complement arc set.  That mostly works, but Nikolay  
Shaplov found a case where it doesn't: if we decide to delete that  
sub-NFA later because it's inside a "{0}" quantifier, delsub()  
suffered an assertion failure.  That's because delsub() relies on  
the target sub-NFA being fully connected.  That was always true  
before, and the best fix seems to be to restore that property.  
Hence, invent a new arc type CANTMATCH that can be generated in  
place of an empty color complement, and drop it again later when we  
start NFA optimization.  (At that point we don't need to do delsub()  
any more, and besides there are other cases where NFA optimization can  
lead to disconnected subgraphs.)  
  
It appears that this bug has no consequences in a non-assert-enabled  
build: there will be some transiently leaked NFA states/arcs, but  
they'll get cleaned up eventually.  Still, we don't like assertion  
failures, so back-patch to v14 where rainbow arcs were introduced.  
  
Per bug #18708 from Nikolay Shaplov.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ea1649c352f67bde3eafc0005fb70e3766f1153a    
  
author   : Álvaro Herrera <[email protected]>    
date     : Fri, 15 Nov 2024 12:53:12 +0100    
  
committer: Álvaro Herrera <[email protected]>    
date     : Fri, 15 Nov 2024 12:53:12 +0100    

Previously, in unlucky cases, it was possible for pg_rewind to remove  
certain WAL segments from the rewound demoted primary.  In particular  
this happens if those files have been marked for archival (i.e., their  
.ready files were created) but not yet archived; the newly promoted node  
no longer has such files because of them having been recycled, but they  
are likely critical for recovery in the demoted node.  If pg_rewind  
removes them, recovery is not possible anymore.  
  
Fix this by maintaining a hash table of files in this situation in the  
scan that looks for a checkpoint, which the decide_file_actions phase  
can consult so that it knows to preserve them.  
  
Backpatch to 14.  The problem also exists in 13, but that branch was not  
blessed with commit eb00f1d4bf96, so this patch is difficult to apply  
there.  Users of older releases will just have to continue to be extra  
careful when rewinding.  
  
Co-authored-by: Полина Бунгина (Polina Bungina) <[email protected]>  
Co-authored-by: Alexander Kukushkin <[email protected]>  
Reviewed-by: Kyotaro Horiguchi <[email protected]>  
Reviewed-by: Atsushi Torikoshi <[email protected]>  
Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com  

commit   : afa20845dd1348f602989155864fb2ff57f481d8    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 15 Nov 2024 11:32:16 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 15 Nov 2024 11:32:16 +0900    

This fixes a set of race conditions with cumulative statistics where a  
shared stats entry could be dropped while it should still be valid in  
the event when it is reused: an entry may refer to a different object  
but requires the same hash key.  This can happen with various stats  
kinds, like:  
- Replication slots that compute internally an index number, for  
different slot names.  
- Stats kinds that use an OID in the object key, where a wraparound  
causes the same key to be used if an OID is used for the same object.  
- As of PostgreSQL 18, custom pgstats kinds could also be an issue,  
depending on their implementation.  
  
This issue is fixed by introducing a counter called "generation" in the  
shared entries via PgStatShared_HashEntry, initialized at 0 when an  
entry is created and incremented when the same entry is reused, to avoid  
concurrent issues on drop because of other backends still holding a  
reference to it.  This "generation" is copied to the local copy that a  
backend holds when looking at an object, then cross-checked with the  
shared entry to make sure that the entry is not dropped even if its  
"refcount" justifies that if it has been reused.  
  
This problem could show up when a backend shuts down and needs to  
discard any entries it still holds, causing statistics to be removed  
when they should not, or even an assertion failure.  Another report  
involved a failure in a standby after an OID wraparound, where the  
startup process would FATAL on a "can only drop stats once", stopping  
recovery abruptly.  The buildfarm has been sporadically complaining  
about the problem, as well, but the window is hard to reach with the  
in-core tests.  
  
Note that the issue can be reproduced easily by adding a sleep before  
dshash_find() in pgstat_release_entry_ref() to enlarge the problematic  
window while repeating test_decoding's isolation test oldest_xmin a  
couple of times, for example, as pointed out by Alexander Lakhin.  
  
Reported-by: Alexander Lakhin, Peter Smith  
Author: Kyotaro Horiguchi, Michael Paquier  
Reviewed-by: Bertrand Drouvot  
Discussion: https://postgr.es/m/CAA4eK1KxuMVyAryz_Vk5yq3ejgKYcL6F45Hj9ZnMNBS-g+PuZg@mail.gmail.com  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 15  

commit   : 05aac2e83752b5d5e88638949b05017be6b8f1db    
  
author   : Peter Geoghegan <[email protected]>    
date     : Tue, 12 Nov 2024 20:57:41 -0500    
  
committer: Peter Geoghegan <[email protected]>    
date     : Tue, 12 Nov 2024 20:57:41 -0500    

Maintain the pg_stat_user_indexes.idx_scan pgstat counter during  
contrib/Bloom index scans.  
  
Oversight in commit 9ee014fc, which added the Bloom index contrib  
module.  
  
Author: Masahiro Ikeda <[email protected]>  
Reviewed-By: Peter Geoghegan <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch: 13- (all supported branches).  

commit   : b242aba02666db82390fdb5a4734efde77156f61    
  
author   : Alexander Korotkov <[email protected]>    
date     : Tue, 12 Nov 2024 01:44:20 +0200    
  
committer: Alexander Korotkov <[email protected]>    
date     : Tue, 12 Nov 2024 01:44:20 +0200    

The current code calls array_eq() and does not provide FmgrInfo.  This commit  
provides initialization of FmgrInfo and uses C collation as the safe option  
for text comparison because we don't know anything about the semantics of  
opclass options.  
  
Backpatch to 13, where opclass options were introduced.  
  
Reported-by: Nicolas Maus  
Discussion: https://postgr.es/m/18692-72ea398df3ec6712%40postgresql.org  
Backpatch-through: 13