PostgreSQL 12.9 commit log

commit   : 8a94efd9bb71c2d5473836ce4899aedb9b4cfb2e    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 17:02:19 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 17:02:19 -0500    

commit   : 9f06c0755e8f9dc8d196b3dc8722138bf04d5d59    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 14:02:16 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 14:02:16 -0500    

Security: CVE-2021-23214, CVE-2021-23222  

commit   : 36bb95ef2b5fdefaa99afbd859889a360e3c7763    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 11:14:56 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 11:14:56 -0500    

libpq collects up to a bufferload of data whenever it reads data from  
the socket.  When SSL or GSS encryption is requested during startup,  
any additional data received with the server's yes-or-no reply  
remained in the buffer, and would be treated as already-decrypted data  
once the encryption handshake completed.  Thus, a man-in-the-middle  
with the ability to inject data into the TCP connection could stuff  
some cleartext data into the start of a supposedly encryption-protected  
database session.  
  
This could probably be abused to inject faked responses to the  
client's first few queries, although other details of libpq's behavior  
make that harder than it sounds.  A different line of attack is to  
exfiltrate the client's password, or other sensitive data that might  
be sent early in the session.  That has been shown to be possible with  
a server vulnerable to CVE-2021-23214.  
  
To fix, throw a protocol-violation error if the internal buffer  
is not empty after the encryption handshake.  
  
Our thanks to Jacob Champion for reporting this problem.  
  
Security: CVE-2021-23222  

commit   : d1bd26740a62b979e9aacb6507593946a402e39c    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 11:01:43 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 8 Nov 2021 11:01:43 -0500    

The server collects up to a bufferload of data whenever it reads data  
from the client socket.  When SSL or GSS encryption is requested  
during startup, any additional data received with the initial  
request message remained in the buffer, and would be treated as  
already-decrypted data once the encryption handshake completed.  
Thus, a man-in-the-middle with the ability to inject data into the  
TCP connection could stuff some cleartext data into the start of  
a supposedly encryption-protected database session.  
  
This could be abused to send faked SQL commands to the server,  
although that would only work if the server did not demand any  
authentication data.  (However, a server relying on SSL certificate  
authentication might well not do so.)  
  
To fix, throw a protocol-violation error if the internal buffer  
is not empty after the encryption handshake.  
  
Our thanks to Jacob Champion for reporting this problem.  
  
Security: CVE-2021-23214  

commit   : f914b8badc21d646109fc62a4b84b4ef97347866    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 8 Nov 2021 09:17:24 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 8 Nov 2021 09:17:24 -0300    

Introduced in 1d97d3d0867f.  
  
Co-authored-by: Alexander Lakhin <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7678f76df4b8f933fbd2ac605f5456551322955c    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 8 Nov 2021 10:09:13 +0100    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 8 Nov 2021 10:09:13 +0100    

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 9128065fbbbb7b7b489a292773618c9273ff5c53  

commit   : 0bc98fb741f40e3f24a636afa562b8540a9f6d50    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 7 Nov 2021 14:21:51 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 7 Nov 2021 14:21:51 -0500    

commit   : 8f779a1a3eb6716faf47ef10e7bf0b883b8a1170    
  
author   : Alexander Korotkov <[email protected]>    
date     : Sat, 6 Nov 2021 18:31:21 +0300    
  
committer: Alexander Korotkov <[email protected]>    
date     : Sat, 6 Nov 2021 18:31:21 +0300    

Currently, lastOverflowedXid is never reset.  It's just adjusted on new  
transactions known to be overflowed.  But if there are no overflowed  
transactions for a long time, snapshots could be mistakenly marked as  
suboverflowed due to wraparound.  
  
This commit fixes this issue by resetting lastOverflowedXid when needed  
altogether with KnownAssignedXids.  
  
Backpatch to all supported versions.  
  
Reported-by: Stan Hu  
Discussion: https://postgr.es/m/CAMBWrQ%3DFp5UAsU_nATY7EMY7NHczG4-DTDU%3DmCvBQZAQ6wa2xQ%40mail.gmail.com  
Author: Kyotaro Horiguchi, Alexander Korotkov  
Reviewed-by: Stan Hu, Simon Riggs, Nikolay Samokhvalov, Andrey Borodin, Dmitry Dolgov  

commit   : 3eff168a29ca6d737611e8608504537d4ff18d50    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 5 Nov 2021 12:29:34 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 5 Nov 2021 12:29:34 -0300    

When a role being dropped contains is referenced by catalog objects that  
are concurrently also being dropped, a crash can result while trying to  
construct the string that describes the objects.  Suppress that by  
ignoring objects whose descriptions are returned as NULL.  
  
The majority of relevant codesites were already cautious about this  
already; we had just missed a couple.  
  
This is an old bug, so backpatch all the way back.  
  
Reported-by: Alexander Lakhin <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7dc9a310e5227920a9dbd3bfaf21d98f9f507ff9    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Wed, 3 Nov 2021 19:38:17 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Wed, 3 Nov 2021 19:38:17 +0200    

Previous commit added a test to 'largeobject', but neglected the  
alternative expected output file 'largeobject_1.source'. Per failure  
on buildfarm animal 'hamerkop'.  
  
Discussion: https://www.postgresql.org/message-id/[email protected]  

commit   : 11a399f9cfc3854f0029fc4dd355e296a03439f1    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Wed, 3 Nov 2021 10:28:52 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Wed, 3 Nov 2021 10:28:52 +0200    

If lo_export() fails to open the target file or to write to it, it leaks  
the created LargeObjectDesc and its snapshot in the top-transaction  
context and resource owner. That's pretty harmless, it's a small leak  
after all, but it gives the user a "Snapshot reference leak" warning.  
  
Fix by using a short-lived memory context and no resource owner for  
transient LargeObjectDescs that are opened and closed within one function  
call. The leak is easiest to reproduce with lo_export() on a directory  
that doesn't exist, but in principle the other lo_* functions could also  
fail.  
  
Backpatch to all supported versions.  
  
Reported-by: Andrew B  
Reviewed-by: Alvaro Herrera  
Discussion: https://www.postgresql.org/message-id/[email protected]  

commit   : da782bc9341ca3ebeab1ec214f2610c4778234c2    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 1 Nov 2021 13:07:23 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 1 Nov 2021 13:07:23 -0300    

Failing to do so results in inability of logical decoding to process the  
WAL stream.  Handle it by doing nothing.  
  
Backpatch all the way back.  
  
Reported-by: Petr Jelínek <[email protected]>  

commit   : 16d0da5c8dda7340d6b1e724ad183313e6a254f2    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 31 Oct 2021 19:13:48 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 31 Oct 2021 19:13:48 -0400    

Windows fails on a request to read() more than INT_MAX bytes,  
and perhaps other platforms could have similar issues.  Let's  
adjust this code to read at most 1GB per call.  
  
(One would not have thought the file could get that big, but now  
we have a field report of trouble, so it can.  We likely ought to  
add some mechanism to limit the size of the query-texts file  
separately from the size of the hash table.  That is not this  
patch, though.)  
  
Per bug #17254 from Yusuke Egashira.  It's been like this for  
awhile, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 14b8d25d66bebd0becf03680a8684bb20f8f706c    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 29 Oct 2021 11:38:18 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 29 Oct 2021 11:38:18 -0400    

DST law changes in Fiji, Jordan, Palestine, and Samoa.  Historical  
corrections for Barbados, Cook Islands, Guyana, Niue, Portugal, and  
Tonga.  
  
Also, the Pacific/Enderbury zone has been renamed to Pacific/Kanton.  
The following zones have been merged into nearby, more-populous zones  
whose clocks have agreed since 1970: Africa/Accra, America/Atikokan,  
America/Blanc-Sablon, America/Creston, America/Curacao,  
America/Nassau, America/Port_of_Spain, Antarctica/DumontDUrville,  
and Antarctica/Syowa.  

commit   : 141cd0ef0b0b462a8c6e1ee5ae29c7ae4221b90e    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 28 Oct 2021 11:45:14 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 28 Oct 2021 11:45:14 -0400    

Commits fdd965d07 and 3cd9c3b92 tested CREATE INDEX CONCURRENTLY by  
launching two separate pgbench runs concurrently.  This was needed so  
that only a single client thread would run CREATE INDEX CONCURRENTLY,  
avoiding deadlock between two CICs.  However, there's a better way,  
which is to use an advisory lock to prevent concurrent CICs.  That's  
better in part because the test code is shorter and more readable, but  
mostly because it automatically scales things to launch an appropriate  
number of CICs relative to the number of INSERT transactions.  
As committed, typically half to three-quarters of the CIC transactions  
were pointless because the INSERT transactions had already stopped.  
  
In passing, remove background_pgbench, which was added to support  
these tests and isn't needed anymore.  We can always put it back  
if we find a use for it later.  
  
Back-patch to v12; older pgbench versions lack the  
conditional-execution features needed for this method.  
  
Tom Lane and Andrey Borodin  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 4badc594d6396fb53582d154d0bfbf2f5e8e2bed    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 28 Oct 2021 09:26:21 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 28 Oct 2021 09:26:21 +0900    

Reported-by: Anton Voloshin  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 10  

commit   : ad4aaf7221a70048da642652d1c379f0c5a83f50    
  
author   : Magnus Hagander <[email protected]>    
date     : Wed, 27 Oct 2021 16:20:02 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Wed, 27 Oct 2021 16:20:02 +0200    

Make this more clear both in the help message and docs.  
  
Reviewed-By: Michael Paquier  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/CABUevEw6Je0WUFTLhPKOk4+BoBuDrE-fKw3N4ckqgDBMFu4paA@mail.gmail.com  

commit   : 8fef901e3fe2119c434c93c9f05063633a43eaad    
  
author   : Thomas Munro <[email protected]>    
date     : Tue, 26 Oct 2021 12:54:55 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Tue, 26 Oct 2021 12:54:55 +1300    

It doesn't work (it could, but hasn't been implemented).  
Back-patch to 12, where shared_memory_type arrived.  
  
Reported-by: Alexander Lakhin <[email protected]>  
Reviewed-by: Alexander Lakhin <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : 90eb17088a3659d0a60f3d1d59cef9325705aa81    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 23 Oct 2021 19:36:26 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 23 Oct 2021 19:36:26 -0700    

Back-patch commit 0516f94d18c57cc5dce72ff8fb84a05d24a67063 to v12 and  
v11.  Other back-patches will bring in code written to later standards.  
Per buildfarm member crake.  

commit   : fe5d44a1d38fc6c2a1adddf2eaf28726f93d2399    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 23 Oct 2021 18:36:38 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 23 Oct 2021 18:36:38 -0700    

The purpose of commit 8a54e12a38d1545d249f1402f66c8cde2837d97c was to  
fix this, and it sufficed when the PREPARE TRANSACTION completed before  
the CIC looked for lock conflicts.  Otherwise, things still broke.  As  
before, in a cluster having used CIC while having enabled prepared  
transactions, queries that use the resulting index can silently fail to  
find rows.  It may be necessary to reindex to recover from past  
occurrences; REINDEX CONCURRENTLY suffices.  Fix this for future index  
builds by making CIC wait for arbitrarily-recent prepared transactions  
and for ordinary transactions that may yet PREPARE TRANSACTION.  As part  
of that, have PREPARE TRANSACTION transfer locks to its dummy PGPROC  
before it calls ProcArrayClearTransaction().  Back-patch to 9.6 (all  
supported versions).  
  
Andrey Borodin, reviewed (in earlier versions) by Andres Freund.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0869e53d3a077a6840dbcfe86d4b4fc6b2906fab    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 23 Oct 2021 18:36:38 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 23 Oct 2021 18:36:38 -0700    

CIC and REINDEX CONCURRENTLY assume backends see their catalog changes  
no later than each backend's next transaction start.  That failed to  
hold when a backend absorbed a relevant invalidation in the middle of  
running RelationBuildDesc() on the CIC index.  Queries that use the  
resulting index can silently fail to find rows.  Fix this for future  
index builds by making RelationBuildDesc() loop until it finishes  
without accepting a relevant invalidation.  It may be necessary to  
reindex to recover from past occurrences; REINDEX CONCURRENTLY suffices.  
Back-patch to 9.6 (all supported versions).  
  
Noah Misch and Andrey Borodin, reviewed (in earlier versions) by Andres  
Freund.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : fb1aa489e481eb28aee18d4d2d268e76d46740ed    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 23 Oct 2021 14:43:48 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 23 Oct 2021 14:43:48 +0900    

The documentation was imprecise about the starting LSN used for WAL  
streaming if nothing can be found in the local archive directory  
defined with the pg_receivewal command, so be more talkative on this  
matter.  
  
Extracted from a larger patch by the same author.  
  
Author: Ronan Dunklau, Michael Paquier  
Discussion: https://postgr.es/m/18708360.4lzOvYHigE@aivenronan  
Backpatch-through: 10  

commit   : 52b927a731e531005c9fb9f895390a66d75cddb7    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 22 Oct 2021 15:22:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 22 Oct 2021 15:22:26 -0400    

Non-global default privilege entries should be dumped as-is,  
not made relative to the default ACL for their object type.  
This would typically only matter if one had revoked some  
on-by-default privileges in a global entry, and then wanted  
to grant them again in a non-global entry.  
  
Per report from Boris Korzun.  This is an old bug, so back-patch  
to all supported branches.  
  
Neil Chen, test case by Masahiko Sawada  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/CAA3qoJnr2+1dVJObNtfec=qW4Z0nz=A9+r5bZKoTSy5RDjskMw@mail.gmail.com  

commit   : 4048fc458175dbcd7cd81e8ef75ca1ba0ab9bdae    
  
author   : Amit Kapila <[email protected]>    
date     : Thu, 21 Oct 2021 09:50:06 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Thu, 21 Oct 2021 09:50:06 +0530    

This was originally done in commit 5e77625b26 for 15 only, as a  
troubleshooting aid but multiple people showed interest in back-patching  
this.  
  
Author: Jeremy Schneider  
Reviewed-by: Amit Kapila  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/[email protected]  

commit   : 3c8c49945dfcbb6e045d9cb9d1da59fed822649d    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 20 Oct 2021 13:05:42 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 20 Oct 2021 13:05:42 -0300    

Discussion: https://postgr.es/m/YW/[email protected]  

commit   : 1539e0ecd6a447fca14920834d4546c60ff20f10    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 20 Oct 2021 16:49:03 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 20 Oct 2021 16:49:03 +0900    

The build scripts of Visual Studio would fail to detect properly a 3.0.0  
build as the check on the second digit was failing.  This is adjusted  
where needed, allowing the builds to complete.  Note that the MSIs of  
OpenSSL mentioned in the documentation have not changed any library  
names for Win32 and Win64, making this change straight-forward.  
  
Reported-by: htalaco, via github  
Reviewed-by: Daniel Gustafsson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 6c8d1c197bd67e78c12c68d8f0bd111a59e7bccd    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 19 Oct 2021 19:08:45 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 19 Oct 2021 19:08:45 -0300    

Commit 1b5d797cd4f7 intended to relax the lock level used to rename  
indexes, but inadvertently allowed *any* relation to be renamed with a  
lowered lock level, as long as the command is spelled ALTER INDEX.  
That's undesirable for other relation types, so retry the operation with  
the higher lock if the relation turns out not to be an index.  
  
After this fix, ALTER INDEX <sometable> RENAME will require access  
exclusive lock, which it didn't before.  
  
Author: Nathan Bossart <[email protected]>  
Author: Álvaro Herrera <[email protected]>  
Reported-by: Onder Kalaci <[email protected]>  
Discussion: https://postgr.es/m/PH0PR21MB1328189E2821CDEC646F8178D8AE9@PH0PR21MB1328.namprd21.prod.outlook.com  

commit   : 9aef4015573905ff8edec19e38c47ff820277153    
  
author   : Andres Freund <[email protected]>    
date     : Tue, 19 Oct 2021 10:14:49 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Tue, 19 Oct 2021 10:14:49 -0700    

ldapsearch's deprecated -h/-p arguments were removed, need to use -H now -  
which has been around for over 20 years.  
  
As perltidy insists on reflowing the parameters anyway, change order and  
"phrasing" to yield a less confusing layout (per suggestion from Tom Lane).  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch: 11-, where the tests were added.  

commit   : ae7b1dd5900156c3d58d88bf47532adcb27cca26    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 19 Oct 2021 13:54:46 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 19 Oct 2021 13:54:46 -0400    

An update such as "UPDATE ... SET fld[n].subfld = whatever"  
failed if the array elements were domains rather than plain  
composites.  That's because isAssignmentIndirectionExpr()  
failed to cope with the CoerceToDomain node that would appear  
in the expression tree in this case.  The result would typically  
be a crash, and even if we accidentally didn't crash, we'd not  
correctly preserve other fields of the same array element.  
  
Per report from Onder Kalaci.  Back-patch to v11 where arrays of  
domains came in.  
  
Discussion: https://postgr.es/m/PH0PR21MB132823A46AA36F0685B7A29AD8BD9@PH0PR21MB1328.namprd21.prod.outlook.com  

commit   : 079ac0192b90e0f1d884c00dd3a3fca2cded50b2    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 19 Oct 2021 11:35:15 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 19 Oct 2021 11:35:15 -0400    

I think when I added this assertion (in commit 8f889b108), I was only  
thinking of the use of transformExpressionList at top level of INSERT  
and VALUES.  But it's also called by transformRowExpr(), which can  
certainly occur in an UPDATE targetlist, so it's inappropriate to  
suppose that p_multiassign_exprs must be empty.  Besides, since the  
input is not expected to contain ResTargets, there's no reason it  
should contain MultiAssignRefs either.  Hence this code need not  
be concerned about the state of p_multiassign_exprs, and we should  
just drop the assertion.  
  
Per bug #17236 from ocean_li_996.  It's been wrong for years,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : e788883de715be00733615c2224149332f49b0f0    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Tue, 19 Oct 2021 12:59:54 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Tue, 19 Oct 2021 12:59:54 +0200    

If the blob TOC file cannot be parsed, the error message was failing  
to print the filename as the variable holding it was shadowed by the  
destination buffer for parsing.  When the filename fails to parse,  
the error will print an empty string:  
  
 ./pg_restore -d foo -F d dump  
 pg_restore: error: invalid line in large object TOC file "": ..  
  
..instead of the intended error message:  
  
 ./pg_restore -d foo -F d dump  
 pg_restore: error: invalid line in large object TOC file "dump/blobs.toc": ..  
  
Fix by renaming both variables as the shared name was too generic to  
store either and still convey what the variable held.  
  
Backpatch all the way down to 9.6.  
  
Reviewed-by: Tom Lane  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 57bf8f7b75ec3f1509175006ccf3c139c0d1adc8    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Tue, 19 Oct 2021 12:59:50 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Tue, 19 Oct 2021 12:59:50 +0200    

Make sure that the string parsing is limited by the size of the  
destination buffer.  
  
In pg_basebackup the available values sent from the server  
is limited to two characters so there was no risk of overflow.  
  
In pg_dump the buffer is bounded by MAXPGPATH, and thus the limit  
must be inserted via preprocessor expansion and the buffer increased  
by one to account for the terminator. There is no risk of overflow  
here, since in this case, the buffer scanned is smaller than the  
destination buffer.  
  
Backpatch the pg_basebackup fix to 11 where it was introduced, and  
the pg_dump fix all the way down to 9.6.  
  
Reviewed-by: Tom Lane  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 11 and 9.6  

commit   : 8b26be8a32c31d45d3e11bcdab39526969201c88    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 18 Oct 2021 19:08:25 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 18 Oct 2021 19:08:25 -0300    

Failing to do that, any direct inserts/updates of those partitions  
would fail to enforce the correct constraint, that is, one that  
considers the new partition constraint of their parent table.  
  
Backpatch to 10.  
  
Reported by: Hou Zhijie <[email protected]>  
Author: Amit Langote <[email protected]>  
Author: Álvaro Herrera <[email protected]>  
Reviewed-by: Nitin Jadhav <[email protected]>  
Reviewed-by: Pavel Borisov <[email protected]>  
  
Discussion: https://postgr.es/m/OS3PR01MB5718DA1C4609A25186D1FBF194089%40OS3PR01MB5718.jpnprd01.prod.outlook.com  

commit   : a207b8521346bf9575d7d0c7585a3de8250d4ef8    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 18 Oct 2021 11:56:54 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 18 Oct 2021 11:56:54 +0900    

During a replication slot creation, an ERROR generated in the same  
transaction as the one creating a to-be-exported snapshot would have  
left the backend in an inconsistent state, as the associated static  
export snapshot state was not being reset on transaction abort, but only  
on the follow-up command received by the WAL sender that created this  
snapshot on replication slot creation.  This would trigger inconsistency  
failures if this session tried to export again a snapshot, like during  
the creation of a replication slot.  
  
Note that a snapshot export cannot happen in a transaction block, so  
there is no need to worry resetting this state for subtransaction  
aborts.  Also, this inconsistent state would very unlikely show up to  
users.  For example, one case where this could happen is an  
out-of-memory error when building the initial snapshot to-be-exported.  
Dilip found this problem while poking at a different patch, that caused  
an error in this code path for reasons unrelated to HEAD.  
  
Author: Dilip Kumar  
Reviewed-by: Michael Paquier, Zhihong Yu  
Discussion: https://postgr.es/m/CAFiTN-s0zA1Kj0ozGHwkYkHwa5U0zUE94RSc_g81WrpcETB5=w@mail.gmail.com  
Backpatch-through: 9.6  

commit   : 5b0b2983a095f2f6ed18d9c3387582e4b33e95a1    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 16 Oct 2021 15:02:55 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 16 Oct 2021 15:02:55 -0400    

Commit f0e21f2f6 missed adding a tgisinternal output column  
to getTriggers' query for pre-8.3 servers.  Back-patch to v11,  
like that commit.  

commit   : fd182a92ad90e2328c8e1fb601ca41f89a6a5585    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 16 Oct 2021 12:23:57 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 16 Oct 2021 12:23:57 -0400    

It was clearly the intent to do so all along, but the original coding  
fat-fingered this by checking the wrong array element.  We fixed it  
in passing in 403a3d91c, but that later got reverted, and we forgot  
to keep this bug fix.  
  
Most of the time this'd be relatively harmless, since once we lock  
any of the partitioned table's leaf partitions, that would suffice  
to prevent major DDL on the partitioned table itself.  However, a  
childless partitioned table would get dumped with no relevant lock  
whatsoever, possibly allowing dump failure or inconsistent output.  
  
Unlike 403a3d91c, there are no versioning concerns, since every server  
version that has partitioned tables will allow you to lock one.  
  
Back-patch to v10 where partitioned tables were introduced.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ab11f6e461b9f2f4f6c4d812218a07af5d1c238d    
  
author   : Jeff Davis <[email protected]>    
date     : Thu, 14 Oct 2021 12:25:07 -0700    
  
committer: Jeff Davis <[email protected]>    
date     : Thu, 14 Oct 2021 12:25:07 -0700    

An extension may want to call GetSecurityLabel() on a shared object  
before the shared relcaches are fully initialized. For instance, a  
ClientAuthentication_hook might want to retrieve the security label on  
a role.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 4b7abbe48a118e2b00f681e191da605e0d86b9c7    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 13 Oct 2021 18:49:27 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 13 Oct 2021 18:49:27 -0300    

The test code added with ff9f111bce24 fails under valgrind, and probably  
other slow cases too, because if (say) autovacuum runs in between and  
produces WAL of its own, the large INSERT fails to account for that in  
the LSN calculations.  Rewrite to use a DO loop.  
  
Per complaint from Andres Freund  
  
Backpatch to all branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 48dc8479c1c431c1b9da372f819abe0d0790a740    
  
author   : Etsuro Fujita <[email protected]>    
date     : Wed, 13 Oct 2021 19:00:05 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Wed, 13 Oct 2021 19:00:05 +0900    

The comments were misplaced when adding postgres_fdw.  Fix that by  
moving the comments to more appropriate functions.  
  
Author: Etsuro Fujita  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/CAPmGK164sAXQtC46mDFyu6d-T25Mzvh5qaRNkit06VMmecYnOA%40mail.gmail.com  

commit   : afa09e4a9af6dd6491a67c4e25b21a45b3a78006    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 13 Oct 2021 09:22:00 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 13 Oct 2021 09:22:00 +0900    

This fixes a set of issues that cause different breakages or annoyances  
when using pg_upgrade's test.sh to do upgrades across different major  
versions:  
- test.sh is completely broken when using v14 as new version because of  
the removal of testtablespace/ as Makefile rule.  Older versions of  
pg_regress don't support --make-tablespacedir, blocking the creation of  
the tablespace.  In order to fix that, it is simple enough to create  
those directories in the script itself, but only do that when an old  
version is involved.  This fix is needed on HEAD and REL_14_STABLE.  
- The script would fail when using PG <= v11 as old version because of  
WITH OIDS relations not supported in v12.  In order to fix this, this  
steals a method from the buildfarm that uses a DO block to change all  
the relations marked as WITH OIDS, allowing pg_upgrade to pass.  This is  
more portable than using ALTER TABLE queries on the relations causing  
issues.  This is fixed down to v12, and authored originally by Andrew  
Dunstan.  
- Not using --extra-float-digits=0 with v11 as old version causes  
a lot of diffs in the dumps, making the whole unreadable.  This gets  
only done when using v11 as old version.  This is fixed down to v12.  
The buildfarm code uses that already.  
  
Note that the addition of --wal-segsize and --allow-group-access breaks  
the script when using v10 or older at initdb time as these got added in  
11.  10 would be EOL'd next year and nobody has complained about those  
problems yet, so nothing is done about that.  This means that this  
commit fixes upgrade tests using test.sh with v11 as minimum older  
version, up to HEAD, and that it is enough to apply this change down to  
12.  The old and new dumps still generate diffs, still require manual  
checks, and more could be done to reduce the noise, but this allows the  
tests to run with a rather minimal amount of them.  
  
I have tested this commit and test.sh with v11 as minimum across all the  
branches where this is applied.  Note that this commit has no impact on  
the normal pg_upgrade test run with a simple "make check".  
  
Author:  Justin Pryzby, Andrew Dunstan, Michael Paquier  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

commit   : d5ab331a339d1a5f484e514e8cd231c291261dec    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 12 Oct 2021 11:16:30 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 12 Oct 2021 11:16:30 +0900    

Incrementing the level of the call stack reported is useful for  
debugging purposes as it allows to control which part of the test is  
exactly failing, especially if a test is structured with subroutines  
that call routines from Test::More.  
  
This adds more incrementations of $Test::Builder::Level where debugging  
gets improved (for example it does not make sense for some paths like  
pg_rewind where long subroutines are used).  
  
A note is added to src/test/perl/README about that, based on a  
suggestion from Andrew Dunstan and a wording coming from both of us.  
  
Usage of Test::Builder::Level has spread in 12, so a backpatch down to  
this version is done.  
  
Reviewed-by: Andrew Dunstan, Peter Eisentraut, Daniel Gustafsson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

commit   : 2288973744e8bd4715d1daaaf2cd1d9b534aa1bf    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 6 Oct 2021 15:50:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 6 Oct 2021 15:50:24 -0400    

Commit c7b7311f6 adjusted conversion_error_callback to always use  
information from the query's rangetable, to avoid doing catalog lookups  
in an already-failed transaction.  However, as a result of the utterly  
inadequate documentation for make_tuple_from_result_row, I failed to  
realize that fsstate could be NULL in some contexts.  That led to a  
crash if we got a conversion error in such a context.  Fix by falling  
back to the previous coding when fsstate is NULL.  Improve the  
commentary, too.  
  
Per report from Andrey Borodin.  Back-patch to 9.6, like the previous  
patch.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 676218034fa8361cc79b1b4631e220b5a7ec6074    
  
author   : Dean Rasheed <[email protected]>    
date     : Wed, 6 Oct 2021 13:21:27 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Wed, 6 Oct 2021 13:21:27 +0100    

This fixes a loss of precision that occurs when the first input is  
very close to 1, so that its logarithm is very small.  
  
Formerly, during the initial low-precision calculation to estimate the  
result weight, the logarithm was computed to a local rscale that was  
capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be  
as close as 1e-16383 to 1, hence its logarithm may be as small as  
1e-16383, and so the local rscale needs to be allowed to exceed 16383,  
otherwise all precision is lost, leading to a poor choice of rscale  
for the full-precision calculation.  
  
Fix this by removing the cap on the local rscale during the initial  
low-precision calculation, as we already do in the full-precision  
calculation. This doesn't change the fact that the initial calculation  
is a low-precision approximation, computing the logarithm to around 8  
significant digits, which is very fast, especially when the base is  
very close to 1.  
  
Patch by me, reviewed by Alvaro Herrera.  
  
Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com  

commit   : b6cf89b025f82a9ab5c2961f216ca6a2751ad793    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 5 Oct 2021 10:24:14 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 5 Oct 2021 10:24:14 -0400    

queries.sgml failed to mention the rather important point that  
INTERSECT binds more tightly than UNION or EXCEPT.  I thought  
it could also use more discussion of the role of parentheses  
in these constructs.  
  
Per gripe from Christopher Painter-Wakefield.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 36b58ac6f6d0c4ce51ffb413ba376c2e8f95126f    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 4 Oct 2021 17:10:59 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 4 Oct 2021 17:10:59 -0400    

The old URL was HTTP 404 and the git link didn't build.  Also update two  
other ICU links.  If we ever get a good link we will add it back.  
  
Reported-by: Anton Voloshin  
  
Author: Laurenz Albe  
  
Backpatch-through: 10  

commit   : cd1b2334b8a2c18f54e7ade408c9133805cae557    
  
author   : Andres Freund <[email protected]>    
date     : Mon, 4 Oct 2021 13:28:06 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Mon, 4 Oct 2021 13:28:06 -0700    

3c5b0685b921 used setFilePointer() to set the position of the filehandle, but  
passed the wrong filehandle, always leaving the position at 0. Instead of just  
fixing that, remove use of setFilePointer(), we have a perl fd at this point,  
so we can just use perl's seek().  
  
Additionally, the perl filehandle wasn't closed, just the windows filehandle.  
  
Reviewed-By: Andrew Dunstan <[email protected]>  
Author: Andres Freund <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch: 9.6-, like 3c5b0685b921  

commit   : 07873a5dc925059fc76ddfd81db9a545faf06d33    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 4 Oct 2021 14:52:17 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 4 Oct 2021 14:52:17 -0400    

Per discussion, let's just follow CLDR's default zone mappings  
faithfully.  There are two changes here that are clear improvements:  
  
* Mapping "Greenwich Standard Time" to Atlantic/Reykjavik is actually  
a better fit than using London, because Iceland hasn't observed DST  
since 1968, so this is more nearly what people might expect.  
  
* Since the "Samoa" zone is specified to be UTC+13:00, we must map  
it to Pacific/Apia not Pacific/Samoa; the latter refers to American  
Samoa which is now on the other side of the date line.  
  
The rest of these changes look like they're choosing the most populous  
IANA zone as representative.  Whatever the details, we're just going  
to say "if you don't like this mapping, complain to CLDR".  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 3c3f118d503ceebc01366ed1c09c967bac153405    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 4 Oct 2021 14:05:55 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 4 Oct 2021 14:05:55 +0900    

Some specific logic is done at the end of recovery when involving 2PC  
transactions:  
1) Call RecoverPreparedTransactions(), to recover the state of 2PC  
transactions into memory (re-acquire locks, etc.).  
2) ShutdownRecoveryTransactionEnvironment(), to move back to normal  
operations, mainly cleaning up recovery locks and KnownAssignedXids  
(including any 2PC transaction tracked previously).  
3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is  
the tipping point for any process calling RecoveryInProgress() to check  
if the cluster is still in recovery or not.  
  
Any snapshot taken between steps 2) and 3) would be empty, causing any  
transaction relying on a snapshot at this point to potentially corrupt  
data as there could still be some 2PC transactions to track, with  
RecentXmin moving backwards on successive calls to GetSnapshotData() in  
the same transaction.  
  
As SharedRecoveryState is the point to take into account to know if it  
is safe to discard KnownAssignedXids, this commit moves step 2) after  
step 3), so as we can never finish with empty snapshots.  
  
This exists since the introduction of hot standby, so backpatch all the  
way down.  The window with incorrect snapshots is extremely small, but I  
have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm  
member fairywren.  Thomas Munro also found it independently.  Special  
thanks to Andres Freund for taking the time to analyze this issue.  
  
Reported-by: Thomas Munro, Michael Paquier  
Analyzed-by: Andres Freund  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : e5b25f19b37b81cd4933abf4416858a626706291    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 2 Oct 2021 16:05:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 2 Oct 2021 16:05:42 -0400    

This corrects a bunch of entries in win32_tzmap[], and adds a few  
new ones, based on the CLDR project's windowsZones.xml file.  
Non-cosmetic changes fall into four main categories:  
  
* Flat-out errors:  
  
US/Aleutan doesn't exist  
America/Salvador doesn't exist  
Asia/Baku is wrong for Yerevan  
Asia/Dhaka (Bangladesh) is wrong for Astana (Kazakhstan)  
Europe/Bucharest is wrong for Chisinau  
America/Mexico_City is wrong for Chetumal  
America/Buenos_Aires is wrong for Cayenne  
America/Caracas has its own zone, so poor fit for La Paz  
US/Eastern is wrong for Haiti  
US/Eastern is wrong for Indiana (East)  
Asia/Karachi is wrong for Tashkent  
Etc/UTC+12 doesn't exist  
Signs of Etc/GMT zones were backwards  
  
* Judgment calls:  
  
(These changes follow CLDR's choices, except for the first one)  
  
Use Europe/London for "Greenwich Standard Time", since that seems much  
more likely than Africa/Casablanca to be what people will think that  
zone name means.  CLDR has Atlantic/Reykjavik here, but that's no better.  
  
Asia/Shanghai seems a better fit than Hong Kong for "China Standard  
Time".  
  
Europe/Sarajevo is now a link to Belgrade, ie "Central Europe Standard  
Time"; so use Warsaw for "Central European Standard Time".  
  
America/Sao_Paulo seems more representative than Araguaina for  
"E. South America Standard Time".  
  
Africa/Johannesburg seems more representative than Harare for  
"South Africa Standard Time".  
  
* New Windows zone names:  
  
"Israel Standard Time"  
"Kaliningrad Standard Time"  
"Russia Time Zone N" for various N  
"Singapore Standard Time"  
"South Sudan Standard Time"  
"W. Central Africa Standard Time"  
"West Bank Standard Time"  
"Yukon Standard Time"  
  
Some of these replace older spellings, but I kept the older spellings  
too in case our code runs on a machine with the older data.  
  
* Replace aliases (tzdb Links) with underlying city-named zones:  
  
(This tracks tzdb's longstanding practice, and reduces inconsistency  
with the rest of the entries, as well as with CLDR.)  
  
US/Alaska  
Asia/Kuwait  
Asia/Muscat  
Canada/Atlantic  
Australia/Canberra  
Canada/Saskatchewan  
US/Central  
US/Eastern  
US/Hawaii  
US/Mountain  
Canada/Newfoundland  
US/Pacific  
  
Back-patch to all supported branches, as is our usual practice for  
time zone data updates.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 4721e8aa6223f214fcd5761334e6e682fef66636    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 2 Oct 2021 16:05:10 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 2 Oct 2021 16:05:10 -0400    

The original intent seems to have been to sort case-insensitively  
by the Windows zone name, but various changes over the years did  
not get that memo.  This commit just moves a few entries to  
restore exact alphabetic order, to ease comparison to the outputs  
of processing scripts.  
  
Back-patch to all supported branches, as is our usual practice for  
time zone data updates.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 334fb8c3de4da7bcf91359db2e796fd4e9ee2062    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 1 Oct 2021 14:59:35 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 1 Oct 2021 14:59:35 -0400    

get_variable_range() would incautiously believe that statistics  
containing only an MCV list are sufficient to derive a range estimate.  
That's okay for an enum-like column that contains only MCVs, but  
otherwise the estimate could be pretty bad.  Make it report that the  
range is indeterminate unless the MCVs plus nullfrac account for  
the whole table.  
  
I don't think this needs a dedicated test case, since a quick code  
coverage check verifies that the existing regression tests traverse  
all the alternatives.  There is room to doubt that a future-proof  
test case could be built anyway, given that the submitted example  
accidentally doesn't fail before v11.  
  
Per bug #17207 from Simon Perepelitsa.  Back-patch to v10.  
In principle this has been broken all along, but I'm hesitant to  
make such changes in 9.6, since if anyone is unhappy with 9.6.24's  
behavior there will be no second chance to fix it.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : cded2c46099eecc9fc8572bc09675407fb178c44    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 1 Oct 2021 11:10:12 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 1 Oct 2021 11:10:12 -0400    

Commit 84f5c2908 forgot to consider the possibility that  
EnsurePortalSnapshotExists could run inside a subtransaction with  
lifespan shorter than the Portal's.  In that case, the new active  
snapshot would be popped at the end of the subtransaction, leaving  
a dangling pointer in the Portal, with mayhem ensuing.  
  
To fix, make sure the ActiveSnapshot stack entry is marked with  
the same subtransaction nesting level as the associated Portal.  
It's certainly safe to do so since we won't be here at all unless  
the stack is empty; hence we can't create an out-of-order stack.  
  
Let's also apply this logic in the case where PortalRunUtility  
sets portalSnapshot, just to be sure that path can't cause similar  
problems.  It's slightly less clear that that path can't create  
an out-of-order stack, so add an assertion guarding it.  
  
Report and patch by Bertrand Drouvot (with kibitzing by me).  
Back-patch to v11, like the previous commit.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : f2cf745a038c984092b74fe5b82b892721f9310e    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 30 Sep 2021 16:23:10 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 30 Sep 2021 16:23:10 -0400    

Computing related timestamps by subtracting "N days" is sensitive  
to the prevailing timezone, since we interpret that as "same local  
time on the N'th prior day".  Even though the intervals in question  
are only two to four days, through remarkable bad luck they managed  
to cross the end of Ramadan in 2014, causing the test's output to  
change if timezone is set to Africa/Casablanca.  (Maybe in other  
Muslim areas as well; I didn't check.)  There's absolutely no reason  
for this test to exercise interval subtraction, so just get rid of  
that and use plain timestamptz constants representing the intended  
values.  
  
Per report from Andres Freund.  Back-patch to v10 where this test  
script came in.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 1df0a914d58f2bdb03c11dfcd2cb9cd01c286d59    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 29 Sep 2021 11:21:51 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 29 Sep 2021 11:21:51 -0300    

Physical replication always ships WAL segment files to replicas once  
they are complete.  This is a problem if one WAL record is split across  
a segment boundary and the primary server crashes before writing down  
the segment with the next portion of the WAL record: WAL writing after  
crash recovery would happily resume at the point where the broken record  
started, overwriting that record ... but any standby or backup may have  
already received a copy of that segment, and they are not rewinding.  
This causes standbys to stop following the primary after the latter  
crashes:  
  LOG:  invalid contrecord length 7262 at A8/D9FFFBC8  
because the standby is still trying to read the continuation record  
(contrecord) for the original long WAL record, but it is not there and  
it will never be.  A workaround is to stop the replica, delete the WAL  
file, and restart it -- at which point a fresh copy is brought over from  
the primary.  But that's pretty labor intensive, and I bet many users  
would just give up and re-clone the standby instead.  
  
A fix for this problem was already attempted in commit 515e3d84a0b5, but  
it only addressed the case for the scenario of WAL archiving, so  
streaming replication would still be a problem (as well as other things  
such as taking a filesystem-level backup while the server is down after  
having crashed), and it had performance scalability problems too; so it  
had to be reverted.  
  
This commit fixes the problem using an approach suggested by Andres  
Freund, whereby the initial portion(s) of the split-up WAL record are  
kept, and a special type of WAL record is written where the contrecord  
was lost, so that WAL replay in the replica knows to skip the broken  
parts.  With this approach, we can continue to stream/archive segment  
files as soon as they are complete, and replay of the broken records  
will proceed across the crash point without a hitch.  
  
Because a new type of WAL record is added, users should be careful to  
upgrade standbys first, primaries later. Otherwise they risk the standby  
being unable to start if the primary happens to write such a record.  
  
A new TAP test that exercises this is added, but the portability of it  
is yet to be seen.  
  
This has been wrong since the introduction of physical replication, so  
backpatch all the way back.  In stable branches, keep the new  
XLogReaderState members at the end of the struct, to avoid an ABI  
break.  
  
Author: Álvaro Herrera <[email protected]>  
Reviewed-by: Kyotaro Horiguchi <[email protected]>  
Reviewed-by: Nathan Bossart <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : c5f7e702d78f5b49a80b6e8b69e2b4019c1fca4b    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 29 Sep 2021 21:01:10 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 29 Sep 2021 21:01:10 +0900    

Previously socket errors such as invalid socket or socket wait method failures  
during benchmark caused pgbench to exit with status 0. Instead, errors during  
the run should result in exit status 2.  
  
Back-patch to v12 where pgbench started reporting exit status.  
  
Original complaint and patch by Hayato Kuroda.  
  
Author: Yugo Nagata, Fabien COELHO  
Reviewed-by: Kyotaro Horiguchi, Fujii Masao  
Discussion: https://postgr.es/m/TYCPR01MB5870057375ACA8A73099C649F5349@TYCPR01MB5870.jpnprd01.prod.outlook.com  

commit   : 2d8a8b18fe5a88146cb8baff591137fa724d86da    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 28 Sep 2021 17:34:31 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 28 Sep 2021 17:34:31 -0400    

It turns out that the instability complained of in commit d3c09b9b1  
has an embarrassingly simple explanation.  The test script waits for  
the standby to flush incoming WAL to disk, but it should wait for  
the WAL to be replayed, since we are testing for the effects of that  
to be visible.  
  
While at it, use wait_for_catchup instead of reinventing that logic,  
and adjust $Test::Builder::Level to improve future error reports.  
  
Back-patch to v12 where the necessary infrastructure came in  
(cf. aforesaid commit).  Also back-patch 7d1aa6bf1 so that the  
test will actually get run.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : b7647c4260c444142b7231a9bf3d313fc2d74dbc    
  
author   : Michael Paquier <[email protected]>    
date     : Sun, 26 Sep 2021 19:18:30 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sun, 26 Sep 2021 19:18:30 +0900    

Author: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 97c5651cea0f98f281131f3a1f970f875f5951a6    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 25 Sep 2021 10:53:54 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 25 Sep 2021 10:53:54 -0400    

Be a little more vocal about the risks of remote collations not  
matching local ones.  Actually fixing these risks seems hard,  
and I've given up on the idea that it might be back-patchable.  
So the best we can do for the back branches is add documentation.  
  
Per discussion of bug #16583 from Jiří Fejfar.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7b6ce36fbab522ced3860fa43fcb587efda29ac8    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Sat, 25 Sep 2021 11:27:28 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Sat, 25 Sep 2021 11:27:28 +0200    

OpenSSL 3 introduced the concept of providers to support modularization,  
and moved the outdated ciphers to the new legacy provider. In case it's  
not loaded in the users openssl.cnf file there will be a lot of regress  
test failures, so add alternative outputs covering those.  
  
Also document the need to load the legacy provider in order to use older  
ciphers with OpenSSL-enabled pgcrypto.  
  
This will be backpatched to all supported version once there is sufficient  
testing in the buildfarm of OpenSSL 3.  
  
Reviewed-by: Michael Paquier  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 00c72da4a22d9883b1e511ff140bd47cc75d536d    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Sat, 25 Sep 2021 11:27:20 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Sat, 25 Sep 2021 11:27:20 +0200    

The PX layer in pgcrypto is handling digest padding on its own uniformly  
for all backend implementations. Starting with OpenSSL 3.0.0, DecryptUpdate  
doesn't flush the last block in case padding is enabled so explicitly  
disable it as we don't use it.  
  
This will be backpatched to all supported version once there is sufficient  
testing in the buildfarm of OpenSSL 3.  
  
Reviewed-by: Peter Eisentraut, Michael Paquier  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 90cfd269f226ecf287df7b8df04f6a46515000d4    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Sat, 25 Sep 2021 11:25:48 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Sat, 25 Sep 2021 11:25:48 +0200    

This has previously not been a problem (that anyone ever reported),  
but in future OpenSSL versions (3.0.0), where legacy ciphers are/can  
be disabled, this is the place where this is reported.  So we need to  
catch the error here, otherwise the higher-level functions would  
return garbage.  The nearby encryption code already handled errors  
similarly.  
  
Author: Peter Eisentraut <[email protected]>  
Reviewed-by: Daniel Gustafsson <[email protected]>  
Discussion: https://www.postgresql.org/message-id/[email protected]  
Backpatch-through: 9.6  

commit   : 0872ccbfd8c988d69c79a93dd85f0e10f2ee6349    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 25 Sep 2021 15:12:04 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 25 Sep 2021 15:12:04 +0900    

Index vacuums may happen multiple times depending on the number of dead  
tuples stored, as of maintenance_work_mem for a manual VACUUM.  For  
autovacuum, this is controlled by autovacuum_work_mem instead, if set.  
The documentation mentioned the former, but not the latter in the  
context of autovacuum.  
  
Reported-by: Nikolai Berkoff  
Author: Laurenz Albe, Euler Taveira  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : c1ddcdb5c3a7687b71bcdd0b18b4ba5a2aa620b3    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 25 Sep 2021 14:48:17 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 25 Sep 2021 14:48:17 +0900    

Reported-by: rir  
Discussion: https://postgr.es/m/20210924183658.3syyitp3yuxjv2fp@localhost  
Backpatch-through: 9.6  

commit   : 4185632e93c1552c5e70279f2de394cd17700549    
  
author   : Tomas Vondra <[email protected]>    
date     : Tue, 21 Sep 2021 01:13:11 +0200    
  
committer: Tomas Vondra <[email protected]>    
date     : Tue, 21 Sep 2021 01:13:11 +0200    

Calculating degree of a functional dependency may allocate a lot of  
memory - we have released mot of the explicitly allocated memory, but  
e.g. detoasted varlena values were left behind. That may be an issue,  
because we consider a lot of dependencies (all combinations), and the  
detoasting may happen for each one again.  
  
Fixed by calling dependency_degree() in a dedicated context, and  
resetting it after each call. We only need the calculated dependency  
degree, so we don't need to copy anything.  
  
Backpatch to PostgreSQL 10, where extended statistics were introduced.  
  
Backpatch-through: 10  
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com  

commit   : 16d394c05065d93b2ceca5b9be8dfc608193043a    
  
author   : Tomas Vondra <[email protected]>    
date     : Tue, 21 Sep 2021 01:14:11 +0200    
  
committer: Tomas Vondra <[email protected]>    
date     : Tue, 21 Sep 2021 01:14:11 +0200    

Until now, all extended statistics on a given relation were built in the  
same memory context, without resetting. Some of the memory was released  
explicitly, but not all of it - for example memory allocated while  
detoasting values is hard to free. This is how it worked since extended  
statistics were introduced in PostgreSQL 10, but adding support for  
extended stats on expressions made the issue somewhat worse as it  
increases the number of statistics to build.  
  
Fixed by adding a memory context which gets reset after building each  
statistics object (all the statistics kinds included in it). Resetting  
it after building each statistics kind would be even better, but it  
would require more invasive changes and copying of results, making it  
harder to backpatch.  
  
Backpatch to PostgreSQL 10, where extended statistics were introduced.  
  
Author: Justin Pryzby  
Reported-by: Justin Pryzby  
Reviewed-by: Tomas Vondra  
Backpatch-through: 10  
Discussion: https://www.postgresql.org/message-id/20210915200928.GP831%40telsasoft.com  

commit   : 76001de031a562a0f350daae381196fa44576eb3    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 22 Sep 2021 08:43:20 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 22 Sep 2021 08:43:20 +0900    

Contrary to the output of native perl, Msys perl generates outputs with  
CRLFs characters.  There are already places in the TAP code where CRLFs  
(\r\n) are automatically converted to LF (\n) on Msys, but we missed a  
couple of places when running commands and using their output for  
comparison, that would lead to failures.  
  
This problem has been found thanks to the test added in 5adb067 using  
TestLib::command_checks_all(), but after a closer look more code paths  
were missing a filter.  
  
This is backpatched all the way down to prevent any surprises if a new  
test is introduced in stable branches.  
  
Reviewed-by: Andrew Dunstan, Álvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : e8b0bcae63ff3eed36353e2ec591cb7d2f683feb    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 21 Sep 2021 19:06:33 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 21 Sep 2021 19:06:33 -0400    

Before commit 84f5c2908, a STABLE function in a plpgsql CALL  
statement's argument list would see an up-to-date snapshot,  
because exec_stmt_call would push a new snapshot.  I got rid of  
that because the possibility of the snapshot disappearing within  
COMMIT made it too hard to manage a snapshot across the CALL  
statement.  That's fine so far as the procedure itself goes,  
but I forgot to think about the possibility of STABLE functions  
within the CALL argument list.  As things now stand, those'll  
be executed with the Portal's snapshot as ActiveSnapshot,  
keeping them from seeing updates more recent than Portal startup.  
  
(VOLATILE functions don't have a problem because they take their  
own snapshots; which indeed is also why the procedure itself  
doesn't have a problem.  There are no STABLE procedures.)  
  
We can fix this by pushing a new snapshot transiently within  
ExecuteCallStmt itself.  Popping the snapshot before we get  
into the procedure proper eliminates the management problem.  
The possibly-useless extra snapshot-grab is slightly annoying,  
but it's no worse than what happened before 84f5c2908.  
  
Per bug #17199 from Alexander Nawratil.  Back-patch to v11,  
like the previous patch.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : cd35d3909bf1665b688eac691a0f2087c55610de    
  
author   : Peter Geoghegan <[email protected]>    
date     : Mon, 20 Sep 2021 14:26:20 -0700    
  
committer: Peter Geoghegan <[email protected]>    
date     : Mon, 20 Sep 2021 14:26:20 -0700    

A broken HOT chain is not an unexpected condition, even when the offset  
number points past the end of the page's line pointer array.  
heap_prune_chain() does not (and never has) treated this condition as  
unexpected, so derivative code in heap_index_delete_tuples() shouldn't  
do so either.  
  
Oversight in commit 4228817449.  
  
The assertion can probably only fail on Postgres 14 and master.  Earlier  
releases don't have commit 3c3b8a4b, which taught VACUUM to truncate the  
line pointer array of heap pages.  Backpatch all the same, just to be  
consistent.  
  
Author: Peter Geoghegan <[email protected]>  
Reported-By: Alexander Lakhin <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch: 12-, just like commit 4228817449.  

commit   : f230614da28b1b60a4c8faf1fe0f51f866c3093c    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 20 Sep 2021 11:48:52 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 20 Sep 2021 11:48:52 -0400    

Casting a value that's already of a type with a specific typmod  
to an unspecified typmod doesn't do anything so far as run-time  
behavior is concerned.  However, it really ought to change the  
exposed type of the expression to match.  Up to now,  
coerce_type_typmod hasn't bothered with that, which creates gotchas  
in contexts such as recursive unions.  If for example one side of  
the union is numeric(18,3), but it needs to be plain numeric to  
match the other side, there's no direct way to express that.  
  
This is easy enough to fix, by inserting a RelabelType to update the  
exposed type of the expression.  However, it's a bit nervous-making  
to change this behavior, because it's stood for a really long time.  
But no complaints have emerged about 14beta3, so go ahead and  
back-patch.  
  
Back-patch of 5c056b0c2 into previous supported branches.  
  
Discussion: https://postgr.es/m/CABNQVagu3bZGqiTjb31a8D5Od3fUMs7Oh3gmZMQZVHZ=uWWWfQ@mail.gmail.com  
Discussion: https://postgr.es/m/[email protected]  

commit   : c9d07ee376b56be7bb6bcd9b948df8b87ea35289    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 19 Sep 2021 11:36:53 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 19 Sep 2021 11:36:53 -0400    

"PGcon" should be "PGconn".  Noted by D. Frey.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : febe013cad5992ba8994db309cd1b7d85062c6d3    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 17 Sep 2021 15:41:16 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 17 Sep 2021 15:41:16 -0400    

Commit 55dc86eca changed pull_varnos to use (if possible) the associated  
ph_eval_at for a PlaceHolderVar.  I missed a fine point though: we might  
be looking at a PHV in the quals or tlist of a child appendrel, in which  
case we need to compute a ph_eval_at value that's been translated in the  
same way that the PHV itself has been (cf. adjust_appendrel_attrs).  
Fortunately, enough info is available in the PlaceHolderInfo to make  
such translation possible without additional outside data, so we don't  
need another round of uglification of planner APIs.  This is a little  
bit complicated, but since it's a hard-to-hit corner case, I'm not much  
worried about adding cycles here.  
  
Per report from Jaime Casanova.  Back-patch to v12, like the previous  
commit.  
  
Discussion: https://postgr.es/m/20210915230959.GB17635@ahch-to  

commit   : 24c57aa62937d2dc8b5e91c1ec95e0f8966fb0c4    
  
author   : Fujii Masao <[email protected]>    
date     : Thu, 16 Sep 2021 13:07:29 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Thu, 16 Sep 2021 13:07:29 +0900    

ProcArrayGroupClearXid function has a parameter named "proc",  
but the same name was used for its local variables. This commit fixes  
this variable shadowing, to improve code readability.  
  
Back-patch to all supported versions, to make future back-patching  
easy though this patch is classified as refactoring only.  
  
Reported-by: Ranier Vilela  
Author: Ranier Vilela, Aleksander Alekseev  
https://postgr.es/m/CAEudQAqyoTZC670xWi6w-Oe2_Bk1bfu2JzXz6xRfiOUzm7xbyQ@mail.gmail.com  

commit   : 43849b65f3338acad50d9bdf607d2d573f8079c6    
  
author   : Andres Freund <[email protected]>    
date     : Mon, 13 Sep 2021 18:07:19 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Mon, 13 Sep 2021 18:07:19 -0700    

If an allocation failed within LLVM it is not safe to call back into LLVM as  
LLVM is not generally safe against exceptions / stack-unwinding. Thus errors  
while in LLVM code are promoted to FATAL. However llvm_shutdown() did call  
back into LLVM even in such cases, while llvm_release_context() was careful  
not to do so.  
  
We cannot generally skip shutting down LLVM, as that can break profiling. But  
it's OK to do so if there was an error from within LLVM.  
  
Reported-By: Jelte Fennema <[email protected]>  
Author: Andres Freund <[email protected]>  
Author: Justin Pryzby <[email protected]>  
Discussion: https://postgr.es/m/AM5PR83MB0178C52CCA0A8DEA0207DC14F7FF9@AM5PR83MB0178.EURPRD83.prod.outlook.com  
Backpatch: 11-, where jit was introduced  

commit   : b1de90699e73a081ac450420b7d4f3fd1d49981b    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 13 Sep 2021 12:42:03 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 13 Sep 2021 12:42:03 -0400    

Ordinarily, using EXIT this way would draw "control reached end of  
function without RETURN".  However, if the function is one where we  
don't require an explicit RETURN (such as a DO block), that should  
not happen.  It did anyway, because add_dummy_return() neglected to  
account for the case.  
  
Per report from Herwig Goemans.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 99a1c5d068d84f5097ab40842ecd766d69491d6c    
  
author   : Etsuro Fujita <[email protected]>    
date     : Mon, 13 Sep 2021 17:30:03 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Mon, 13 Sep 2021 17:30:03 +0900    

The type information for FDW options is only added to HEAD; remove this  
from back branches.  Oversight in commit aa769f80e.  
  
Apply the patch to v12, v13, and v14.  
  
Discussion: https://postgr.es/m/CAPmGK14z92twaKwRoccHbbh5Va5vbRDZcTYYTx50+0JTQ8xx_g@mail.gmail.com  

commit   : b34dcf87f685ab6c7705d4dd504f23025bfc75b0    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 13 Sep 2021 13:24:27 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 13 Sep 2021 13:24:27 +0900    

An out-of-memory failure happening when allocating the structures to  
store the connection parameter keywords and values would mess up with  
the set of connections saved, as on failure the pthread mutex would  
still be hold with the new connection object listed but free()'d.  
  
Rather than just unlocking the mutex, which would leave the static list  
of connections into an inconsistent state, move the allocation for the  
structures of the connection parameters before beginning the test  
manipulation.  This ensures that the list of connections and the  
connection mutex remain consistent all the time in this code path.  
  
This error is unlikely going to happen, but this could mess up badly  
with ECPG clients in surprising ways, so backpatch all the way down.  
  
Reported-by: ryancaicse  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 3adde7eb6633bea734c4513ebd96adca78b7737c    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 11 Sep 2021 15:19:31 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 11 Sep 2021 15:19:31 -0400    

If search_start is greater than the length of the string, we should just  
return REG_NOMATCH immediately.  (Note that the equality case should  
*not* be rejected, since the pattern might be able to match zero  
characters.)  This guards various internal assumptions that the min of a  
range of string positions is not more than the max.  Violation of those  
assumptions could allow an attempt to fetch string[search_start-1],  
possibly causing a crash.  
  
Jaime Casanova pointed out that this situation is reachable with the  
new regexp_xxx functions that accept a user-specified start position.  
I don't believe it's reachable via any in-core call site in v14 and  
below.  However, extensions could possibly call pg_regexec with an  
out-of-range search_start, so let's back-patch the fix anyway.  
  
Discussion: https://postgr.es/m/20210911180357.GA6870@ahch-to  

commit   : ba408fc960b6be83c8ac0d74f64c02116cd8bd4c    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 10 Sep 2021 13:18:32 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 10 Sep 2021 13:18:32 -0400    

We have long forbidden fetching backwards from a NO SCROLL cursor,  
but the prohibition didn't extend to cases in which we rewind the  
query altogether and then re-fetch forwards.  I think the reason is  
that this logic was mainly meant to protect plan nodes that can't  
be run in the reverse direction.  However, re-reading the query output  
is problematic if the query is volatile (which includes SELECT FOR  
UPDATE, not just queries with volatile functions): the re-read can  
produce different results, which confuses the cursor navigation logic  
completely.  Another reason for disliking this approach is that some  
code paths will either fetch backwards or rewind-and-fetch-forwards  
depending on the distance to the target row; so that seemingly  
identical use-cases may or may not draw the "cursor can only scan  
forward" error.  Hence, let's clean things up by disallowing rewind  
as well as fetch-backwards in a NO SCROLL cursor.  
  
Ordinarily we'd only make such a definitional change in HEAD, but  
there is a third reason to consider this change now.  Commit ba2c6d6ce  
created some new user-visible anomalies for non-scrollable cursors  
WITH HOLD, in that navigation in the cursor result got confused if the  
cursor had been partially read before committing.  The only good way  
to resolve those anomalies is to forbid rewinding such a cursor, which  
allows removal of the incorrect cursor state manipulations that  
ba2c6d6ce added to PersistHoldablePortal.  
  
To minimize the behavioral change in the back branches (including  
v14), refuse to rewind a NO SCROLL cursor only when it has a holdStore,  
ie has been held over from a previous transaction due to WITH HOLD.  
This should avoid breaking most applications that have been sloppy  
about whether to declare cursors as scrollable.  We'll enforce the  
prohibition across-the-board beginning in v15.  
  
Back-patch to v11, as ba2c6d6ce was.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 2e75e969c88f51a2dd571540252f6423c936d49c    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 9 Sep 2021 13:36:31 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 9 Sep 2021 13:36:31 -0400    

Some plan node types don't react well to being called again after  
they've already returned NULL.  PortalRunSelect() has long dealt  
with this by calling the executor with NoMovementScanDirection  
if it sees that we've already run the portal to the end.  However,  
commit ba2c6d6ce overlooked this point, so that persisting an  
already-fully-fetched cursor would fail if it had such a plan.  
  
Per report from Tomas Barton.  Back-patch to v11, as the faulty  
commit was.  (I've omitted a test case because the type of plan  
that causes a problem isn't all that stable.)  
  
Discussion: https://postgr.es/m/CAPV2KRjd=ErgVGbvO2Ty20tKTEZZr6cYsYLxgN_W3eAo9pf5sw@mail.gmail.com  

commit   : a7a73ce301715a713954278ae1de36aaf39221ab    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 9 Sep 2021 11:45:48 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 9 Sep 2021 11:45:48 -0400    

We don't allow relations to exceed 2^32-1 blocks, because block  
numbers are 32 bits and the last possible block number is reserved  
to mean InvalidBlockNumber.  There is a check for this in mdextend,  
but that's really way too late, because the smgr API requires us to  
create a buffer for the block-to-be-added, and we do not want to  
have any buffer with blocknum InvalidBlockNumber.  (Such a case  
can trigger assertions in bufmgr.c, plus I think it might confuse  
ReadBuffer's logic for data-past-EOF later on.)  So put the check  
into ReadBuffer.  
  
Per report from Christoph Berg.  It's been like this forever,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 4665352543d73a182a8fe638065430ef7db8b2b9    
  
author   : Fujii Masao <[email protected]>    
date     : Thu, 9 Sep 2021 23:58:54 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Thu, 9 Sep 2021 23:58:54 +0900    

Previously, walreceiver always closed the currently-opened WAL segment  
and created its archive notification file, after it finished writing  
the current segment up and received any WAL data that should be  
written into the next segment. If walreceiver exited just before  
any WAL data in the next segment arrived at standby, it did not  
create the archive notification file of the current segment  
even though that's known completed. This behavior could cause  
WAL archiving of the segment to be delayed until subsequent  
restartpoints or checkpoints created its notification file.  
  
To fix the issue, this commit changes walreceiver so that it creates  
an archive notification file of a current WAL segment immediately  
if that's known completed before receiving next WAL data.  
  
Back-patch to all supported branches.  
  
Reported-by: Kyotaro Horiguchi  
Author: Fujii Masao  
Reviewed-by: Kyotaro Horiguchi  
Discussion: https://postgr.es/m/[email protected]  

commit   : df290e5f38fbef3920d28dd817bbb3ff5cc8ad9e    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 8 Sep 2021 15:09:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 8 Sep 2021 15:09:42 -0400    

Coverity complained that one caller of getFormattedTypeName() failed  
to free the returned string.  Which is true, but rather than fixing  
that one, let's get rid of this tedious and error-prone requirement.  
Now that getFormattedTypeName() caches its result, strdup'ing that  
result and expecting the caller to free it accomplishes little except  
to waste cycles.  We do create a leak in the case where getTypes didn't  
make a TypeInfo for the type, but that basically shouldn't ever happen.  
  
Back-patch, as commit 6c450a861 was.  This isn't a particularly  
interesting bug fix, but the API change seems like a hazard for  
future back-patching activity if we don't back-patch it.  

commit   : 1fedbcc7ab26174686cbad7f8b836616c906d267    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 8 Sep 2021 12:05:43 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 8 Sep 2021 12:05:43 -0400    

If we copy data-modifying CTEs from the original query to a replacement  
query (from a DO INSTEAD rule), we must set hasModifyingCTE properly  
in the replacement query.  Failure to do this can cause various  
unpleasantness, such as unsafe usage of parallel plans.  The code also  
neglected to propagate hasRecursive, though that's only cosmetic at  
the moment.  
  
A difficulty arises if the rule action is an INSERT...SELECT.  We  
attach the original query's RTEs and CTEs to the sub-SELECT Query, but  
data-modifying CTEs are only allowed to appear in the topmost Query.  
For the moment, throw an error in such cases.  It would probably be  
possible to avoid this error by attaching the CTEs to the top INSERT  
Query instead; but that would require a bunch of new code to adjust  
ctelevelsup references.  Given the narrowness of the use-case, and  
the need to back-patch this fix, it does not seem worth the trouble  
for now.  We can revisit this if we get field complaints.  
  
Per report from Greg Nancarrow.  Back-patch to all supported branches.  
(The test case added here does not fail before v10, but there are  
plenty of places checking top-level hasModifyingCTE in 9.6, so I have  
no doubt that this code change is necessary there too.)  
  
Greg Nancarrow and Tom Lane  
  
Discussion: https://postgr.es/m/CAJcOf-f68DT=26YAMz_i0+Au3TcLO5oiHY5=fL6Sfuits6r+_w@mail.gmail.com  
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com  

commit   : 2eb09f27db6c5e2e017ac98f208d982486e0edc5    
  
author   : Amit Kapila <[email protected]>    
date     : Wed, 8 Sep 2021 10:39:13 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Wed, 8 Sep 2021 10:39:13 +0530    

Updates/Deletes on a relation were allowed even without replica identity  
after we define the publication for all tables. This would later lead to  
an error on subscribers. The reason was that for such publications we were  
not invalidating the relcache and the publication information for  
relations was not getting rebuilt. Similarly, we were not invalidating the  
relcache after dropping of such publications which will prohibit  
Updates/Deletes without replica identity even without any publication.  
  
Author: Vignesh C and Hou Zhijie  
Reviewed-by: Hou Zhijie, Kyotaro Horiguchi, Amit Kapila  
Backpatch-through: 10, where it was introduced  
Discussion: https://postgr.es/m/CALDaNm0pF6zeWqCA8TCe2sDuwFAy8fCqba=nHampCKag-qLixg@mail.gmail.com  

commit   : eb3c8d24808a4d3ee8a257f4b856d5583b5cfdaa    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Sep 2021 11:29:52 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Sep 2021 11:29:52 -0400    

timetz_zone() delivered completely wrong answers if the zone was  
specified by a dynamic TZ abbreviation, because it failed to account  
for the difference between the POSIX conventions for field values in  
struct pg_tm and the conventions used in PG-specific datetime code.  
  
As a stopgap fix, just adjust the tm_year and tm_mon fields to match  
PG conventions.  This is fixed in a different way in HEAD (388e71af8)  
but I don't want to back-patch the change of reference point.  
  
Discussion: https://postgr.es/m/CAJ7c6TOMG8zSNEZtCn5SPe+cCk3Lfxb71ZaQwT2F4T7PJ_t=KA@mail.gmail.com  

commit   : 60bf7e69b07851d28e59d3c19fcbd6893866a519    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 6 Sep 2021 09:41:03 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 6 Sep 2021 09:41:03 +0200    

Since ea53100d5 (PostgreSQL 12), the shipped pkg-config files have  
been broken for statically linking libpq because libpgcommon and  
libpgport are missing.  This patch adds those two missing private  
dependencies (in a non-hardcoded way).  
  
Reported-by: Filip Gospodinov <[email protected]>  
Discussion: https://www.postgresql.org/message-id/flat/[email protected]  

commit   : fd295d0c687eb8f2e9ec6d9bb1cedf0cc45e1dd8    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 4 Sep 2021 16:29:08 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 4 Sep 2021 16:29:08 -0400    

Attempting to make hashfloat4() look as much as possible like  
hashfloat8(), I'd figured I could replace NaNs with get_float4_nan()  
before widening to float8.  However, results from protosciurus  
and topminnow show that on some platforms that produces a different  
bit-pattern from get_float8_nan(), breaking the intent of ce773f230.  
Rearrange so that we use the result of get_float8_nan() for all NaN  
cases.  As before, back-patch.  

commit   : e456167a8af428dffb1621ae5c9e29147cb005f2    
  
author   : Alvaro Herrera <[email protected]>    
date     : Sat, 4 Sep 2021 12:14:30 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Sat, 4 Sep 2021 12:14:30 -0400    

This reverts commit 515e3d84a0b5 and equivalent commits in back  
branches.  This solution to the problem has a number of problems, so  
we'll try again with a different approach.  
  
Per note from Andres Freund  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 3b302eb1ea2e9cb610ccc3a4f819dddfc89418ca    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 3 Sep 2021 21:04:44 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 3 Sep 2021 21:04:44 -0400    

Replace fixed-length command buffers with psprintf() calls.  We didn't  
have anything as convenient as psprintf() when this code was written,  
but now that we do, there's little reason for the limitation to  
stand.  Removing it eliminates some corner cases where (for example)  
starting the postmaster with a whole lot of options fails.  
  
Most individual file names that pg_ctl deals with are still restricted  
to MAXPGPATH, but we've seldom had complaints about that limitation  
so long as it only applies to one filename.  
  
Back-patch to all supported branches.  
  
Phil Krylov  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 9046a05368de54d8b20a79ac3e42b363bac6ff17    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 3 Sep 2021 16:38:55 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 3 Sep 2021 16:38:55 -0400    

Previously this was allowed, but the collation effectively vanished  
into the ether because of the way lookup_collation() works: you could  
not use the collation, nor even drop it.  Seems better to give an  
error up front than to leave the user wondering why it doesn't work.  
  
(Because this test is in DefineCollation not CreateCollation, it does  
not prevent pg_import_system_collations from creating ICU collations,  
regardless of the initially-chosen encoding.)  
  
Per bug #17170 from Andrew Bille.  Back-patch to v10 where ICU support  
was added.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 1fab33c0b612c7d87b97842fe5eb07871ec2c222    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 3 Sep 2021 10:01:02 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 3 Sep 2021 10:01:02 -0400    

Modern POSIX seems to require strtod() to accept "-NaN", but there's  
nothing about NaN in SUSv2, and some of our oldest buildfarm members  
don't like it.  Let's try writing it as -'NaN' instead; that seems  
to produce the same result, at least on Intel hardware.  
  
Per buildfarm.  

commit   : a3bf136732b13cc3b8f143378e645263ea1aba5e    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 2 Sep 2021 17:24:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 2 Sep 2021 17:24:42 -0400    

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,  
of which at least two ("NaN" and "-NaN") are pretty easy to produce  
from SQL on most machines.  This is problematic because our btree  
comparison functions deem all NaNs to be equal, but our float hash  
functions know nothing about NaNs and will happily produce varying  
hash codes for them.  That causes unexpected results from queries  
that hash a column containing different NaN values.  It could also  
produce unexpected lookup failures when using a hash index on a  
float column, i.e. "WHERE x = 'NaN'" will not find all the rows  
it should.  
  
To fix, special-case NaN in the float hash functions, not too much  
unlike the existing special case that forces zero and minus zero  
to hash the same.  I arranged for the most vanilla sort of NaN  
(that coming from the C99 NAN constant) to still have the same  
hash code as before, to reduce the risk to existing hash indexes.  
  
I dithered about whether to back-patch this into stable branches,  
but ultimately decided to do so.  It's a clear improvement for  
queries that hash internally.  If there is anybody who has -NaN  
in a hash index, they'd be well advised to re-index after applying  
this patch ... but the misbehavior if they don't will not be much  
worse than the misbehavior they had before.  
  
Per bug #17172 from Ma Liangzhu.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 1ad56d595db6b57ff557d950f16bbe11260c7a1b    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 2 Sep 2021 11:36:06 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 2 Sep 2021 11:36:06 +0900    

This makes the documentation more accurate grammatically.  
  
Author: Elena Indrupskaya  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : df6ab94aca9015eebd0fc74da8b9d5cc179a0ced    
  
author   : Amit Kapila <[email protected]>    
date     : Wed, 1 Sep 2021 09:26:44 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Wed, 1 Sep 2021 09:26:44 +0530    

The check to test whether the subscription workers were restarting after a  
change in the subscription was failing. The reason was that the test was  
assuming the walsender started before it reaches the 'streaming' state and  
the walsender was exiting due to an error before that. Now, the walsender  
was erroring out before reaching the 'streaming' state because it tries to  
acquire the slot before the previous walsender has exited.  
  
In passing, improve the die messages so that it is easier to investigate  
the failures in the future if any.  
  
Reported-by: Michael Paquier, as per buildfarm  
Author: Ajin Cherian  
Reviewed-by: Masahiko Sawada, Amit Kapila  
Backpatch-through: 10, where this test was introduced  
Discussion: https://postgr.es/m/[email protected]  

commit   : 6b9667392d815caea3c3bd138d360f0d321f9c36    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 31 Aug 2021 15:04:05 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 31 Aug 2021 15:04:05 -0400    

For no particularly good reason, getPolicies() queried pg_policy  
separately for each table.  We can collect all the policies in  
a single query instead, and attach them to the correct TableInfo  
objects using findTableByOid() lookups.  On the regression  
database, this reduces the number of queries substantially, and  
provides a visible savings even when running against a local  
server.  
  
Per complaint from Hubert Depesz Lubaczewski.  Since this is such  
a simple fix and can have a visible performance benefit, back-patch  
to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 2f1ed9d98c38a62ffa4b0a6589c9fcc529ae0883    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 31 Aug 2021 13:53:33 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 31 Aug 2021 13:53:33 -0400    

There's long been a "TODO: there might be some value in caching  
the results" annotation on pg_dump's getFormattedTypeName function;  
but we hadn't gotten around to checking what it was costing us to  
repetitively look up type names.  It turns out that when dumping the  
current regression database, about 10% of the total number of queries  
issued are duplicative format_type() queries.  However, Hubert Depesz  
Lubaczewski reported a not-unusual case where these account for over  
half of the queries issued by pg_dump.  Individually these queries  
aren't expensive, but when network lag is a factor, they add up to a  
problem.  We can very easily add some caching to getFormattedTypeName  
to solve it.  
  
Since this is such a simple fix and can have a visible performance  
benefit, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 5f8dd5dc1701348bce659b9689d085e011399d61    
  
author   : Tomas Vondra <[email protected]>    
date     : Tue, 31 Aug 2021 19:21:29 +0200    
  
committer: Tomas Vondra <[email protected]>    
date     : Tue, 31 Aug 2021 19:21:29 +0200    

Commit 5be8ce82e8 added a new role to the stats_ext regression suite,  
but the role name did not start with regress_ causing failures when  
running with ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS. Fixed by  
renaming the role to start with the expected regress_ prefix.  
  
Backpatch-through: 10, same as the new regression test  
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com  

commit   : 6c8b98669d1479148f02375c68ff17772ce7f95b    
  
author   : Tomas Vondra <[email protected]>    
date     : Tue, 31 Aug 2021 18:03:05 +0200    
  
committer: Tomas Vondra <[email protected]>    
date     : Tue, 31 Aug 2021 18:03:05 +0200    

When an ownership check on extended statistics object failed, the code  
was calling aclcheck_error_type to report the failure, which is clearly  
wrong, resulting in cache lookup errors. Fix by calling aclcheck_error.  
  
This issue exists since the introduction of extended statistics, so  
backpatch all the way back to PostgreSQL 10. It went unnoticed because  
there were no tests triggering the error, so add one.  
  
Reported-by: Mark Dilger  
Backpatch-through: 10, where extended stats were introduced  
Discussion: https://postgr.es/m/1F238937-7CC2-4703-A1B1-6DC225B8978A%40enterprisedb.com  

commit   : fa8ae19bea7f03a304705aba72b312cc2cea0d75    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 30 Aug 2021 16:29:12 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 30 Aug 2021 16:29:12 -0400    

Most data-corruption reports mention the location of the problem, but  
this one failed to.  Add it.  
  
Backpatch all the way back.  In 12 and older, also assign the  
ERRCODE_DATA_CORRUPTED error code as was done in commit fd6ec93bf890 for  
13 and later.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a3f6088c02983b305659ce900506474c685e0dc9    
  
author   : Amit Kapila <[email protected]>    
date     : Mon, 30 Aug 2021 09:45:20 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Mon, 30 Aug 2021 09:45:20 +0530    

ERRCODE_CONFIGURATION_LIMIT_EXCEEDED was used for checksum failure, use  
ERRCODE_DATA_CORRUPTED instead.  
  
Reported-by: Tatsuhito Kasahara  
Author: Tatsuhito Kasahara  
Backpatch-through: 9.6, where it was introduced  
Discussion: https://postgr.es/m/CAP0=ZVLHtYffs8SOWcFJWrBGoRzT9QQbk+_aP+E5AHLNXiOorA@mail.gmail.com  

commit   : 300a79fc92802fb8347900f9df0e8e224eafedca    
  
author   : Alvaro Herrera <[email protected]>    
date     : Sat, 28 Aug 2021 11:45:47 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Sat, 28 Aug 2021 11:45:47 -0400    

Strictly speaking this isn't a bug, but since all references to catalog  
objects are schema-qualified, we might as well be consistent.  The  
omission first appeared in commit 1c5d9270e339, so backpatch to 12.  
  
Author: Justin Pryzby <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : a494f10232645c3456bee7f6fbea5546f8a8166d    
  
author   : Noah Misch <[email protected]>    
date     : Fri, 27 Aug 2021 23:33:23 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Fri, 27 Aug 2021 23:33:23 -0700    

If the system crashed between CREATE TABLESPACE and the next checkpoint,  
the result could be some files in the tablespace unexpectedly containing  
no rows.  Affected files would be those for which the system did not  
write WAL; see the wal_skip_threshold documentation.  Before v13, a  
different set of conditions governed the writing of WAL; see v12's  
<sect2 id="populate-pitr">.  (The v12 conditions were broader in some  
ways and narrower in others.)  Users may want to audit non-default  
tablespaces for unexpected short files.  The bug could have truncated an  
index without affecting the associated table, and reindexing the index  
would fix that particular problem.  
  
This fixes the bug by making create_tablespace_directories() more like  
TablespaceCreateDbspace().  create_tablespace_directories() was  
recursively removing tablespace contents, reasoning that WAL redo would  
recreate everything removed that way.  That assumption holds for other  
wal_level values.  Under wal_level=minimal, the old approach could  
delete files for which no other copy existed.  Back-patch to 9.6 (all  
supported versions).  
  
Reviewed by Robert Haas and Prabhat Sahu.  Reported by Robert Haas.  
  
Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com  

commit   : 187b5fea982e6fd56a0311ca9f899f792307e534    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Aug 2021 19:42:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Aug 2021 19:42:42 -0400    

Somehow, spgist overlooked the need to call pgstat_count_index_scan().  
Hence, pg_stat_all_indexes.idx_scan and equivalent columns never  
became nonzero for an SP-GiST index, although the related per-tuple  
counters worked fine.  
  
This fix works a bit differently from other index AMs, in that the  
counter increment occurs in spgrescan not spggettuple/spggetbitmap.  
It looks like this won't make the user-visible semantics noticeably  
different, so I won't go to the trouble of introducing an is-this-  
the-first-call flag just to make the counter bumps happen in the  
same places.  
  
Per bug #17163 from Christian Quest.  Back-patch to all supported  
versions.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 38ed1566ee3417319f39ae7fb8a3fb55d9b26eeb    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Fri, 27 Aug 2021 22:50:19 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Fri, 27 Aug 2021 22:50:19 +0200    

Author: Dave Cramer <[email protected]>  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/CADK3HHLZmqAQZ2ByPDQQ9yhGqax36kksq6sDkV0yYzsxw6ipvQ@mail.gmail.com  

commit   : f4b77e82ebf44f5cbe9b9ddda5688422853c75d3    
  
author   : Robert Haas <[email protected]>    
date     : Wed, 25 Aug 2021 08:32:04 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Wed, 25 Aug 2021 08:32:04 -0400    

Pengchengliu reported an assertion failure in a parallel woker while  
performing a parallel scan using an overflowed snapshot. The proximate  
cause is that TransactionXmin was set to an incorrect value.  The  
underlying cause is incorrect snapshot handling in parallel.c.  
  
In particular, InitializeParallelDSM() was unconditionally calling  
GetTransactionSnapshot(), because I (rhaas) mistakenly thought that  
was always retrieving an existing snapshot whereas, at isolation  
levels less than REPEATABLE READ, it's actually taking a new one. So  
instead do this only at higher isolation levels where there actually  
is a single snapshot for the whole transaction.  
  
By itself, this is not a sufficient fix, because we still need to  
guarantee that TransactionXmin gets set properly in the workers. The  
easiest way to do that seems to be to install the leader's active  
snapshot as the transaction snapshot if the leader did not serialize a  
transaction snapshot. This doesn't affect the results of future  
GetTrasnactionSnapshot() calls since those have to take a new snapshot  
anyway; what we care about is the side effect of setting TransactionXmin.  
  
Report by Pengchengliu. Patch by Greg Nancarrow, except for some comment  
text which I supplied.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : e35705f549f0bb8f0318cd8a08244ba40ba9bdd2    
  
author   : Amit Kapila <[email protected]>    
date     : Wed, 25 Aug 2021 09:32:56 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Wed, 25 Aug 2021 09:32:56 +0530    

Commit 325f2ec555 introduced pg_class.relwrite to skip operations on  
tables created as part of a heap rewrite during DDL. It links such  
transient heaps to the original relation OID via this new field in  
pg_class but forgot to do anything about toast tables. So, logical  
decoding was not able to skip operations on internally created toast  
tables. This leads to an error when we tried to decode the WAL for the  
next operation for which it appeared that there is a toast data where  
actually it didn't have any toast data.  
  
To fix this, we set pg_class.relwrite for internally created toast tables  
as well which allowed skipping operations on them during logical decoding.  
  
Author: Bertrand Drouvot  
Reviewed-by: David Zhang, Amit Kapila  
Backpatch-through: 11, where it was introduced  
Discussion: https://postgr.es/m/[email protected]  

commit   : f53ceaea8a1da638a5c68a09676c67e90b7de870    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 25 Aug 2021 11:46:25 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 25 Aug 2021 11:46:25 +0900    

There are two identical error messages about valid value of modulus for  
hash partition, in PostgreSQL source code. Commit 0e1275fb07 improved  
only one of them so that ambiguous word "positive" was avoided there,  
and forgot to improve the other. This commit improves the other.  
Which would reduce translator burden.  
  
Back-pach to v11 where the error message exists.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

commit   : 69b93a0127bc0ccfd7458c32174f5174bcf8524d    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 25 Aug 2021 11:43:56 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 25 Aug 2021 11:43:56 +0900    

The distance in phrase operator must be an integer value between zero  
and MAXENTRYPOS inclusive. But previously the error message about  
its valid value included the information about its upper limit  
but not lower limit (i.e., zero). This commit improves the error message  
so that it also includes the information about its lower limit.  
  
Back-patch to v9.6 where full-text phrase search was supported.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

commit   : 92620e82f6a14d5078c5004feb86d0c99a528626    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 24 Aug 2021 16:37:27 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 24 Aug 2021 16:37:27 -0400    

Regexps like "(.){0}...\1" drew an "invalid backreference number".  
That's not unreasonable on its face, since the capture group will  
never be matched if it's iterated zero times.  However, other engines  
such as Perl's don't complain about this, nor do we throw an error for  
related cases such as "(.)|\1", even though that backref can never  
succeed either.  Also, if the zero-iterations case happens at runtime  
rather than compile time --- say, "(x)*...\1" when there's no "x" to  
be found --- that's not an error, we just deem the backref to not  
match.  Making this even less defensible, no error was thrown for  
nested cases such as "((.)){0}...\2"; and to add insult to injury,  
those cases could result in assertion failures instead.  (It seems  
that nothing especially bad happened in non-assert builds, though.)  
  
Let's just fix it so that no error is thrown and instead the backref  
is deemed to never match, so that compile-time detection of no  
iterations behaves the same as run-time detection.  
  
Per report from Mark Dilger.  This appears to be an aboriginal error  
in Spencer's library, so back-patch to all supported versions.  
  
Pre-v14, it turns out to also be necessary to back-patch one aspect of  
commits cb76fbd7e/00116dee5, namely to create capture-node subREs with  
the begin/end states of their subexpressions, not the current lp/rp  
of the outer parseqatom invocation.  Otherwise delsub complains that  
we're trying to disconnect a state from itself.  This is a bit scary  
but code examination shows that it's safe: in the pre-v14 code, if we  
want to wrap iteration around the subexpression, the first thing we do  
is overwrite the atom's begin/end fields with new states.  So the  
bogus values didn't survive long enough to be used for anything, except  
if no iteration is required, in which case it doesn't matter.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : b9521a1f97e18d4098ce34fd711d24d2064f41c0    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Aug 2021 17:41:07 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Aug 2021 17:41:07 -0400    

The recursion in cdissect() was careless about clearing match data  
for capturing parentheses after rejecting a partial match.  This  
could allow a later back-reference to succeed when by rights it  
should fail for lack of a defined referent.  
  
To fix, think a little more rigorously about what the contract  
between different levels of cdissect's recursion needs to be.  
With the right spec, we can fix this using fewer rather than more  
resets of the match data; the key decision being that a failed  
sub-match is now explicitly responsible for clearing any matches  
it may have set.  
  
There are enough other cross-checks and optimizations in the code  
that it's not especially easy to exhibit this problem; usually, the  
match will fail as-expected.  Plus, regexps that are even potentially  
vulnerable are most likely user errors, since there's just not much  
point in writing a back-ref that doesn't always have a referent.  
These facts perhaps explain why the issue hasn't been detected,  
even though it's almost certainly a couple of decades old.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 5065aeafb0b7593c04d3bc5bc2a86037f32143fc    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 23 Aug 2021 15:50:35 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 23 Aug 2021 15:50:35 -0400    

WAL records may span multiple segments, but XLogWrite() does not  
wait for the entire record to be written out to disk before  
creating archive status files.  Instead, as soon as the last WAL page of  
the segment is written, the archive status file is created, and the  
archiver may process it.  If PostgreSQL crashes before it is able to  
write and flush the rest of the record (in the next WAL segment), the  
wrong version of the first segment file lingers in the archive, which  
causes operations such as point-in-time restores to fail.  
  
To fix this, keep track of records that span across segments and ensure  
that segments are only marked ready-for-archival once such records have  
been completely written to disk.  
  
This has always been wrong, so backpatch all the way back.  
  
Author: Nathan Bossart <[email protected]>  
Reviewed-by: Kyotaro Horiguchi <[email protected]>  
Reviewed-by: Ryo Matsumura <[email protected]>  
Reviewed-by: Andrey Borodin <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : adbfde3db9d53a9e32ad38337fa3682b9f3183bf    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 20 Aug 2021 14:19:04 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 20 Aug 2021 14:19:04 -0400    

After detecting a sub-match "dissect" failure (i.e., a backref match  
failure) in the i'th sub-match of an iteration node, we should proceed  
by adjusting the attempted length of the i'th submatch.  As coded,  
though, these functions changed the attempted length of the *last*  
sub-match, and only after exhausting all possibilities for that would  
they back up to adjust the next-to-last sub-match, and then the  
second-from-last, etc; all of which is wasted effort, since only  
changing the start or length of the i'th sub-match can possibly make  
it succeed.  This oversight creates the possibility for exponentially  
bad performance.  Fortunately the problem is masked in most cases by  
optimizations or constraints applied elsewhere; which explains why  
we'd not noticed it before.  But it is possible to reach the problem  
with fairly simple, if contrived, regexps.  
  
Oversight in my commit 173e29aa5.  That's pretty ancient now,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0c13ee198f0d690765550fc1c694f9191e9785d6    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 19 Aug 2021 12:12:35 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 19 Aug 2021 12:12:35 -0400    

transformLockingClause neglected to exclude the pseudo-RTEs for  
OLD/NEW when processing a rule's query.  This led to odd errors  
or even crashes later on.  This bug is very ancient, but it's  
not terribly surprising that nobody noticed, since the use-case  
for SELECT FOR UPDATE in a non-view rule is somewhere between  
thin and non-existent.  Still, crashing is not OK.  
  
Per bug #17151 from Zhiyong Wu.  Thanks to Masahiko Sawada  
for analysis of the problem.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : eb2f59b34e256a34f65518d01ebe7c282052ae4e    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 18 Aug 2021 18:12:51 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 18 Aug 2021 18:12:51 -0400    

Recursion into the FILTER clause was mis-implemented, such that a  
relevant Var or Aggref at the very top of the FILTER clause would  
be ignored.  (Of course, that'd have to be a plain boolean Var or  
boolean-returning aggregate.)  The consequence would be  
mis-identification of the correct semantic level of the aggregate,  
which could lead to not-per-spec query behavior.  If the FILTER  
expression is an aggregate, this could also lead to failure to issue  
an expected "aggregate function calls cannot be nested" error, which  
would likely result in a core dump later on, since the planner and  
executor aren't expecting such cases to appear.  
  
The root cause is that commit b560ec1b0 blindly copied some code  
that assumed it's recursing into a List, and thus didn't examine the  
top-level node.  To forestall questions about why this call doesn't  
look like the others, as well as possible future copy-and-paste  
mistakes, let's change all three check_agg_arguments_walker calls in  
check_agg_arguments, even though only the one for the filter clause  
is really broken.  
  
Per bug #17152 from Zhiyong Wu.  This has been wrong since we  
implemented FILTER, so back-patch to all supported versions.  
(Testing suggests that pre-v11 branches manage to avoid crashing  
in the bad-Aggref case, thanks to "redundant" checks in ExecInitAgg.  
But I'm not sure how thorough that protection is, and anyway the  
wrong-behavior issue remains, so fix 9.6 and 10 too.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ed209db778b49ac42957cfd074410e342e450131    
  
author   : Daniel Gustafsson <[email protected]>    
date     : Tue, 17 Aug 2021 14:27:37 +0200    
  
committer: Daniel Gustafsson <[email protected]>    
date     : Tue, 17 Aug 2021 14:27:37 +0200    

In OpenSSL there are two types of BIO's (I/O abstractions):  
source/sink and filters. A source/sink BIO is a source and/or  
sink of data, ie one acting on a socket or a file. A filter  
BIO takes a stream of input from another BIO and transforms it.  
In order for BIO_find_type() to be able to traverse the chain  
of BIO's and correctly find all BIO's of a certain type they  
shall have the type bit set accordingly, source/sink BIO's  
(what PostgreSQL implements) use BIO_TYPE_SOURCE_SINK and  
filter BIO's use BIO_TYPE_FILTER. In addition to these, file  
descriptor based BIO's should have the descriptor bit set,  
BIO_TYPE_DESCRIPTOR.  
  
The PostgreSQL implementation didn't set the type bits, which  
went unnoticed for a long time as it's only really relevant  
for code auditing the OpenSSL installation, or doing similar  
tasks. It is required by the API though, so this fixes it.  
  
Backpatch through 9.6 as this has been wrong for a long time.  
  
Author: Itamar Gafni  
Discussion: https://postgr.es/m/SN6PR06MB39665EC10C34BB20956AE4578AF39@SN6PR06MB3966.namprd06.prod.outlook.com  
Backpatch-through: 9.6  

commit   : 9454265eb26ac87e71f4af5853281fa166f55b79    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Tue, 17 Aug 2021 10:00:06 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Tue, 17 Aug 2021 10:00:06 +0300    

The backslash sequences, including \123 and \x12 escapes, are interpreted  
after encoding conversion. The docs failed to mention that.  
  
Backpatch to all supported versions.  
  
Reported-by: Andreas Grob  
Discussion: https://www.postgresql.org/message-id/17142-9181542ca1df75ab%40postgresql.org  

commit   : 84c1bac579db605e6a7d7b15f7f641c583d7e6e9    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 16 Aug 2021 12:11:56 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 16 Aug 2021 12:11:56 +0900    

This commit ensures that the wait interval in the replay delay loop  
waiting for an amount of time defined by recovery_min_apply_delay is  
correctly handled on reload, recalculating the delay if this GUC value  
is updated, based on the timestamp of the commit record being replayed.  
  
The previous behavior would be problematic for example with replay  
still waiting even if the delay got reduced or just cancelled.  If the  
apply delay was increased to a larger value, the wait would have just  
respected the old value set, finishing earlier.  
  
Author: Soumyadeep Chakraborty, Ashwin Agrawal  
Reviewed-by: Kyotaro Horiguchi, Michael Paquier  
Discussion: https://postgr.es/m/CAE-ML+93zfr-HLN8OuxF0BjpWJ17O5dv1eMvSE5jsj9jpnAXZA@mail.gmail.com  
Backpatch-through: 9.6  

commit   : cdda2b247dd5c5b7bdb6cc97ff439fea716b829e    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 13 Aug 2021 13:58:47 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 13 Aug 2021 13:58:47 -0400    

Like the ARM case, just use gcc's __sync_lock_test_and_set();  
that will compile into AMOSWAP.W.AQ which does what we need.  
  
At some point it might be worth doing some work on atomic ops  
for RISC-V, but this should be enough for a creditable port.  
  
Back-patch to all supported branches, just in case somebody  
wants to try them on RISC-V.  
  
Marek Szuba  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 75d8fe8181e6979b96a5cfa9020cf7ef6dc5e6d0    
  
author   : David Rowley <[email protected]>    
date     : Fri, 13 Aug 2021 16:43:13 +1200    
  
committer: David Rowley <[email protected]>    
date     : Fri, 13 Aug 2021 16:43:13 +1200    

This fixes a bug in simplehash.h which caused an incorrect size mask to be  
used when the hash table grew to SH_MAX_SIZE (2^32).  The code was  
incorrectly setting the size mask to 0 when the hash tables reached the  
maximum possible number of buckets.  This would result always trying to  
use the 0th bucket causing an  infinite loop of trying to grow the hash  
table due to there being too many collisions.  
  
Seemingly it's not that common for simplehash tables to ever grow this big  
as this bug dates back to v10 and nobody seems to have noticed it before.  
However, probably the most likely place that people would notice it would  
be doing a large in-memory Hash Aggregate with something close to at least  
2^31 groups.  
  
After this fix, the code now works correctly with up to within 98% of 2^32  
groups and will fail with the following error when trying to insert any  
more items into the hash table:  
  
ERROR:  hash table size exceeded  
  
However, the work_mem (or hash_mem_multiplier in newer versions) settings  
will generally cause Hash Aggregates to spill to disk long before reaching  
that many groups.  The minimal test case I did took a work_mem setting of  
over 192GB to hit the bug.  
  
simplehash hash tables are used in a few other places such as Bitmap Index  
Scans, however, again the size that the hash table can become there is  
also limited to work_mem and it would take a relation of around 16TB  
(2^31) pages and a very large work_mem setting to hit this.  With smaller  
work_mem values the table would become lossy and never grow large enough  
to hit the problem.  
  
Author: Yura Sokolov  
Reviewed-by: David Rowley, Ranier Vilela  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 10, where simplehash.h was added  

commit   : a8096e30f4d08f2b90cc147b44b414191de6155f    
  
author   : Thomas Munro <[email protected]>    
date     : Fri, 13 Aug 2021 10:38:22 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Fri, 13 Aug 2021 10:38:22 +1200    

It's hard to disable ASLR on current macOS releases, for testing with  
-DEXEC_BACKEND.  You could already set the environment variable  
PG_SHMEM_ADDR to something not likely to collide with mappings created  
earlier in process startup.  Let's also provide a default value that  
works on current releases and architectures, for developer convenience.  
  
As noted in the pre-existing comment, this is a horrible hack, but  
-DEXEC_BACKEND is only used by Unix-based PostgreSQL developers for  
testing some otherwise Windows-only code paths, so it seems excusable.  
  
Back-patch to all supported branches.  
  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/20210806032944.m4tz7j2w47mant26%40alap3.anarazel.de  

commit   : cd7d9b6b63f1ed1d60cab3888eb0fc2523de6f0e    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 10 Aug 2021 18:10:30 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 10 Aug 2021 18:10:30 -0400    

As a result of confusion about whether the "char" type is signed or  
unsigned, scans for index searches like "col < 'x'" or "col <= 'x'"  
would start at the middle of the index not the left end, thus missing  
many or all of the entries they should find.  Fortunately, this  
is not a symptom of index corruption.  It's only the search logic  
that is broken, and we can fix it without unpleasant side-effects.  
  
Per report from Jason Kim.  This has been wrong since btree_gin's  
beginning, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]