PostgreSQL 13.3 commit log

commit   : 272d82ec6febb97ab25fd7c67e9c84f4660b16ac    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 16:41:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 16:41:42 -0400    

commit   : 9b93a33f455579ee8953c32721c03ff9163b7f96    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 13:10:29 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 13:10:29 -0400    

Security: CVE-2021-32027, CVE-2021-32028, CVE-2021-32029  

commit   : 4a8656a7ee0c155b0249376af58eb3fc3a90415f    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 11:02:29 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 11:02:29 -0400    

It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE  
list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present.  
If it happens, the ON CONFLICT UPDATE code path would end up storing  
tuples that include the values of the extra resjunk columns.  That's  
fairly harmless in the short run, but if new columns are added to  
the table then the values would become accessible, possibly leading  
to malfunctions if they don't match the datatypes of the new columns.  
  
This had escaped notice through a confluence of missing sanity checks,  
including  
  
* There's no cross-check that a tuple presented to heap_insert or  
heap_update matches the table rowtype.  While it's difficult to  
check that fully at reasonable cost, we can easily add assertions  
that there aren't too many columns.  
  
* The output-column-assignment cases in execExprInterp.c lacked  
any sanity checks on the output column numbers, which seems like  
an oversight considering there are plenty of assertion checks on  
input column numbers.  Add assertions there too.  
  
* We failed to apply nodeModifyTable's ExecCheckPlanOutput() to  
the ON CONFLICT UPDATE tlist.  That wouldn't have caught this  
specific error, since that function is chartered to ignore resjunk  
columns; but it sure seems like a bad omission now that we've seen  
this bug.  
  
In HEAD, the right way to fix this is to make the processing of  
ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists  
now do, that is don't add "SET x = x" entries, and use  
ExecBuildUpdateProjection to evaluate the tlist and combine it with  
old values of the not-set columns.  This adds a little complication  
to ExecBuildUpdateProjection, but allows removal of a comparable  
amount of now-dead code from the planner.  
  
In the back branches, the most expedient solution seems to be to  
(a) use an output slot for the ON CONFLICT UPDATE projection that  
actually matches the target table, and then (b) invent a variant of  
ExecBuildProjectionInfo that can be told to not store values resulting  
from resjunk columns, so it doesn't try to store into nonexistent  
columns of the output slot.  (We can't simply ignore the resjunk columns  
altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.)  
This works back to v10.  In 9.6, projections work much differently and  
we can't cheaply give them such an option.  The 9.6 version of this  
patch works by inserting a JunkFilter when it's necessary to get rid  
of resjunk columns.  
  
In addition, v11 and up have the reverse problem when trying to  
perform ON CONFLICT UPDATE on a partitioned table.  Through a  
further oversight, adjust_partition_tlist() discarded resjunk columns  
when re-ordering the ON CONFLICT UPDATE tlist to match a partition.  
This accidentally prevented the storing-bogus-tuples problem, but  
at the cost that MULTIEXPR_SUBLINK cases didn't work, typically  
crashing if more than one row has to be updated.  Fix by preserving  
resjunk columns in that routine.  (I failed to resist the temptation  
to add more assertions there too, and to do some minor code  
beautification.)  
  
Per report from Andres Freund.  Back-patch to all supported branches.  
  
Security: CVE-2021-32028  

commit   : 467395bfdf33f1ccf67ca388ffdcc927271544cb    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 10:44:38 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 10:44:38 -0400    

While we were (mostly) careful about ensuring that the dimensions of  
arrays aren't large enough to cause integer overflow, the lower bound  
values were generally not checked.  This allows situations where  
lower_bound + dimension overflows an integer.  It seems that that's  
harmless so far as array reading is concerned, except that array  
elements with subscripts notionally exceeding INT_MAX are inaccessible.  
However, it confuses various array-assignment logic, resulting in a  
potential for memory stomps.  
  
Fix by adding checks that array lower bounds aren't large enough to  
cause lower_bound + dimension to overflow.  (Note: this results in  
disallowing cases where the last subscript position would be exactly  
INT_MAX.  In principle we could probably allow that, but there's a lot  
of code that computes lower_bound + dimension and would need adjustment.  
It seems doubtful that it's worth the trouble/risk to allow it.)  
  
Somewhat independently of that, array_set_element() was careless  
about possible overflow when checking the subscript of a fixed-length  
array, creating a different route to memory stomps.  Fix that too.  
  
Security: CVE-2021-32027  

commit   : 4c7ba553bad0254aaaf525668cc1cdb2eb8fcc92    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 10 May 2021 14:32:18 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 10 May 2021 14:32:18 +0200    

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 832086c7a50768dd7a8c548ab063037741530ddf  

commit   : 0d204a4b09f74f2c2ca0d099d5a44ce422d6978d    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 10 May 2021 11:36:26 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 10 May 2021 11:36:26 +0200    

When building without --enable-dtrace, emit dummy  
  
    do {} while (0)  
  
statements for the stubbed-out TRACE_POSTGRESQL_foo() macros  
instead of empty macros that totally elide the original probe  
statement.  
  
This fixes the  
  
    warning: suggest braces around empty body in an ‘if’ statement [-Wempty-body]  
  
introduced by b94409a02f.  
  
Author: Craig Ringer <[email protected]>  
Discussion: https://www.postgresql.org/message-id/flat/20210504221531.cfvpmmdfsou6eitb%40alap3.anarazel.de  

commit   : 55fe672a92e3fab20d5126c05c35dfbc8d2b94a8    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 9 May 2021 13:31:40 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 9 May 2021 13:31:40 -0400    

commit   : 7f4bab7f4a0e42ee9fa14707f726017b7869386b    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 7 May 2021 12:19:30 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 7 May 2021 12:19:30 -0400    

As usual, the release notes for older branches will be made by cutting  
these down, but put them up for community review first.  

commit   : ef70b6f8e28dae452d2bdb379204db234c930d26    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 7 May 2021 11:46:37 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 7 May 2021 11:46:37 -0400    

This patch replaces use of the global "wrconn" variable in  
AlterSubscription_refresh with a local variable of the same name, making  
it consistent with other functions in subscriptioncmds.c (e.g.  
DropSubscription).  
  
The global wrconn is only meant to be used for logical apply/tablesync worker.  
Abusing it this way is known to cause trouble if an apply worker  
manages to do a subscription refresh, such as reported by Jeremy Finzel  
and diagnosed by Andres Freund back in November 2020, at  
https://www.postgresql.org/message-id/[email protected]  
  
Backpatch to 10.  In branch master, also move the connection establishment  
to occur outside the PG_TRY block; this way we can remove a test for NULL in  
PG_FINALLY, and it also makes the code more consistent with similar code in  
the same file.  
  
Author: Peter Smith <[email protected]>  
Reviewed-by: Bharath Rupireddy <[email protected]>  
Reviewed-by: Japin Li <[email protected]>  
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com  

commit   : f518c3df4eb0acc9ad6cb49f98c56f392b76c9c1    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 6 May 2021 17:17:56 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 6 May 2021 17:17:56 -0400    

Backpatch all the way back to 9.6.  
  
Author: Simon Riggs <[email protected]>  
Discussion: https://postgr.es/m/CANbhV-EwxvdhHuOLdfG2ciYrHOHXV=mm6=fD5aMhqcH09Li3Tg@mail.gmail.com  

commit   : 082d9d6b576b5bf9f1292dc7150167acfd5e94a5    
  
author   : Andres Freund <[email protected]>    
date     : Wed, 5 May 2021 22:07:40 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Wed, 5 May 2021 22:07:40 -0700    

Reported-By: Erik Rijkers <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  
Backpatch: 13, where b059d2f45685 introduced the issue.  

commit   : 923c13520f1b01a5563ec272de5a0a59b343f892    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 5 May 2021 12:14:21 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 5 May 2021 12:14:21 -0400    

When ALTER TABLE .. ALTER CONSTRAINT changes deferrability properties  
changed in a partitioned table, we failed to propagate those changes  
correctly to partitions and to triggers.  Repair by adding a recursion  
mechanism to affect all derived constraints and all derived triggers.  
(In particular, recurse to partitions even if their respective parents  
are already in the desired state: it is possible for the partitions to  
have been altered individually.)  Because foreign keys involve tables in  
two sides, we cannot use the standard ALTER TABLE recursion mechanism,  
so we invent our own by following pg_constraint.conparentid down.  
  
When ALTER TABLE .. ALTER CONSTRAINT is invoked on the derived  
pg_constraint object that's automaticaly created in a partition as a  
result of a constraint added to its parent, raise an error instead of  
pretending to work and then failing to modify all the affected triggers.  
Before this commit such a command would be allowed but failed to affect  
all triggers, so it would silently misbehave.  (Restoring dumps of  
existing databases is not affected, because pg_dump does not produce  
anything for such a derived constraint anyway.)  
  
Add some tests for the case.  
  
Backpatch to 11, where foreign key support was added to partitioned  
tables by commit 3de241dba86f.  (A related change is commit f56f8f8da6af  
in pg12 which added support for FKs *referencing* partitioned tables;  
this is what forces us to use an ad-hoc recursion mechanism for this.)  
  
Diagnosed by Tom Lane from bug report from Ron L Johnson.  As of this  
writing, no reviews were offered.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

commit   : 91a6b3862fef95c3075d9bea945b6d974433389f    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 4 May 2021 10:09:11 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 4 May 2021 10:09:11 -0400    

The OID of the constraint is used instead of the OID of the trigger --  
an easy mistake to make.  Apparently the object-alter hooks are not very  
well tested :-(  
  
Backpatch to 12, where this typo was introduced by 578b229718e8  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a6a3a274abd1368a0104fe2cfc7227c9a7e95970    
  
author   : Peter Eisentraut <[email protected]>    
date     : Tue, 4 May 2021 14:03:54 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Tue, 4 May 2021 14:03:54 +0200    

The previous fix for dumping of inherited generated columns  
(0bf83648a52df96f7c8677edbbdf141bfa0cf32b) must not be applied to  
partitions, since, unlike normal inherited tables, they are always  
dumped separately and reattached.  
  
Reported-by: Santosh Udupi <[email protected]>  
Discussion: https://www.postgresql.org/message-id/flat/CACLRvHZ4a-%2BSM_159%2BtcrHdEqxFrG%3DW4gwTRnwf7Oj0UNj5R2A%40mail.gmail.com  

commit   : 64190d65f2995203df401738b77adc5ebd4d2fce    
  
author   : Peter Eisentraut <[email protected]>    
date     : Tue, 4 May 2021 11:45:37 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Tue, 4 May 2021 11:45:37 +0200    

When running ALTER TABLE t2 INHERIT t1, we must check that columns in  
t2 that correspond to a generated column in t1 are also generated and  
have the same generation expression.  Otherwise, this would allow  
creating setups that a normal CREATE TABLE sequence would not allow.  
  
Discussion: https://www.postgresql.org/message-id/[email protected]  

commit   : e48ce7ef0ef8de3d8e9e56be4d23d7830681b18b    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 3 May 2021 12:11:33 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 3 May 2021 12:11:33 +0200    

If dtrace is compiled in but disabled, the lwlock dtrace probes still  
evaluate their arguments.  Since PostgreSQL 13, T_NAME(lock) does  
nontrivial work, so it should be avoided if not needed.  To fix, make  
these calls conditional on the *_ENABLED() macro corresponding to each  
probe.  
  
Reviewed-by: Craig Ringer <[email protected]>  
Discussion: https://www.postgresql.org/message-id/CAGRY4nwxKUS_RvXFW-ugrZBYxPFFM5kjwKT5O+0+Stuga5b4+Q@mail.gmail.com  

commit   : 3eeadc425ccf30d5738bb1204adb2e50a8bea8c1    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:37:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:37:56 -0400    

While we've always allowed such cases, the documentation didn't  
say you could do it.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7c810bd028a7be3ce32beb438db072bd02592869    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:10:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:10:06 -0400    

Mention specifically that you can't call aggregates, window functions,  
or procedures this way (the inability to call SRFs was already  
mentioned).  
  
Also, the claim that PQfn doesn't support NULL arguments or results  
has been a lie since we invented protocol 3.0.  Not sure why this  
text was never updated for that, but do it now.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 4d225ba0e6da73ddcbf207d2d91fc26537cfc5e1    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 14:10:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 14:10:26 -0400    

Reject aggregates, window functions, and procedures.  Aggregates  
failed anyway, though with a somewhat obscure error message.  
Window functions would hit an Assert or null-pointer dereference.  
Procedures seemed to work as long as you didn't try to do  
transaction control, but (a) transaction control is sort of the  
point of a procedure, and (b) it's not entirely clear that no  
bugs lurk in that path.  Given the lack of testing of this area,  
it seems safest to be conservative in what we support.  
  
Also reject proretset functions, as the fastpath protocol can't  
support returning a set.  
  
Also remove an easily-triggered assertion that the given OID  
isn't 0; the subsequent lookups can handle that case themselves.  
  
Per report from Theodor-Arsenij Larionov-Trichkin.  
Back-patch to all supported branches.  (The procedure angle  
only applies in v11+, of course.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : bbcfee0e56a2a563a4907b116da72081a408eadf    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 29 Apr 2021 15:24:37 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 29 Apr 2021 15:24:37 -0400    

Commits 29aeda6e4 et al closed up some oversights involving not checking  
for non-upgradable types within container types, such as arrays and  
ranges.  However, I only looked at version.c, failing to notice that  
there were substantially-equivalent tests in check.c.  (The division  
of responsibility between those files is less than clear...)  
  
In addition, because genbki.pl does not guarantee that auto-generated  
rowtype OIDs will hold still across versions, we need to consider that  
the composite type associated with a system catalog or view is  
non-upgradable.  It seems unlikely that someone would have a user  
column declared that way, but if they did, trying to read it in another  
PG version would likely draw "no such pg_type OID" failures, thanks  
to the type OID embedded in composite Datums.  
  
To support the composite and reg*-type cases, extend the recursive  
query that does the search to allow any base query that returns  
a column of pg_type OIDs, rather than limiting it to exactly one  
starting type.  
  
As before, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 896cedc7d578b9fd9c966f87a32ec628ee6c01aa    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 29 Apr 2021 11:31:24 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 29 Apr 2021 11:31:24 -0400    

Backpatch to 12, where 87259588d0ab introduced the current behavior.  
  
Per note from Justin Pryzby.  
  
Co-authored-by: Justin Pryzby <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7bbcfb4d584d0eac4a1f68af09f4e15334236e13    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 28 Apr 2021 10:03:28 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 28 Apr 2021 10:03:28 -0400    

Somehow I'd convinced myself that rotating to UTC-12 was the way  
to do this, but upon further review, it's definitely UTC+12.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a928297bc6f135f299ded2888283e73a35f4ade8    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:58:43 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:58:43 +0900    

Spotted by buildfarm member prion, with -DRELCACHE_FORCE_RELEASE.  
  
Introduced in f7aab36.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : f3c4537876843bea9a9d35d9643d347750961848    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:18:17 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:18:17 +0900    

Attempting to use this function with event triggers failed, as, since  
its introduction in a676201, this code has never associated an object  
name with event triggers.  This addresses the failure by adding the  
event trigger name to the set defining its object address.  
  
Note that regression tests are added within event_trigger and not  
object_address to avoid issues with concurrent connections in parallel  
schedules.  
  
Author: Joel Jacobson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : ec5bab9217cddfda134abf1131904ec1958caadb    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 26 Apr 2021 11:50:35 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 26 Apr 2021 11:50:35 -0400    

For some reason, the "julian" option for extract()/date_part() has  
never gotten listed in the manual.  Also, while Appendix B mentioned  
in passing that we don't conform to the usual astronomical definition  
that a Julian date starts at noon UTC, it was kind of vague about what  
we do instead.  Clarify that, and add an example showing how to get  
the astronomical definition if you want it.  
  
It's been like this for ages, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 8019fcb00f8832e3b0d787e078f28dbb853f3151    
  
author   : Fujii Masao <[email protected]>    
date     : Fri, 23 Apr 2021 15:45:46 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Fri, 23 Apr 2021 15:45:46 +0900    

Previously it was documented that if using "-X none" option there was  
no guarantee that all required WAL files were archived at the end of  
pg_basebackup when taking a backup from the standby. But this limitation  
was removed by commit 52f8a59dd9. Now, even when taking a backup  
from the standby, pg_basebackup can wait for all required WAL files  
to be archived. Therefore this commit removes such obsolete  
description from the docs.  
  
Also this commit adds new description about the limitation when  
taking a backup from the standby, into the docs. The limitation is that  
pg_basebackup cannot force the standbfy to switch to a new WAL file  
at the end of backup, which may cause pg_basebackup to wait a long  
time for the last required WAL file to be switched and archived,  
especially when write activity on the primary is low.  
  
Back-patch to v10 where the issue was introduced.  
  
Reported-by: Kyotaro Horiguchi  
Author: Kyotaro Horiguchi, Fujii Masao  
Reviewed-by: Kyotaro Horiguchi, Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

commit   : 2602ee4689c7691196568c59656662acf3be4e87    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 22 Apr 2021 17:30:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 22 Apr 2021 17:30:42 -0400    

Adopt a more consistent policy about what slot-type-specific  
getsysattr functions should do when system attributes are not  
available.  To wit, they should all throw the same user-oriented  
error, rather than variously crashing or emitting developer-oriented  
messages.  
  
This closes a identifiable problem in commits a71cfc56b and  
3fb93103a (in v13 and v12), so back-patch into those branches,  
along with a test case to try to ensure we don't break it again.  
It is not known that any of the former crash cases are reachable  
in HEAD, but this seems like a good safety improvement in any case.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 00037d8d004031d80d276da543262004ed6bff48    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 22 Apr 2021 14:47:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 22 Apr 2021 14:47:26 -0400    

Back-patch to v13; the table layout in older branches is unfriendly  
to adding such details.  
  
Laurenz Albe  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a71cfc56bf6013e3ea1d673acaf73fe7ebbd6bf3    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 22 Apr 2021 11:46:41 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 22 Apr 2021 11:46:41 -0400    

If the source and destination partitions don't have identical  
rowtypes (for example, one has dropped columns the other lacks),  
then the planSlot contents will be different because of that.  
If the query has a RETURNING list that tries to return resjunk  
columns out of the planSlot, that is columns from tables that  
were joined to the target table, we'd get errors or wrong answers.  
That's because we used the RETURNING list generated for the  
destination partition, which expects a planSlot matching that  
partition's subplan.  
  
The most practical fix seems to be to convert the updated destination  
tuple back to the source partition's rowtype, and then apply the  
RETURNING list generated for the source partition.  This avoids making  
fragile assumptions about whether the per-subpartition subplans  
generated all the resjunk columns in the same order.  
  
This has been broken since v11 introduced cross-partition UPDATE.  
The lack of field complaints shows that non-identical partitions  
aren't a common case; therefore, don't stress too hard about  
making the conversion efficient.  
  
There's no such bug in HEAD, because commit 86dc90056 got rid of  
per-target-relation variance in the contents of the planSlot.  
Hence, patch v11-v13 only.  
  
Amit Langote and Etsuro Fujita, small changes by me  
  
Discussion: https://postgr.es/m/CA+HiwqE_UK1jTSNrjb8mpTdivzd3dum6mK--xqKq0Y9VmfwWQA@mail.gmail.com  

commit   : 574a1b8fb42a9a74b84137a7600d561b4b2d48a4    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 11:12:04 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 11:12:04 -0400    

commit   : d31ae9286641b008bd5b100024d0ca4c20645386    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 10:21:22 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 10:21:22 -0400    

This has been found to cause hangs where tcp usage is forced.  
  
Alexey Kodratov  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch to all live branches  

commit   : 7bfba4f1933003716d432d29d4d228bcf28e2e70    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 20 Apr 2021 11:32:02 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 20 Apr 2021 11:32:02 -0400    

An oversight introduced by the incremental-sort patches caused  
"could not find pathkey item to sort" errors in some situations  
where a sort key involves an aggregate or window function.  
  
The basic problem here is that find_em_expr_usable_for_sorting_rel  
isn't properly modeling what prepare_sort_from_pathkeys will do  
later.  Rather than hoping we can keep those functions in sync,  
let's refactor so that they actually share the code for  
identifying a suitable sort expression.  
  
With this refactoring, tlist.c's tlist_member_ignore_relabel  
is unused.  I removed it in HEAD but left it in place in v13,  
in case any extensions are using it.  
  
Per report from Luc Vlaming.  Back-patch to v13 where the  
problem arose.  
  
James Coleman and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : bf5d1f1e0052100b716f7b7b0934aefb28f72e73    
  
author   : Magnus Hagander <[email protected]>    
date     : Tue, 20 Apr 2021 14:35:16 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Tue, 20 Apr 2021 14:35:16 +0200    

Author: Julien Rouhaud  
Backpatch-through: 11  
Discussion: https://postgr.es/m/20210420121659.odjueyd4rpilorn5@nol  

commit   : e480c6dd33f40bf4a2ae561799b08a1cd7c3073b    
  
author   : Andrew Dunstan <[email protected]>    
date     : Fri, 16 Apr 2021 16:54:04 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Fri, 16 Apr 2021 16:54:04 -0400    

In order to avoid getting old logfile contents certain functions in  
PostgresNode were doing one of two things. On Windows it rotated the  
logfile and restarted the server, while elsewhere it truncated the log  
file. Both of these are unnecessary. We borrow from the buildfarm which  
does this instead: note the size of the logfile before we start, and  
then when fetching the logfile skip to that position before accumulating  
contents. This is spelled differently on Windows but the effect is the  
same. This is largely centralized in TestLib's slurp_file function,  
which has a new optional parameter, the offset to skip to before  
starting to reading the file. Code in the client becomes much neater.  
  
Backpatch to all live branches.  
  
Michael Paquier, slightly modified by me.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0e8acd39ecdf9a68ebdaf9652a56b943f34c9c58    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 16 Apr 2021 16:56:25 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 16 Apr 2021 16:56:25 +0900    

Author: Erik Rijkers  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

commit   : c39aa1e878cb9da16ed6bbc32bb1c83ef93595d4    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 15:10:18 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 15:10:18 -0400    

Most GUC check hooks that inspect database state have special checks  
that prevent them from throwing hard errors for state-dependent issues  
when source == PGC_S_TEST.  This allows, for example,  
"ALTER DATABASE d SET default_text_search_config = foo" when the "foo"  
configuration hasn't been created yet.  Without this, we have problems  
during dump/reload or pg_upgrade, because pg_dump has no idea about  
possible dependencies of GUC values and can't ensure a safe restore  
ordering.  
  
However, check_role() and check_session_authorization() hadn't gotten  
the memo about that, and would throw hard errors anyway.  It's not  
entirely clear what is the use-case for "ALTER ROLE x SET role = y",  
but we've now heard two independent complaints about that bollixing  
an upgrade, so apparently some people are doing it.  
  
Hence, fix these two functions to act more like other check hooks  
with similar needs.  (But I did not change their insistence on  
being inside a transaction, as it's still not apparent that setting  
either GUC from the configuration file would be wise.)  
  
Also fix check_temp_buffers, which had a different form of the disease  
of making state-dependent checks without any exception for PGC_S_TEST.  
A cursory survey of other GUC check hooks did not find any more issues  
of this ilk.  (There are a lot of interdependencies among  
PGC_POSTMASTER and PGC_SIGHUP GUCs, which may be a bad idea, but  
they're not relevant to the immediate concern because they can't be  
set via ALTER ROLE/DATABASE.)  
  
Per reports from Charlie Hornsby and Nathan Bossart.  Back-patch  
to all supported branches.  
  
Discussion: https://postgr.es/m/HE1P189MB0523B31598B0C772C908088DB7709@HE1P189MB0523.EURP189.PROD.OUTLOOK.COM  
Discussion: https://postgr.es/m/[email protected]  

commit   : 97b7ad4688848b38ceac604e1dd2e6af39098229    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 13:37:07 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 13:37:07 -0400    

Previously, get_cached_rowtype() cached a pointer to a reference-counted  
tuple descriptor from the typcache, relying on the ExprContextCallback  
mechanism to release the tupdesc refcount when the expression tree  
using the tupdesc was destroyed.  This worked fine when it was designed,  
but the introduction of within-DO-block COMMITs broke it.  The refcount  
is logged in a transaction-lifespan resource owner, but plpgsql won't  
destroy simple expressions made within the DO block (before its first  
commit) until the DO block is exited.  That results in a warning about  
a leaked tupdesc refcount when the COMMIT destroys the original resource  
owner, and then an error about the active resource owner not holding a  
matching refcount when the expression is destroyed.  
  
To fix, get rid of the need to have a shutdown callback at all, by  
instead caching a pointer to the relevant typcache entry.  Those  
survive for the life of the backend, so we needn't worry about the  
pointer becoming stale.  (For registered RECORD types, we can still  
cache a pointer to the tupdesc, knowing that it won't change for the  
life of the backend.)  This mechanism has been in use in plpgsql  
and expandedrecord.c since commit 4b93f5799, and seems to work well.  
  
This change requires modifying the ExprEvalStep structs used by the  
relevant expression step types, which is slightly worrisome for  
back-patching.  However, there seems no good reason for extensions  
to be familiar with the details of these particular sub-structs.  
  
Per report from Rohit Bhogate.  Back-patch to v11 where within-DO-block  
COMMITs became a thing.  
  
Discussion: https://postgr.es/m/CAAV6ZkQRCVBh8qAY+SZiHnz+U+FqAGBBDaDTjF2yiKa2nJSLKg@mail.gmail.com  

commit   : 37e76546a2ba4cdf1ca14b25b1e729aef7235870    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 12:17:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 12:17:24 -0400    

heap_update needs to clear any existing "all visible" flag on  
the old tuple's page (and on the new page too, if different).  
Per coding rules, to do this it must acquire pin on the appropriate  
visibility-map page while not holding exclusive buffer lock;  
which creates a race condition since someone else could set the  
flag whenever we're not holding the buffer lock.  The code is  
supposed to handle that by re-checking the flag after acquiring  
buffer lock and retrying if it became set.  However, one code  
path through heap_update itself, as well as one in its subroutine  
RelationGetBufferForTuple, failed to do this.  The end result,  
in the unlikely event that a concurrent VACUUM did set the flag  
while we're transiently not holding lock, is a non-recurring  
"PANIC: wrong buffer passed to visibilitymap_clear" failure.  
  
This has been seen a few times in the buildfarm since recent VACUUM  
changes that added code paths that could set the all-visible flag  
while holding only exclusive buffer lock.  Previously, the flag  
was (usually?) set only after doing LockBufferForCleanup, which  
would insist on buffer pin count zero, thus preventing the flag  
from becoming set partway through heap_update.  However, it's  
clear that it's heap_update not VACUUM that's at fault here.  
  
What's less clear is whether there is any hazard from these bugs  
in released branches.  heap_update is certainly violating API  
expectations, but if there is no code path that can set all-visible  
without a cleanup lock then it's only a latent bug.  That's not  
100% certain though, besides which we should worry about extensions  
or future back-patch fixes that could introduce such code paths.  
  
I chose to back-patch to v12.  Fixing RelationGetBufferForTuple  
before that would require also back-patching portions of older  
fixes (notably 0d1fe9f74), which is more code churn than seems  
prudent to fix a hypothetical issue.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 13881199ee5760ed5e40df1fa749d9715467f43e    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:41 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:41 -0700    

With the Oracle Developer Studio 12.6 compiler, #line directives alter  
the current source file location for purposes of #include "..."  
directives.  Hence, a VPATH build failed with 'cannot find include file:  
"specscanner.c"'.  With two exceptions, parser-containing directories  
already add "-I. -I$(srcdir)"; eliminate the exceptions.  Back-patch to  
9.6 (all supported versions).  

commit   : 766c8fce5281f8a7cf690e1fe2dc3a19c326578f    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:21 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:21 -0700    

It doesn't support "\(foo\)*" like a POSIX "sed" implementation does;  
see the Autoconf manual.  Back-patch to 9.6 (all supported versions).  

commit   : 16175e278763e88618c814b6489c4ae795501282    
  
author   : Thomas Munro <[email protected]>    
date     : Tue, 13 Apr 2021 12:34:25 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Tue, 13 Apr 2021 12:34:25 +1200    

Commit 6f38d4dac38 failed to heed a warning about the stability of the  
value pointed to by "otid".  The caller is allowed to pass in a pointer to  
newtup->t_self, which will be updated during the execution of the  
function.  Instead, the SSI check should use the value we copy into  
oldtup.t_self near the top of the function.  
  
Not a live bug, because newtup->t_self doesn't really get updated until  
a bit later, but it was confusing and broke the rule established by the  
comment.  
  
Back-patch to 13.  
  
Reported-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/2689164.1618160085%40sss.pgh.pa.us  

commit   : 8a7bd1e6cba37170afadcb252da3cf9c5386ef06    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 12 Apr 2021 14:37:22 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 12 Apr 2021 14:37:22 -0400    

There are hacks in parse_coerce.c to push down a requested coercion  
to below any CollateExpr that may appear.  However, we did that even  
if the requested data type is non-collatable, leading to an invalid  
expression tree in which CollateExpr is applied to a non-collatable  
type.  The fix is just to drop the CollateExpr altogether, reasoning  
that it's useless.  
  
This bug is ten years old, dating to the original addition of  
COLLATE support.  The lack of field complaints suggests that there  
aren't a lot of user-visible consequences.  We noticed the problem  
because it would trigger an assertion in DefineVirtualRelation if  
the invalid structure appears as an output column of a view; however,  
in a non-assert build, you don't see a crash just a (subtly incorrect)  
complaint about applying collation to a non-collatable type.  I found  
that by putting the incorrect structure further down in a view, I could  
make a view definition that would fail dump/reload, per the added  
regression test case.  But CollateExpr doesn't do anything at run-time,  
so this likely doesn't lead to any really exciting consequences.  
  
Per report from Yulin Pei.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com  

commit   : 48d4a8c88ba73c5e68461d1ced9090ac33827028    
  
author   : Amit Kapila <[email protected]>    
date     : Mon, 12 Apr 2021 09:03:16 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Mon, 12 Apr 2021 09:03:16 +0530    

Currently, parallel vacuum workers don't use any buffer access strategy.  
We can fix it either by propagating the access strategy from a leader or  
allow each worker to use BAS_VACUUM access strategy type. We don't see  
much use in propagating this information from leader as both leader and  
workers are supposed to use the same strategy. We might want to use a  
different access strategy for leader and workers but that would be a  
separate optimization not suitable for back-branches. This has been fixed  
in HEAD as commit f6b8f19a08.  
  
Author: Amit Kapila  
Reviewed-by: Sawada Masahiko, Bharath Rupireddy  
Discussion: https://postgr.es/m/CAA4eK1KbmJgRV2W3BbzRnKUSrukN7SbqBBriC4RDB5KBhopkGQ@mail.gmail.com  

commit   : be79debd9688da516f49ba9b825ee2e784d1fab0    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 12 Apr 2021 11:31:26 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 12 Apr 2021 11:31:26 +0900    

Using Roman numbers (via "RM" or "rm") for a conversion to calculate a  
number of months has never considered the case of negative numbers,  
where a conversion could easily cause out-of-bound memory accesses.  The  
conversions in themselves were not completely consistent either, as  
specifying 12 would result in NULL, but it should mean XII.  
  
This commit reworks the conversion calculation to have a more  
consistent behavior:  
- If the number of months and years is 0, return NULL.  
- If the number of months is positive, return the exact month number.  
- If the number of months is negative, do a backward calculation, with  
-1 meaning December, -2 November, etc.  
  
Reported-by: Theodor Arsenij Larionov-Trichkin  
Author: Julien Rouhaud  
Discussion: https://postgr.es/m/[email protected]  
backpatch-through: 9.6  

commit   : 19b28d69128a7ca78705be9f4afca5044541beca    
  
author   : Magnus Hagander <[email protected]>    
date     : Fri, 9 Apr 2021 12:40:14 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Fri, 9 Apr 2021 12:40:14 +0200    

Author: Daniel Westermann  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/GV0P278MB0483A7AA85BAFCC06D90F453D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM  

commit   : dc6d285c2ef997f3a8d4c9f0e0599b62b0cbd054    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 9 Apr 2021 13:53:17 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 9 Apr 2021 13:53:17 +0900    

Comment fixes are applied on HEAD, and documentation improvements are  
applied on back-branches where needed.  
  
Author: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 1aad1d181d28ede484302188413b07100e12383f    
  
author   : Tomas Vondra <[email protected]>    
date     : Wed, 7 Apr 2021 15:58:35 +0200    
  
committer: Tomas Vondra <[email protected]>    
date     : Wed, 7 Apr 2021 15:58:35 +0200    

The code in bringetbitmap() simply added the whole matching page range  
to the TID bitmap, as determined by pages_per_range, even if some of the  
pages were beyond the end of the heap. The query then might fail with  
an error like this:  
  
  ERROR:  could not open file "base/20176/20228.2" (target block  
          262144): previous segment is only 131021 blocks  
  
In this case, the relation has 262093 pages (131072 and 131021 pages),  
but we're trying to acess block 262144, i.e. first block of the 3rd  
segment. At that point _mdfd_getseg() notices the preceding segment is  
incomplete, and fails.  
  
Hitting this in practice is rather unlikely, because:  
  
* Most indexes use power-of-two ranges, so segments and page ranges  
  align perfectly (segment end is also a page range end).  
  
* The table size has to be just right, with the last segment being  
  almost full - less than one page range from full segment, so that the  
  last page range actually crosses the segment boundary.  
  
* Prefetch has to be enabled. The regular page access checks that  
  pages are not beyond heap end, but prefetch does not. On older  
  releases (before 12) the execution stops after hitting the first  
  non-existent page, so the prefetch distance has to be sufficient  
  to reach the first page in the next segment to trigger the issue.  
  Since 12 it's enough to just have prefetch enabled, the prefetch  
  distance does not matter.  
  
Fixed by not adding non-existent pages to the TID bitmap. Backpatch  
all the way back to 9.6 (BRIN indexes were introduced in 9.5, but that  
release is EOL).  
  
Backpatch-through: 9.6  

commit   : b55619948ea18dbc0f4f8ca823aaa687c0fa2567    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 7 Apr 2021 19:58:54 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 7 Apr 2021 19:58:54 +0900    

Instead of writing a query to psql's stdin, which can cause a failure  
where psql exits before writing, reporting a write failure with a broken  
pipe, this changes the logic to use -c.  This was not seen in the  
buildfarm as no animals with a sensitive environment are running the  
kerberos tests, but let's be safe.  
  
HEAD is able to handle the situation as of 6d41dd0 for all the test  
suites doing connection checks.  f44b9b6 has fixed the same problem for  
the LDAP tests.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 11  

commit   : e7bcfd717ef31166617e0391d5c132f3cfb30fab    
  
author   : Fujii Masao <[email protected]>    
date     : Tue, 6 Apr 2021 02:25:37 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Tue, 6 Apr 2021 02:25:37 +0900    

Maxim Orlov reported that the shutdown of standby server could result in  
the following assertion failure. The cause of this issue was that,  
when the shutdown caused the startup process to exit, recovery-time  
transaction tracking was not shut down even if it's already initialized,  
and some locks the tracked transactions were holding could not be released.  
At this situation, if other process was invoked and the PGPROC entry that  
the startup process used was assigned to it, it found such unreleased locks  
and caused the assertion failure, during the initialization of it.  
  
    TRAP: FailedAssertion("SHMQueueEmpty(&(MyProc->myProcLocks[i]))"  
  
This commit fixes this issue by making the startup process shut down  
transaction tracking and release all locks, at the exit of it.  
  
Back-patch to all supported branches.  
  
Reported-by: Maxim Orlov  
Author: Fujii Masao  
Reviewed-by: Maxim Orlov  
Discussion: https://postgr.es/m/[email protected]  

commit   : fd91fed3e802c77b1e8a882b9797fd3c17115bbb    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 4 Apr 2021 17:57:07 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 4 Apr 2021 17:57:07 -0400    

spg_box_quad_leaf_consistent unconditionally returned the leaf  
datum as leafValue, even though in its usage for poly_ops that  
value is of completely the wrong type.  
  
In versions before 12, that was harmless because the core code did  
nothing with leafValue in non-index-only scans ... but since commit  
2a6368343, if we were doing a KNN-style scan, spgNewHeapItem would  
unconditionally try to copy the value using the wrong datatype  
parameters.  Said copying is a waste of time and space if we're not  
going to return the data, but it accidentally failed to fail until  
I fixed the datatype confusion in ac9099fc1.  
  
Hence, change spgNewHeapItem to not copy the datum unless we're  
actually going to return it later.  This saves cycles and dodges  
the question of whether lossy opclasses are returning the right  
type.  Also change spg_box_quad_leaf_consistent to not return  
data that might be of the wrong type, as insurance against  
somebody introducing a similar bug into the core code in future.  
  
It seems like a good idea to back-patch these two changes into  
v12 and v13, although I'm afraid to change spgNewHeapItem's  
mistaken idea of which datatype to use in those branches.  
  
Per buildfarm results from ac9099fc1.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 45aea47ef430bc942b831932adaca7730c6ae775    
  
author   : Bruce Momjian <[email protected]>    
date     : Fri, 2 Apr 2021 16:42:29 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Fri, 2 Apr 2021 16:42:29 -0400    

All other places already use MONTHS_PER_YEAR appropriately.  
  
Backpatch-through: 9.6  

commit   : 4a1b95fcf4710c9578750d1d9ab4672eab3a5a82    
  
author   : Joe Conway <[email protected]>    
date     : Fri, 2 Apr 2021 13:48:45 -0400    
  
committer: Joe Conway <[email protected]>    
date     : Fri, 2 Apr 2021 13:48:45 -0400    

Command-line options, or previous "ALTER (ROLE|DATABASE) ...  
SET ROLE ..." commands, can change the value of the default role  
for a session. In the presence of one of these, RESET ROLE will  
change the current user identifier to the default role rather  
than the session user identifier. Fix the documentation to  
reflect this reality. Backpatch to all supported versions.  
  
Author: Nathan Bossart  
Reviewed-By: Laurenz Albe, David G. Johnston, Joe Conway  
Reported by: Nathan Bossart  
Discussion: https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com  
Backpatch-through: 9.6  

commit   : 104164361cb18d7c36d068e1ddd2453b0e4dc1bb    
  
author   : Fujii Masao <[email protected]>    
date     : Sat, 3 Apr 2021 00:07:00 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Sat, 3 Apr 2021 00:07:00 +0900    

pg_checksums uses two counters, total size and current size,  
to calculate the progress. Previously the progress that  
pg_checksums reported could not reach 100% at the end.  
The cause of this issue was that the sizes of only pages excluding  
new ones in each file were counted as the current size  
while the size of each file is counted as the total size.  
That is, the total size of all new pages could be reported  
as the difference between the total size and current size.  
  
This commit fixes this issue by making pg_checksums count  
the sizes of all pages including new ones in each file as  
the current size.  
  
Back-patch to v12 where progress reporting was added to pg_checksums.  
  
Reported-by: Shinya Kato  
Author: Shinya Kato  
Reviewed-by: Fujii Masao  
Discussion: https://postgr.es/m/TYAPR01MB289656B1ACA0A5E7CAD07BE3C47A9@TYAPR01MB2896.jpnprd01.prod.outlook.com  

commit   : f8c2d491234f4e9e2e7c183109250cc7c8bfd148    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 2 Apr 2021 16:37:07 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 2 Apr 2021 16:37:07 +0900    

The current instructions describing how to write the backup_label and  
tablespace_map files are confusing.  For example, opening a file in text  
mode on Windows and copy-pasting the file's contents would result in a  
failure at recovery because of the extra CRLF characters generated.  The  
documentation was not stating that clearly, and per discussion this is  
not considered as a supported scenario.  
  
This commit extends a bit the documentation to mention that it may be  
required to open the file in binary mode before writing its data.  
  
Reported-by: Wang Shenhao  
Author: David Steele  
Reviewed-by: Andrew Dunstan, Magnus Hagander  
Discussion: https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local  
Backpatch-through: 9.6  

commit   : 75e66ee690ae24c546f9a7d01d99779dbb9e08f1    
  
author   : Bruce Momjian <[email protected]>    
date     : Thu, 1 Apr 2021 21:17:24 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Thu, 1 Apr 2021 21:17:24 -0400    

Also mention that you should read the intervening major releases notes.  
This change was also applied to the website.  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

commit   : 89937d001294b39469790e5a583e438af2ca1235    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 2 Apr 2021 09:44:50 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 2 Apr 2021 09:44:50 +0900    

This test has been using a simple VACUUM with pg_relation_size() to  
check if a relation gets physically truncated or not, but forgot the  
fact that some concurrent activity, like checkpoint buffer writes, could  
cause some pages to be skipped.  The second test enabling  
vacuum_truncate could fail, seeing a non-empty relation.  The first test  
would not have failed, but could finish by testing a behavior different  
than the one aimed for.  Both tests gain a FREEZE option, to make the  
vacuums more aggressive and prevent page skips.  
  
This is similar to the issues fixed in c2dc1a7.  
  
Author: Arseny Sher  
Reviewed-by: Masahiko Sawada  
Discussion: https://postgr.es/m/87tuotr2hh.fsf@ars-thinkpad  
backpatch-through: 12  

commit   : 35421a470de16565e5dca87dc4e2b76fa19c7d08    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 1 Apr 2021 13:34:16 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 1 Apr 2021 13:34:16 -0400    

Despite the clear comments pointing out that the duplicative code  
segments in ReadHead() and _discoverArchiveFormat() needed to be  
in sync, they were not: the latter did not bother to apply any of  
the sanity checks in the former.  We'd missed noticing this partly  
because none of those checks would fail in scenarios we customarily  
test, and partly because the oversight would be masked if both  
segments execute, which they would in cases other than needing to  
autodetect the format of a non-seekable stdin source.  However,  
in a case meeting all these requirements --- for example, trying  
to read a newer-than-supported archive format from non-seekable  
stdin --- pg_restore missed applying the version check and would  
likely dump core or otherwise misbehave.  
  
The whole thing is silly anyway, because there seems little reason  
to duplicate the logic beyond the one-line verification that the  
file starts with "PGDMP".  There seems to have been an undocumented  
assumption that multiple major formats (major enough to require  
separate reader modules) would nonetheless share the first half-dozen  
fields of the custom-format header.  This seems unlikely, so let's  
fix it by just nuking the duplicate logic in _discoverArchiveFormat().  
  
Also get rid of the pointless attempt to seek back to the start of  
the file after successful autodetection.  That wastes cycles and  
it means we have four behaviors to verify not two.  
  
Per bug #16951 from Sergey Koposov.  This has been broken for  
decades, so back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 876ecfba4d829f6d6c70ac4aa5f05b03269fbd95    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 1 Apr 2021 15:28:45 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 1 Apr 2021 15:28:45 +0900    

Some sections of the documentation used "exclusive lock" to describe  
that an ACCESS EXCLUSIVE lock is taken during a given operation.  This  
can be confusing to the reader as ACCESS SHARE is allowed with an  
EXCLUSIVE lock is used, but that would not be the case with what is  
described on those parts of the documentation.  
  
Author: Greg Rychlewski  
Discussion: https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com  
Backpatch-through: 9.6  

commit   : 89e383b30a94d1fb1ebfd63131433a93bcf47625    
  
author   : Stephen Frost <[email protected]>    
date     : Wed, 31 Mar 2021 16:23:18 -0400    
  
committer: Stephen Frost <[email protected]>    
date     : Wed, 31 Mar 2021 16:23:18 -0400    

The new appendix groups information on renamed or removed settings,  
commands, etc into an out-of-the-way part of the docs.  
  
The original id elements are retained in each subsection to ensure that  
the same filenames are produced for HTML docs. This prevents /current/  
links on the web from breaking, and allows users of the web docs  
to follow links from old version pages to info on the changes in the  
new version. Prior to this change, a link to /current/ for renamed  
sections like the recovery.conf docs would just 404. Similarly if  
someone searched for recovery.conf they would find the pg11 docs,  
but there would be no /12/ or /current/ link, so they couldn't easily  
find out that it was removed in pg12 or how to adapt.  
  
Index entries are also added so that there's a breadcrumb trail for  
users to follow when they know the old name, but not what we changed it  
to. So a user who is trying to find out how to set standby_mode in  
PostgreSQL 12+, or where pg_resetxlog went, now has more chance of  
finding that information.  
  
Craig Ringer and Stephen Frost  
Reviewed-by: Euler Taveira  
Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com  
Backpatch-through: 10  

commit   : 78f8b0965a224e650b4fc42e5c174256ca0fc929    
  
author   : Etsuro Fujita <[email protected]>    
date     : Tue, 30 Mar 2021 13:00:02 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Tue, 30 Mar 2021 13:00:02 +0900    

Back-patch to all supported branches.  
  
Author: Etsuro Fujita  
Discussion: https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com  

commit   : f50dc2c725fd546b994f228101211ae50e6858e5    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 29 Mar 2021 18:34:39 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 29 Mar 2021 18:34:39 -0300    

We were never doing clearerr() on the output stream, which results in a  
message being printed after each result once an EOF is seen:  
  
could not print result table: Success  
  
This message was added by commit b03436994bcc (in the pg13 era); before  
that, the error indicator would never be examined.  So backpatch only  
that far back, even though the actual bug (to wit: the fact that the  
error indicator is never cleared) is older.  

commit   : 092d3db05d34e2cd122b066cc61c139e7b90d635    
  
author   : Stephen Frost <[email protected]>    
date     : Sun, 28 Mar 2021 11:28:12 -0400    
  
committer: Stephen Frost <[email protected]>    
date     : Sun, 28 Mar 2021 11:28:12 -0400    

Commit c6763156589 added an acronym reference for "TLS" but the definition  
was never added.  
  
Author: Daniel Gustafsson  
Reviewed-by: Michael Paquier  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/[email protected]  

commit   : 67251c82af87865989eb90c7e8f4546cc0d66e6d    
  
author   : Tomas Vondra <[email protected]>    
date     : Fri, 26 Mar 2021 22:34:53 +0100    
  
committer: Tomas Vondra <[email protected]>    
date     : Fri, 26 Mar 2021 22:34:53 +0100    

When estimating the number of groups using extended statistics, the code  
was discarding information about system attributes. This led to strange  
situation that  
  
    SELECT 1 FROM t GROUP BY ctid;  
  
could have produced higher estimate (equal to pg_class.reltuples) than  
  
    SELECT 1 FROM t GROUP BY a, b, ctid;  
  
with extended statistics on (a,b). Fixed by retaining information about  
the system attribute.  
  
Backpatch all the way to 10, where extended statistics were introduced.  
  
Author: Tomas Vondra  
Backpatch-through: 10  

commit   : c8622999b7fe53741304f2aca73560aade6557d2    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 25 Mar 2021 16:30:22 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 25 Mar 2021 16:30:22 -0300    

On partition detach, we acquire a SHARE lock on all tables that  
reference the partitioned table that we're detaching a partition from,  
but failed to document this fact.  My oversight in commit f56f8f8da6af.  
Repair.  Backpatch to 12.  
  
Author: Álvaro Herrera <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : a00c3068206e6daa236dbff0256cf3ad25ff5ed2    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 25 Mar 2021 10:47:38 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 25 Mar 2021 10:47:38 -0300    

I introduced this duplicate code in commit 8b08f7d4820f for no good  
reason.  Remove it, and backpatch to 11 where it was introduced.  
  
Author: Álvaro Herrera <[email protected]>  

commit   : 092c077c1321c259f86721ad9319fae3c90c9b8b    
  
author   : Fujii Masao <[email protected]>    
date     : Thu, 25 Mar 2021 11:23:30 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Thu, 25 Mar 2021 11:23:30 +0900    

Previously the WAL replay of COMMIT_TS_SETTS record called  
TransactionTreeSetCommitTsData() with the argument write_xlog=true,  
which generated and wrote new COMMIT_TS_SETTS record.  
This should not be acceptable because it's during recovery.  
  
This commit fixes the WAL replay of COMMIT_TS_SETTS record  
so that it calls TransactionTreeSetCommitTsData() with write_xlog=false  
and doesn't generate new WAL during recovery.  
  
Back-patch to all supported branches.  
  
Reported-by: lx zou <[email protected]>  
Author: Fujii Masao  
Reviewed-by: Alvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  

commit   : c6eac71a88447d82d5f6d28b6bed2e9c6971d714    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 23 Mar 2021 14:27:50 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 23 Mar 2021 14:27:50 -0400    

Jasen Betts reported yet another unintended side effect of commit  
85c54287a: reconnecting with "\c service=whatever" did not have the  
expected results.  The reason is that starting from the output of  
PQconndefaults() effectively allows environment variables (such  
as PGPORT) to override entries in the service file, whereas the  
normal priority is the other way around.  
  
Not using PQconndefaults at all would require yet a third main code  
path in do_connect's parameter setup, so I don't really want to fix  
it that way.  But we can have the logic effectively ignore all the  
default values for just a couple more lines of code.  
  
This patch doesn't change the behavior for "\c -reuse-previous=on  
service=whatever".  That remains significantly different from before  
85c54287a, because many more parameters will be re-used, and thus  
not be possible for service entries to replace.  But I think this  
is (mostly?) intentional.  In any case, since libpq does not report  
where it got parameter values from, it's hard to do differently.  
  
Per bug #16936 from Jasen Betts.  As with the previous patches,  
back-patch to all supported branches.  (9.5 is unfortunately now  
out of support, so this won't get fixed there.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : d4791ac35cb1d7417ea3cff6cc604623463ef0ea    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 23 Mar 2021 11:24:16 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 23 Mar 2021 11:24:16 -0400    

end_heap_rewrite was not careful to ensure that the target relation  
is open at the smgr level before performing its final smgrimmedsync.  
In ordinary cases this is no problem, because it would have been  
opened earlier during the rewrite.  However a crash can be reproduced  
by re-clustering an empty table with CLOBBER_CACHE_ALWAYS enabled.  
  
Although that exact scenario does not crash in v13, I think that's  
a chance result of unrelated planner changes, and the problem is  
likely still reachable with other test cases.  The true proximate  
cause of this failure is commit c6b92041d, which replaced a call to  
heap_sync (which was careful about opening smgr) with a direct call  
to smgrimmedsync.  Hence, back-patch to v13.  
  
Amul Sul, per report from Neha Sharma; cosmetic changes  
and test case by me.  
  
Discussion: https://postgr.es/m/CANiYTQsU7yMFpQYnv=BrcRVqK_3U3mtAzAsJCaqtzsDHfsUbdQ@mail.gmail.com  

commit   : 1faaad260d8f56d4f73af76ab576b85e9013beca    
  
author   : Tomas Vondra <[email protected]>    
date     : Tue, 23 Mar 2021 04:45:26 +0100    
  
committer: Tomas Vondra <[email protected]>    
date     : Tue, 23 Mar 2021 04:45:26 +0100    

A couple error messages and comments used 'statistic kind', not the  
correct 'statistics kind'. Fix and backpatch all the way back to 10,  
where extended statistics were introduced.  
  
Backpatch-through: 10  

commit   : 34279fd4fabdedba0bc72b05dc32753e1193d599    
  
author   : Fujii Masao <[email protected]>    
date     : Tue, 23 Mar 2021 09:53:08 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Tue, 23 Mar 2021 09:53:08 +0900    

pg_waldump --stats=record identifies a record by a combination  
of the RmgrId and the four bits of the xl_info field of the record.  
But XACT records use the first bit of those four bits for an optional  
flag variable, and the following three bits for the opcode to  
identify a record. So previously the same type of XACT record  
could have different four bits (three bits are the same but the  
first one bit is different), and which could cause  
pg_waldump --stats=record to show two lines of per-record statistics  
for the same XACT record. This is a bug.  
  
This commit changes pg_waldump --stats=record so that it processes  
only XACT record differently, i.e., filters the opcode out of xl_info  
and uses a combination of the RmgrId and those three bits as  
the identifier of a record, only for XACT record. For other records,  
the four bits of the xl_info field are still used.  
  
Back-patch to all supported branches.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Shinya Kato, Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

commit   : 78c24e97dd189f62187a99ef84016d0eb35a7978    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 22 Mar 2021 14:02:36 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 22 Mar 2021 14:02:36 +0900    

This commit is mostly a revert of aaa3aed, that switched the routine  
doing the internal renaming of recycled WAL segments to use on Windows a  
combination of CreateHardLinkA() plus unlink() instead of rename().  As  
reported by several users of Postgres 13, this is causing concurrency  
issues when manipulating WAL segments, mostly in the shape of the  
following error:  
LOG:  could not rename file "pg_wal/000000XX000000YY000000ZZ":  
Permission denied  
  
This moves back to a logic where a single rename() (well, pgrename() for  
Windows) is used.  This issue has proved to be hard to hit when I tested  
it, facing it only once with an archive_command that was not able to do  
its work, so it is environment-sensitive.  The reporters of this issue  
have been able to confirm that the situation improved once we switched  
back to a single rename().  In order to check things, I have provided to  
the reporters a patched build based on 13.2 with aaa3aed reverted, to  
test if the error goes away, and an unpatched build of 13.2 to test if  
the error still showed up (just to make sure that I did not mess up my  
build process).  
  
Extra thanks to Fujii Masao for pointing out what looked like the  
culprit commit, and to all the reporters for taking the time to test  
what I have sent them.  
  
Reported-by: Andrus, Guy Burgess, Yaroslav Pashinsky, Thomas Trenz  
Reviewed-by: Tom Lane, Andres Freund  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/16927-67c570d968c99567%40postgresql.org  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 13  

commit   : 48664e41688ca113e01b7055d41957c600e565ca    
  
author   : Noah Misch <[email protected]>    
date     : Sun, 21 Mar 2021 19:09:29 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Sun, 21 Mar 2021 19:09:29 -0700    

Back-patch to v13, which introduced the test code in question.  

commit   : 7bdc717a5438650c13b7dcde427828eb1743fd8c    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 22 Mar 2021 09:51:14 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 22 Mar 2021 09:51:14 +0900    

The test added by 595b9cb forgot that on Windows it is necessary to set  
up pg_hba.conf (see PostgresNode::set_replication_conf) with a specific  
entry or base backups fail.  Any node that requires to support  
replication just needs to pass down allows_streaming at initialization.  
This updates the test to do so.  Simplify things a bit while on it.  
  
Per buildfarm member fairywren.  Any Windows hosts running this test  
would have failed, and I have reproduced the problem as well.  
  
Backpatch-through: 10  

commit   : 6e5ce888ad1e7b7da5de507d89d03bc83d954923    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 22 Mar 2021 08:31:01 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 22 Mar 2021 08:31:01 +0900    

Any transactions found as still prepared by a checkpoint have their  
state data read from the WAL records generated by PREPARE TRANSACTION  
before being moved into their new location within pg_twophase/.  While  
reading such records, the WAL reader uses the callback  
read_local_xlog_page() to read a page, that is shared across various  
parts of the system.  This callback, since 1148e22a, has introduced an  
update of ThisTimeLineID when reading a record while in recovery, which  
is potentially helpful in the context of cascading WAL senders.  
  
This update of ThisTimeLineID interacts badly with the checkpointer if a  
promotion happens while some 2PC data is read from its record, as, by  
changing ThisTimeLineID, any follow-up WAL records would be written to  
an timeline older than the promoted one.  This results in consistency  
issues.  For instance, a subsequent server restart would cause a failure  
in finding a valid checkpoint record, resulting in a PANIC, for  
instance.  
  
This commit changes the code reading the 2PC data to reset the timeline  
once the 2PC record has been read, to prevent messing up with the static  
state of the checkpointer.  It would be tempting to do the same thing  
directly in read_local_xlog_page().  However, based on the discussion  
that has led to 1148e22a, users may rely on the updates of  
ThisTimeLineID when a WAL record page is read in recovery, so changing  
this callback could break some cases that are working currently.  
  
A TAP test reproducing the issue is added, relying on a PITR to  
precisely trigger a promotion with a prepared transaction still  
tracked.  
  
Per discussion with Heikki Linnakangas, Kyotaro Horiguchi, Fujii Masao  
and myself.  
  
Author: Soumyadeep Chakraborty, Jimmy Yih, Kevin Yeap  
Discussion: https://postgr.es/m/CAE-ML+_EjH_fzfq1F3RJ1=XaaNG=-Jz-i3JqkNhXiLAsM3z-Ew@mail.gmail.com  
Backpatch-through: 10  

commit   : 4b41f6923458ec736ff5d81e48229b88cf39b635    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 20 Mar 2021 12:47:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 20 Mar 2021 12:47:21 -0400    

While back-patching e0e569e1d, I noted that there were some other  
places where we ought to be applying DH_free(); namely, where we  
load some DH parameters from a file and then reject them as not  
being sufficiently secure.  While it seems really unlikely that  
anybody would hit these code paths in production, let alone do  
so repeatedly, let's fix it for consistency.  
  
Back-patch to v10 where this code was introduced.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 12354839e874c1f6ebc5e1e5ebc1c6b5519eeea0    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 22:21:58 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 22:21:58 -0400    

We leaked the error report from PQconninfoParse, when there was  
one.  It seems unlikely that real usage patterns would repeat  
the failure often enough to create serious bloat, but let's  
back-patch anyway to keep the code similar in all branches.  
  
Found via valgrind testing.  
Back-patch to v10 where this code was added.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 642b0b69b0638261fe81dfd9ca3b92927496f1cb    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 22:09:41 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 22:09:41 -0400    

Because guc.c prefers to keep all its string values in malloc'd  
not palloc'd storage, it has to be more careful than usual to  
avoid leaks.  Error exits out of string GUC hook checks failed  
to clear the proposed value string, and error exits out of  
ProcessGUCArray() failed to clear the malloc'd results of  
ParseLongOption().  
  
Found via valgrind testing.  
This problem is ancient, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : eba939551afe5709d15fe7779e7771ae0c8bf830    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 21:44:43 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 21:44:43 -0400    

The text search cache mechanisms assume that we can clean up  
an invalidated dictionary cache entry simply by resetting the  
associated long-lived memory context.  However, that does not work  
for ispell affixes that make use of regular expressions, because  
the regex library deals in plain old malloc.  Hence, we leaked  
compiled regex(es) any time we dropped such a cache entry.  That  
could quickly add up, since even a fairly trivial regex can use up  
tens of kB, and a large one can eat megabytes.  Add a memory context  
callback to ensure that a regex gets freed when its owning cache  
entry is cleared.  
  
Found via valgrind testing.  
This problem is ancient, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ea3989f3496c9c6a0c392680842518997d317e38    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 20:50:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 20:50:56 -0400    

Some code paths in this function perform syscache lookups, which  
can lead to table accesses and possibly leakage of cruft into  
the caller's context.  If said context is CacheMemoryContext,  
we eventually will have visible bloat.  But fixing this is no  
harder than moving one memory context switch step.  (The other  
callers don't have a problem.)  
  
Andres Freund and I independently found this via valgrind testing.  
Back-patch to v12 where this code was added.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

commit   : 5368369701447b4f2a767c34cfde1f9f4eb53f7a    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 20:37:09 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 20:37:09 -0400    

Although these lists are usually NIL, and even when not empty  
are unlikely to be large, constant relcache update traffic could  
eventually result in visible bloat of CacheMemoryContext.  
  
Found via valgrind testing.  
Back-patch to v10 where this field was added.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0c3079e3ef6956feefc9cc4f62ad1f9acc7c84e9    
  
author   : Magnus Hagander <[email protected]>    
date     : Thu, 18 Mar 2021 11:17:42 +0100    
  
committer: Magnus Hagander <[email protected]>    
date     : Thu, 18 Mar 2021 11:17:42 +0100    

pg_read_file() is the function that's in core, pg_file_read() is in  
adminpack. But when using pg_file_read() in adminpack it calls the *C*  
level function pg_read_file() in core, which probably threw the original  
author off. But the error hint should be about the SQL function.  
  
Reported-By: Sergei Kornilov  
Backpatch-through: 11  
Discussion: https://postgr.es/m/[email protected]  

commit   : b77e5d73bcfc0810e7a4eab29cfbc2859adb8db9    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 17 Mar 2021 16:10:37 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 17 Mar 2021 16:10:37 -0400    

Robert Foggia of Trustwave reported that read_tablespace_map()  
fails to prevent an overrun of its on-stack input buffer.  
Since the tablespace map file is presumed trustworthy, this does  
not seem like an interesting security vulnerability, but still  
we should fix it just in the name of robustness.  
  
While here, document that pg_basebackup's --tablespace-mapping option  
doesn't work with tar-format output, because it doesn't.  To make it  
work, we'd have to modify the tablespace_map file within the tarball  
sent by the server, which might be possible but I'm not volunteering.  
(Less-painful solutions would require changing the basebackup protocol  
so that the source server could adjust the map.  That's not very  
appetizing either.)  

commit   : 69739ec317aef5df0ce9258142b3124b6c58b202    
  
author   : Thomas Munro <[email protected]>    
date     : Thu, 18 Mar 2021 01:00:56 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Thu, 18 Mar 2021 01:00:56 +1300    

This reverts commit 4e0f0995e923948631c4114ab353b256b51b58ad.  
  
Discussion: https://postgr.es/m/CA%2BhUKGJmcqAE3MZeDCLLXa62cWM0AJbKmp2JrJYaJ86bz36LFA%40mail.gmail.com  

commit   : 4e0f0995e923948631c4114ab353b256b51b58ad    
  
author   : Thomas Munro <[email protected]>    
date     : Wed, 17 Mar 2021 17:46:39 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Wed, 17 Mar 2021 17:46:39 +1300    

With very unlucky timing and parallel_leader_participation off, PHJ  
could attempt to access per-batch state just as it was being freed.  
There was code intended to prevent that by checking for a cleared  
pointer, but it was buggy.  
  
Fix, by introducing an extra barrier phase.  The new phase  
PHJ_BUILD_RUNNING means that it's safe to access the per-batch state to  
find a batch to help with, and PHJ_BUILD_DONE means that it is too late.  
The last to detach will free the array of per-batch state as before, but  
now it will also atomically advance the phase at the same time, so that  
late attachers can avoid the hazard, without the data race.  This  
mirrors the way per-batch hash tables are freed (see phases  
PHJ_BATCH_PROBING and PHJ_BATCH_DONE).  
  
Revealed by a one-off build farm failure, where BarrierAttach() failed a  
sanity check assertion, because the memory had been clobbered by  
dsa_free().  
  
Back-patch to 11, where the code arrived.  
  
Reported-by: Michael Paquier <[email protected]>  
Discussion: https://postgr.es/m/20200929061142.GA29096%40paquier.xyz  

commit   : 4d072bf2a031f343ef796dac6d324d9a03121183    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 16 Mar 2021 16:02:49 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 16 Mar 2021 16:02:49 -0400    

After reading the root cert list from the ssl_ca_file, immediately  
install it as client CA list of the new SSL context.  That gives the  
SSL context ownership of the list, so that SSL_CTX_free will free it.  
This avoids a permanent memory leak if we fail further down in  
be_tls_init(), which could happen if bogus CRL data is offered.  
  
The leak could only amount to something if the CRL parameters get  
broken after server start (else we'd just quit) and then the server  
is SIGHUP'd many times without fixing the CRL data.  That's rather  
unlikely perhaps, but it seems worth fixing, if only because the  
code is clearer this way.  
  
While we're here, add some comments about the memory management  
aspects of this logic.  
  
Noted by Jelte Fennema and independently by Andres Freund.  
Back-patch to v10; before commit de41869b6 it doesn't matter,  
since we'd not re-execute this code during SIGHUP.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 6ed05993310703f2f216142dada48bc1f10868fb    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 12 Mar 2021 12:20:15 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 12 Mar 2021 12:20:15 -0500    

psql's editing commands decide whether the user has edited the file  
by checking for change of modification timestamp.  This is probably  
fine for a pre-existing file, but with a temporary file that is  
created within the command, it's possible for a fast typist to  
save-and-exit in less than the one-second granularity of stat(2)  
timestamps.  On Windows FAT filesystems the granularity is even  
worse, 2 seconds, making the race a bit easier to hit.  
  
To fix, try to set the temp file's mod time to be two seconds ago.  
It's unlikely this would fail, but then again the race condition  
itself is unlikely, so just ignore any error.  
  
Also, we might as well check the file size as well as its mod time.  
  
While this is a difficult bug to hit, it still seems worth  
back-patching, to ensure that users' edits aren't lost.  
  
Laurenz Albe, per gripe from Jacob Champion; based on fix suggestions  
from Jacob and myself  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 8a2297776667483643823e32fc037a675447d0bb    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 12 Mar 2021 11:08:42 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 12 Mar 2021 11:08:42 -0500    

GENERATED ALWAYS AS IDENTITY implies NOT NULL, but the code failed  
to complain if you overrode that with "GENERATED ALWAYS AS IDENTITY  
NULL".  One might think the old behavior was a feature, but it was  
inconsistent because the outcome varied depending on the order of  
the clauses, so it seems to have been just an oversight.  
  
Per bug #16913 from Pavel Boev.  Back-patch to v10 where identity  
columns were introduced.  
  
Vik Fearing (minor tweaks by me)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 3580b4a0cde03d39e47c70a2078c23d2c0a96c97    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 11 Mar 2021 14:43:45 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 11 Mar 2021 14:43:45 -0500    

Commit 92785dac2 copied some logic related to advancement of inStart  
from pqParseInput3 into getRowDescriptions and getAnotherTuple,  
because it wanted to allow user-defined row processor callbacks to  
potentially longjmp out of the library, and inStart would have to be  
updated before that happened to avoid an infinite loop.  We later  
decided that that API was impossibly fragile and reverted it, but  
we didn't undo all of the related code changes, and this bit of  
messiness survived.  Undo it now so that there's just one place in  
pqParseInput3's processing where inStart is advanced; this will  
simplify addition of better tracing support.  
  
getParamDescriptions had grown similar processing somewhere along  
the way (not in 92785dac2; I didn't track down just when), but it's  
actually buggy because its handling of corrupt-message cases seems to  
have been copied from the v2 logic where we lacked a known message  
length.  The cases where we "goto not_enough_data" should not simply  
return EOF, because then we won't consume the message, potentially  
creating an infinite loop.  That situation now represents a  
definitively corrupt message, and we should report it as such.  
  
Although no field reports of getParamDescriptions getting stuck in  
a loop have been seen, it seems appropriate to back-patch that fix.  
I chose to back-patch all of this to keep the logic looking more alike  
in supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 635048eb92e819d7a24202d74949e449ac5c7ef3    
  
author   : Peter Geoghegan <[email protected]>    
date     : Wed, 10 Mar 2021 22:10:34 -0800    
  
committer: Peter Geoghegan <[email protected]>    
date     : Wed, 10 Mar 2021 22:10:34 -0800    

Oversight in commit 9f3665fb.  
  
Backpatch: 13-, just like commit 9f3665fb.  

commit   : 4ecf359fbac5b4da749147ce873e3332a613fe5c    
  
author   : Bruce Momjian <[email protected]>    
date     : Wed, 10 Mar 2021 20:25:18 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Wed, 10 Mar 2021 20:25:18 -0500    

This is a follow-on patch to 92c12e46d5.  In that patch, we renamed  
"altitude" to "elevation" in the docs, based on these details:  
  
   https://mapscaping.com/blogs/geo-candy/what-is-the-difference-between-elevation-relief-and-altitude  
  
This renames the tutorial SQL files to match the documentation.  
  
Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

commit   : 1fc5a57386d11221258efa8af444a90f344a18a2    
  
author   : Peter Geoghegan <[email protected]>    
date     : Wed, 10 Mar 2021 17:07:55 -0800    
  
committer: Peter Geoghegan <[email protected]>    
date     : Wed, 10 Mar 2021 17:07:55 -0800    

vacuumlazy.c sometimes fails to update pg_class entries for each index  
(to ensure that pg_class.reltuples is current), even though analyze.c  
assumed that that must have happened during VACUUM ANALYZE.  There are  
at least a couple of reasons for this.  For example, vacuumlazy.c could  
fail to update pg_class when the index AM indicated that its statistics  
are merely an estimate, per the contract for amvacuumcleanup() routines  
established by commit e57345975cf back in 2006.  
  
Stop assuming that pg_class must have been updated with accurate  
statistics within VACUUM ANALYZE -- update pg_class for indexes at the  
same time as the table relation in all cases.  That way VACUUM ANALYZE  
will never fail to keep pg_class.reltuples reasonably accurate.  
  
The only downside of this approach (compared to the old approach) is  
that it might inaccurately set pg_class.reltuples for indexes whose heap  
relation ends up with the same inaccurate value anyway.  This doesn't  
seem too bad.  We already consistently called vac_update_relstats() (to  
update pg_class) for the heap/table relation twice during any VACUUM  
ANALYZE -- once in vacuumlazy.c, and once in analyze.c.  We now make  
sure that we call vac_update_relstats() at least once (though often  
twice) for each index.  
  
This is follow up work to commit 9f3665fb, which dealt with issues in  
btvacuumcleanup().  Technically this fixes an unrelated issue, though.  
btvacuumcleanup() no longer provides an accurate num_index_tuples value  
following commit 9f3665fb (when there was no btbulkdelete() call during  
the VACUUM operation in question), but hashvacuumcleanup() has worked in  
the same way for many years now.  
  
Author: Peter Geoghegan <[email protected]>  
Reviewed-By: Masahiko Sawada <[email protected]>  
Discussion: https://postgr.es/m/CAH2-WzknxdComjhqo4SUxVFk_Q1171GJO2ZgHZ1Y6pion6u8rA@mail.gmail.com  
Backpatch: 13-, just like commit 9f3665fb.  

commit   : 9663d124466f09696170631a2f2c0b40114166c8    
  
author   : Peter Geoghegan <[email protected]>    
date     : Wed, 10 Mar 2021 16:26:58 -0800    
  
committer: Peter Geoghegan <[email protected]>    
date     : Wed, 10 Mar 2021 16:26:58 -0800    

Remove the entire idea of "stale stats" within nbtree VACUUM (stop  
caring about stats involving the number of inserted tuples).  Also  
remove the vacuum_cleanup_index_scale_factor GUC/param on the master  
branch (though just disable them on postgres 13).  
  
The vacuum_cleanup_index_scale_factor/stats interface made the nbtree AM  
partially responsible for deciding when pg_class.reltuples stats needed  
to be updated.  This seems contrary to the spirit of the index AM API,  
though -- it is not actually necessary for an index AM's bulk delete and  
cleanup callbacks to provide accurate stats when it happens to be  
inconvenient.  The core code owns that.  (Index AMs have the authority  
to perform or not perform certain kinds of deferred cleanup based on  
their own considerations, such as page deletion and recycling, but that  
has little to do with pg_class.reltuples/num_index_tuples.)  
  
This issue was fairly harmless until the introduction of the  
autovacuum_vacuum_insert_threshold feature by commit b07642db, which had  
an undesirable interaction with the vacuum_cleanup_index_scale_factor  
mechanism: it made insert-driven autovacuums perform full index scans,  
even though there is no real benefit to doing so.  This has been tied to  
a regression with an append-only insert benchmark [1].  
  
Also have remaining cases that perform a full scan of an index during a  
cleanup-only nbtree VACUUM indicate that the final tuple count is only  
an estimate.  This prevents vacuumlazy.c from setting the index's  
pg_class.reltuples in those cases (it will now only update pg_class when  
vacuumlazy.c had TIDs for nbtree to bulk delete).  This arguably fixes  
an oversight in deduplication-related bugfix commit 48e12913.  
  
[1] https://smalldatum.blogspot.com/2021/01/insert-benchmark-postgres-is-still.html  
  
Author: Peter Geoghegan <[email protected]>  
Reviewed-By: Masahiko Sawada <[email protected]>  
Discussion: https://postgr.es/m/CAD21AoA4WHthN5uU6+WScZ7+J_RcEjmcuH94qcoUPuB42ShXzg@mail.gmail.com  
Backpatch: 13-, where autovacuum_vacuum_insert_threshold was added.  

commit   : 9a4e4af42023451944a5beca124d647dd5c2c26f    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 10 Mar 2021 11:33:50 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 10 Mar 2021 11:33:50 -0500    

Clarify the discussion in "User-Defined Procedures", by laying out  
the key differences between functions and procedures in a bulleted  
list.  Notably, this avoids burying the lede about procedures being  
able to do transaction control.  Make the back-link in the CREATE  
FUNCTION reference page more prominent, and add one in CREATE  
PROCEDURE.  
  
Per gripe from Guyren Howe.  Thanks to David Johnston for discussion.  
  
Discussion: https://postgr.es/m/BYAPR03MB4903C53A8BB7EFF5EA289674A6949@BYAPR03MB4903.namprd03.prod.outlook.com  

commit   : fe2b5386b2edb7d454fcab36bb3dfbbe272d362f    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 8 Mar 2021 18:21:51 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 8 Mar 2021 18:21:51 -0500    

"SELECT pg_import_system_collations(0)" caused an assertion failure.  
With a random nonzero argument --- or indeed with zero, in non-assert  
builds --- it would happily make pg_collation entries with garbage  
values of collnamespace.  These are harmless as far as I can tell  
(unless maybe the OID happens to become used for a schema, later on?).  
In any case this isn't a security issue, since the function is  
superuser-only.  But it seems like a gotcha for unwary DBAs, so let's  
add a check that the given OID belongs to some schema.  
  
Back-patch to v10 where this function was introduced.  

commit   : 21d5a065fd5f0ed71e0f6726a869c64d13716ceb    
  
author   : Amit Kapila <[email protected]>    
date     : Wed, 3 Mar 2021 10:17:47 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Wed, 3 Mar 2021 10:17:47 +0530    

It was not clear in the docs that the max_replication_slots is also used  
to track replication origins on the subscriber side.  
  
Author: Paul Martinez  
Reviewed-by: Amit Kapila  
Backpatch-through: 10 where logical replication was introduced  
Discussion: https://postgr.es/m/CACqFVBZgwCN_pHnW6dMNCrOS7tiHCw6Retf_=U2Vvj3aUSeATw@mail.gmail.com  

commit   : b52fd1e7c76c21298d2e2ecc006c5a2d99b46da9    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 2 Mar 2021 15:39:34 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 2 Mar 2021 15:39:34 -0300    

On Windows, CMD.EXE allegedly does not run a command that uses forward slashes,  
so let's convert the path to use backslashes instead.  
  
Backpatch to 10.  
  
Author: Nitin Jadhav <[email protected]>  
Reviewed-by: Juan José Santamaría Flecha <[email protected]>  
Discussion: https://postgr.es/m/CAMm1aWaNDuaPYFYMAqDeJrZmPtNvLcJRS++CcZWY8LT6KcoBZw@mail.gmail.com  

commit   : 621e8a603d5668467eac50ff13ce8a9006fd56be    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 2 Mar 2021 13:19:07 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 2 Mar 2021 13:19:07 +0900    

The same test for REINDEX (VERBOSE) was done twice, while it is clear  
that the second test should use --concurrently.  Issue introduced in  
5dc92b8, for what looks like a copy-paste mistake.  
  
Reviewed-by: Mark Dilger  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

commit   : 2688852a49ea52e5663c09f91cdcf43697e10814    
  
author   : Alvaro Herrera <[email protected]>    
date     : Sat, 27 Feb 2021 18:09:15 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Sat, 27 Feb 2021 18:09:15 -0300    

AfterTriggerSaveEvent() wrongly allocates the slot in execution-span  
memory context, whereas the correct thing is to allocate it in  
a transaction-span context, because that's where the enclosing  
AfterTriggersTableData instance belongs into.  
  
Backpatch to 12 (the test back to 11, where it works well with no code  
changes, and it's good to have to confirm that the case was previously  
well supported); this bug seems introduced by commit ff11e7f4b9ae.  
  
Reported-by: Bertrand Drouvot <[email protected]>  
Author: Amit Langote <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : 57449318307a3eaa02d129869293e887c5b48ab0    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 26 Feb 2021 15:24:00 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 26 Feb 2021 15:24:00 -0500    

Break the synopsis into named parts to make it less confusing.  
Make more than zero effort at applying SGML markup.  Do a bit  
of copy-editing of nearby text.  
  
The synopsis revision is by Alvaro Herrera and Paul Förster,  
the rest is my fault.  Back-patch to v10 where multi-host  
connection strings appeared.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 49076fd3ba6ba7cda2ac16d9aaf554025f0bba2b    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 25 Feb 2021 20:47:32 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 25 Feb 2021 20:47:32 -0500    

makeDependencyGraphWalker and checkWellFormedRecursionWalker  
thought they could hold onto a pointer to a list's first  
cons cell while the list was modified by recursive calls.  
That was okay when the cons cell was actually separately  
palloc'd ... but since commit 1cff1b95a, it's quite unsafe,  
leading to core dumps or incorrect complaints of faulty  
WITH nesting.  
  
In the field this'd require at least a seven-deep WITH nest  
to cause an issue, but enabling DEBUG_LIST_MEMORY_USAGE  
allows the bug to be seen with lesser nesting depths.  
  
Per bug #16801 from Alexander Lakhin.  Back-patch to v13.  
  
Michael Paquier and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 1f56ae322943e9c9b3be4b2c480891a291861bf7    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 25 Feb 2021 16:07:03 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 25 Feb 2021 16:07:03 +0900    

PGHOST, PGPORT and PGUSER were already mentioned, but not PGDATABASE.  
Like 5aaa584, backpatch down to 12.  
  
Reported-by: Christophe Courtois  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

commit   : 9de839fb4ab08aa5e5a640b7b47c5fc9908453d3    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 24 Feb 2021 16:13:56 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 24 Feb 2021 16:13:56 +0900    

The portions fixing the documentation are backpatched where needed.  
  
Author: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
backpatch-through: 9.6  

commit   : 28f4b61083b2a5b2bd1b7859d7cb2c6cfcb2b964    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 23 Feb 2021 17:30:21 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 23 Feb 2021 17:30:21 -0300    

Commit 866e24d47db1 added an assert that HEAP_XMAX_LOCK_ONLY and  
HEAP_KEYS_UPDATED cannot appear together, on the faulty assumption that  
the latter necessarily referred to an update and not a tuple lock; but  
that's wrong, because SELECT FOR UPDATE can use precisely that  
combination, as evidenced by the amcheck test case added here.  
  
Remove the Assert(), and also patch amcheck's verify_heapam.c to not  
complain if the combination is found.  Also, out of overabundance of  
caution, update (across all branches) README.tuplock to be more explicit  
about this.  
  
Author: Julien Rouhaud <[email protected]>  
Reviewed-by: Mahendra Singh Thalor <[email protected]>  
Reviewed-by: Dilip Kumar <[email protected]>  
Discussion: https://postgr.es/m/20210124061758.GA11756@nol  

commit   : 186f6168b73de6472e8aeb6e96b2ec6c73fd7b89    
  
author   : Magnus Hagander <[email protected]>    
date     : Mon, 22 Feb 2021 13:00:54 +0100    
  
committer: Magnus Hagander <[email protected]>    
date     : Mon, 22 Feb 2021 13:00:54 +0100    

Building the docs with STYLE=website referenced a stylesheet that long  
longer exists on the website, since we changed it to use versioned  
references.  
  
To make it less likely for this to happen again, point to a single  
stylesheet on the website which will in turn import the required one.  
That puts the process entirely within the scope of the website  
repository, so next time a version is switched that's the only place  
changes have to be made, making them less likely to be missed.  
  
Per (off-list) discussion with Peter Geoghegan and Jonathan Katz.  

commit   : 5190ce845c59727ed7177dadd62626d6450a55bd    
  
author   : Thomas Munro <[email protected]>    
date     : Mon, 22 Feb 2021 14:37:28 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Mon, 22 Feb 2021 14:37:28 +1300    

Commit b09ff536 left behind some outdated advice in the long_desc field  
of the GUC "effective_io_concurrency".  Remove it.  
  
Back-patch to 13.  
  
Reported-by: Andrew Gierth <[email protected]>  
Reviewed-by: Julien Rouhaud <[email protected]>  
Discussion: https://postgr.es/m/CA%2BhUKGJyyWqFBxL9gEj-qtjBThGjhAOBE8GBnF8MUJOJ3vrfag%40mail.gmail.com  

commit   : be7485a1e31122f92b6dd9ad02427c1daa870b63    
  
author   : Fujii Masao <[email protected]>    
date     : Fri, 19 Feb 2021 22:01:25 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Fri, 19 Feb 2021 22:01:25 +0900    

When ON_ERROR_ROLLBACK is enabled, psql releases a temporary savepoint  
if it's idle in a valid transaction block after executing a query. But psql  
doesn't do that after RELEASE or ROLLBACK is executed because a temporary  
savepoint has already been destroyed in that case.  
  
This commit changes psql's ON_ERROR_ROLLBACK so that it doesn't release  
a temporary savepoint also when COMMIT AND CHAIN is executed. A temporary  
savepoint doesn't need to be released in that case because  
COMMIT AND CHAIN also destroys any savepoints defined within the transaction  
to commit. Otherwise psql tries to release the savepoint that  
COMMIT AND CHAIN has already destroyed and cause an error  
"ERROR:  savepoint "pg_psql_temporary_savepoint" does not exist".  
  
Back-patch to v12 where transaction chaining was added.  
  
Reported-by: Arthur Nascimento  
Author: Arthur Nascimento  
Reviewed-by: Fujii Masao, Vik Fearing  
Discussion: https://postgr.es/m/[email protected]  

commit   : 422012c98f8d929f9aa2b2e706b29512f61544e1    
  
author   : Fujii Masao <[email protected]>    
date     : Fri, 19 Feb 2021 21:57:52 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Fri, 19 Feb 2021 21:57:52 +0900    

This commit fixes COMMIT AND CHAIN command so that it starts new transaction  
immediately even if savepoints are defined within the transaction to commit.  
Previously COMMIT AND CHAIN command did not in that case because  
commit 280a408b48 forgot to make CommitTransactionCommand() handle  
a transaction chaining when the transaction state was TBLOCK_SUBCOMMIT.  
  
Also this commit adds the regression test for COMMIT AND CHAIN command  
when savepoints are defined.  
  
Back-patch to v12 where transaction chaining was added.  
  
Reported-by: Arthur Nascimento  
Author: Fujii Masao  
Reviewed-by: Arthur Nascimento, Vik Fearing  
Discussion: https://postgr.es/m/[email protected]  

commit   : bf9d3a5f847e91154b7cde255da7f24cd129ff35    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Feb 2021 22:38:55 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Feb 2021 22:38:55 -0500    

While poking at the regex code, I happened to notice that the bug  
squashed in commit afcc8772e had a sibling: next() failed to return  
a specific value associated with the '}' token for a "\{m,n\}"  
quantifier when parsing in basic RE mode.  Again, this could result  
in treating the quantifier as non-greedy, which it never should be in  
basic mode.  For that to happen, the last character before "\}" that  
sets "nextvalue" would have to set it to zero, or it'd have to have  
accidentally been zero from the start.  The failure can be provoked  
repeatably with, for example, a bound ending in digit "0".  
  
Like the previous patch, back-patch all the way.  

commit   : d4b667e9353a70c4ee2847603e5ad2e14e20f82e    
  
author   : Fujii Masao <[email protected]>    
date     : Thu, 18 Feb 2021 23:28:58 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Thu, 18 Feb 2021 23:28:58 +0900    

Commit 2c8dd05d6c added the atomic variable writtenUpto into  
walreceiver's shared memory information. It's initialized only  
when walreceiver started up but could be read via pg_stat_wal_receiver  
view anytime, i.e., even before it's initialized. In the server built  
with --disable-atomics and --disable-spinlocks, this uninitialized  
atomic variable read could cause "invalid spinlock number: 0" error.  
  
This commit changed writtenUpto so that it's initialized at  
the postmaster startup, to avoid the uninitialized variable read  
via pg_stat_wal_receiver and fix the error.  
  
Also this commit moved the read of writtenUpto after the release  
of spinlock protecting walreceiver's shared variables. This is  
necessary to prevent new spinlock from being taken by atomic  
variable read while holding another spinlock, and to shorten  
the spinlock duration. This change leads writtenUpto not to be  
consistent with the other walreceiver's shared variables protected  
by a spinlock. But this is OK because writtenUpto should not be  
used for data integrity checks.  
  
Back-patch to v13 where commit 2c8dd05d6c introduced the bug.  
  
Author: Fujii Masao  
Reviewed-by: Michael Paquier, Thomas Munro, Andres Freund  
Discussion: https://postgr.es/m/[email protected]  

commit   : daf2e708edfbc0741f8a229a0c30fdd0168b525e    
  
author   : Magnus Hagander <[email protected]>    
date     : Wed, 17 Feb 2021 13:53:26 +0100    
  
committer: Magnus Hagander <[email protected]>    
date     : Wed, 17 Feb 2021 13:53:26 +0100    

Author: Daniel Gustafsson <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0d779d22a290a89b6c892137a37280b9588ad0cc    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 16 Feb 2021 12:07:14 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 16 Feb 2021 12:07:14 -0500    

Commit 2f2007fbb did this partially, but there were two remaining  
warts.  checkcondition_gin handled some uncertain cases by setting  
the out-of-band recheck flag, some by returning TS_MAYBE, and some  
by doing both.  Meanwhile, TS_execute arbitrarily converted a  
TS_MAYBE result to TS_YES.  Thus, if checkcondition_gin chose to  
only return TS_MAYBE, the outcome would be TS_YES with no recheck  
flag, potentially resulting in wrong query outputs.  
  
The case where this'd happen is if there were GIN_MAYBE entries  
in the indexscan results passed to gin_tsquery_[tri]consistent,  
which so far as I can see would only happen if the tidbitmap used  
to accumulate indexscan results grew large enough to become lossy.  
  
I initially thought of fixing this by ensuring we always set the  
recheck flag as well as returning TS_MAYBE in uncertain cases.  
But that errs in the other direction, potentially forcing rechecks  
of rows that provably match the query (since the recheck flag  
remains set even if TS_execute later finds that the answer must be  
TS_YES).  Instead, let's get rid of the out-of-band recheck flag  
altogether and rely on returning TS_MAYBE.  This requires exporting  
a version of TS_execute that will actually return the full ternary  
result of the evaluation ... but we likely should have done that  
to start with.  
  
Unfortunately it doesn't seem practical to add a regression test case  
that covers this: the amount of data needed to cause the GIN bitmap to  
become lossy results in a longer runtime than I think we want to have  
in the tests.  (I'm wondering about allowing smaller work_mem settings  
to ameliorate that, but it'd be a matter for a separate patch.)  
  
Per bug #16865 from Dimitri Nüscheler.  Back-patch to v13 where  
the faulty commit came in.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 80dc07aa361b9b0028e49bbdf7a947775211b812    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 15 Feb 2021 10:17:58 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 15 Feb 2021 10:17:58 -0500    

The inner loop in switchToPresortedPrefixMode() can be implemented  
as a conventional integer-counter for() loop, removing a couple of  
redundant boolean state variables.  The old logic here was a remnant  
of earlier development, but as things now stand there's no reason  
for extra complexity.  
  
Also, annotate the test case added by 82e0e2930 to explain why it  
manages to hit the corner case fixed in that commit, and add an  
EXPLAIN to verify that it's creating an incremental-sort plan.  
  
Back-patch to v13, like the previous patch.  
  
James Coleman and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 18cacf89b9fe5523941b57fbd01d408585e70737    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Mon, 15 Feb 2021 09:28:08 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Mon, 15 Feb 2021 09:28:08 +0200    

If ExecGetInsertedCols(), ExecGetUpdatedCols() or ExecGetExtraUpdatedCols()  
were called with a ResultRelInfo that's not in the range table and isn't a  
partition routing target, the functions would dereference a NULL pointer,  
relinfo->ri_RootResultRelInfo. Such ResultRelInfos are created when firing  
RI triggers in tables that are not modified directly. None of the current  
callers of these functions pass such relations, so this isn't a live bug,  
but let's make them more robust.  
  
Also update comment in ResultRelInfo; after commit 6214e2b228,  
ri_RangeTableIndex is zero for ResultRelInfos created for partition tuple  
routing.  
  
Noted by Coverity. Backpatch down to v11, like commit 6214e2b228.  
  
Reviewed-by: Tom Lane, Amit Langote  

commit   : 6c23e5ae9ee12ff1f5183573885bfaa4eb97b243    
  
author   : Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 15:43:39 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 15:43:39 +1300    

FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to  
choose open_datasync as its default value.  That may not be a good  
choice for all systems, and performs worse than fdatasync in some  
scenarios.  Let's preserve the existing default behavior for now.  
  
Like commit 576477e73c4, which did the same for Linux, back-patch to all  
supported releases.  
  
Discussion: https://postgr.es/m/CA%2BhUKGLsAMXBQrCxCXoW-JsUYmdOL8ALYvaX%3DCrHqWxm-nWbGA%40mail.gmail.com  

commit   : 9fe40913c45dcb78d3271fdc2dcf21ff15bee583    
  
author   : Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 13:32:58 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 13:32:58 +1300    

While cleaning up after a parallel query or parallel index creation that  
created temporary files, we could be interrupted by a statement timeout.  
The error handling path would then fail to clean up the files when it  
ran dsm_detach() again, because the callback was already popped off the  
list.  Prevent this hazard by holding interrupts while the cleanup code  
runs.  
  
Thanks to Heikki Linnakangas for this suggestion, and also to Kyotaro  
Horiguchi, Masahiko Sawada, Justin Pryzby and Tom Lane for discussion of  
this and earlier ideas on how to fix the problem.  
  
Back-patch to all supported releases.  
  
Reported-by: Justin Pryzby <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : c86eae39ce1e5f842bbf75e381d903de43ad4c80    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 13 Feb 2021 17:49:08 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 13 Feb 2021 17:49:08 -0500    

Modern gcc and clang compilers offer alignment sanitizers, which help to detect  
pointer misalignment.  However, our codebase already contains x86-specific  
crc32 computation code, which uses unalignment access.  Thankfully, those  
compilers also support the attribute, which disables alignment sanitizers at  
the function level.  This commit adds pg_attribute_no_sanitize_alignment(),  
which wraps this attribute, and applies it to pg_comp_crc32c_sse42() function.  
  
Back-patch of commits 993bdb9f9 and ad2ad698a, to enable doing  
alignment testing in all supported branches.  
  
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com  
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us  
Author: Alexander Korotkov, revised by Tom Lane  
Reviewed-by: Tom Lane  

commit   : bcd5e95754bff330151dacbf7db5140d01aabe3a    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 13 Feb 2021 16:06:34 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 13 Feb 2021 16:06:34 +0900    

This grammar flavor has been added by 5fc7039.  
  
Author: Ian Lawrence Barwick  
Discussion: https://postgr.es/m/CAB8KJ=ii6JScodxkA6-DO8bjatsMYU3OcewnL0mdN9geR+tTaw@mail.gmail.com  
Backpatch-through: 13  

commit   : 3a02d68a96edb54d981879b95c28fe4ba79b87f9    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 12 Feb 2021 16:26:47 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 12 Feb 2021 16:26:47 -0500    

Given a regex pattern with a very long fixed prefix (approaching 500  
characters), the result of pow(FIXED_CHAR_SEL, fixed_prefix_len) can  
underflow to zero.  Typically the preceding selectivity calculation  
would have underflowed as well, so that we compute 0/0 and get NaN.  
In released branches this leads to an assertion failure later on.  
That doesn't happen in HEAD, for reasons I've not explored yet,  
but it's surely still a bug.  
  
To fix, just skip the division when the pow() result is zero, so  
that we'll (most likely) return a zero selectivity estimate.  In  
the edge cases where "sel" didn't yet underflow, perhaps this  
isn't desirable, but I'm not sure that the case is worth spending  
a lot of effort on.  The results of regex_selectivity_sub() are  
barely worth the electrons they're written on anyway :-(  
  
Per report from Alexander Lakhin.  Back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : c6cd20d91ce9b100b89997757b88a415c3e10747    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 10 Feb 2021 16:59:04 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 10 Feb 2021 16:59:04 +0900    

Oversight in bd12080.  
  
Reported-by: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

commit   : 8493831385814c4f22b1d5b5d8a9227a2eb82712    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 10 Feb 2021 13:09:09 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 10 Feb 2021 13:09:09 +0900    

For an index, attstattarget can be updated using ALTER INDEX SET  
STATISTICS.  This data was lost on the new index after REINDEX  
CONCURRENTLY.  
  
The update of this field is done when the old and new indexes are  
swapped to make the fix back-patchable.  Another approach we could look  
after in the long-term is to change index_create() to pass the wanted  
values of attstattarget when creating the new relation, but, as this  
would cause an ABI breakage this can be done only on HEAD.  
  
Reported-by: Ronan Dunklau  
Author: Michael Paquier  
Reviewed-by: Ronan Dunklau, Tomas Vondra  
Discussion: https://postgr.es/m/16628084.uLZWGnKmhe@laptop-ronand  
Backpatch-through: 12