PostgreSQL 9.6.22 commit log

commit   : 836dda6f1b31c5f7c70ebf52537364ffc3eff577    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 16:50:15 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 16:50:15 -0400    

commit   : cc6c63f8a26d9706bc4b256ac075401bb3d5f759    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 13:10:30 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 13:10:30 -0400    

Security: CVE-2021-32027, CVE-2021-32028, CVE-2021-32029  

commit   : 0fcb8e2e0154dedea5c3c7da6dd2cffb731aac06    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 11:02:30 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 11:02:30 -0400    

It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE  
list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present.  
If it happens, the ON CONFLICT UPDATE code path would end up storing  
tuples that include the values of the extra resjunk columns.  That's  
fairly harmless in the short run, but if new columns are added to  
the table then the values would become accessible, possibly leading  
to malfunctions if they don't match the datatypes of the new columns.  
  
This had escaped notice through a confluence of missing sanity checks,  
including  
  
* There's no cross-check that a tuple presented to heap_insert or  
heap_update matches the table rowtype.  While it's difficult to  
check that fully at reasonable cost, we can easily add assertions  
that there aren't too many columns.  
  
* The output-column-assignment cases in execExprInterp.c lacked  
any sanity checks on the output column numbers, which seems like  
an oversight considering there are plenty of assertion checks on  
input column numbers.  Add assertions there too.  
  
* We failed to apply nodeModifyTable's ExecCheckPlanOutput() to  
the ON CONFLICT UPDATE tlist.  That wouldn't have caught this  
specific error, since that function is chartered to ignore resjunk  
columns; but it sure seems like a bad omission now that we've seen  
this bug.  
  
In HEAD, the right way to fix this is to make the processing of  
ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists  
now do, that is don't add "SET x = x" entries, and use  
ExecBuildUpdateProjection to evaluate the tlist and combine it with  
old values of the not-set columns.  This adds a little complication  
to ExecBuildUpdateProjection, but allows removal of a comparable  
amount of now-dead code from the planner.  
  
In the back branches, the most expedient solution seems to be to  
(a) use an output slot for the ON CONFLICT UPDATE projection that  
actually matches the target table, and then (b) invent a variant of  
ExecBuildProjectionInfo that can be told to not store values resulting  
from resjunk columns, so it doesn't try to store into nonexistent  
columns of the output slot.  (We can't simply ignore the resjunk columns  
altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.)  
This works back to v10.  In 9.6, projections work much differently and  
we can't cheaply give them such an option.  The 9.6 version of this  
patch works by inserting a JunkFilter when it's necessary to get rid  
of resjunk columns.  
  
In addition, v11 and up have the reverse problem when trying to  
perform ON CONFLICT UPDATE on a partitioned table.  Through a  
further oversight, adjust_partition_tlist() discarded resjunk columns  
when re-ordering the ON CONFLICT UPDATE tlist to match a partition.  
This accidentally prevented the storing-bogus-tuples problem, but  
at the cost that MULTIEXPR_SUBLINK cases didn't work, typically  
crashing if more than one row has to be updated.  Fix by preserving  
resjunk columns in that routine.  (I failed to resist the temptation  
to add more assertions there too, and to do some minor code  
beautification.)  
  
Per report from Andres Freund.  Back-patch to all supported branches.  
  
Security: CVE-2021-32028  

commit   : 0c1caa48d3ccb7a5d1343b53aa32fcae45dc2d00    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 10:44:38 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 10 May 2021 10:44:38 -0400    

While we were (mostly) careful about ensuring that the dimensions of  
arrays aren't large enough to cause integer overflow, the lower bound  
values were generally not checked.  This allows situations where  
lower_bound + dimension overflows an integer.  It seems that that's  
harmless so far as array reading is concerned, except that array  
elements with subscripts notionally exceeding INT_MAX are inaccessible.  
However, it confuses various array-assignment logic, resulting in a  
potential for memory stomps.  
  
Fix by adding checks that array lower bounds aren't large enough to  
cause lower_bound + dimension to overflow.  (Note: this results in  
disallowing cases where the last subscript position would be exactly  
INT_MAX.  In principle we could probably allow that, but there's a lot  
of code that computes lower_bound + dimension and would need adjustment.  
It seems doubtful that it's worth the trouble/risk to allow it.)  
  
Somewhat independently of that, array_set_element() was careless  
about possible overflow when checking the subscript of a fixed-length  
array, creating a different route to memory stomps.  Fix that too.  
  
Security: CVE-2021-32027  

commit   : 28a1164ad01616157e38a117e26b281c5fc060ff    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 10 May 2021 14:24:16 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 10 May 2021 14:24:16 +0200    

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 9ff8d81b53760d6603761384e52e7c643cf88b3a  

commit   : 727a27f5a371cbb5d0da50a4a4d8fdb5e9074034    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 9 May 2021 13:31:40 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 9 May 2021 13:31:40 -0400    

commit   : f760137d441b10fea341fc1af8bb982db88c9d9f    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 6 May 2021 17:17:56 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 6 May 2021 17:17:56 -0400    

Backpatch all the way back to 9.6.  
  
Author: Simon Riggs <[email protected]>  
Discussion: https://postgr.es/m/CANbhV-EwxvdhHuOLdfG2ciYrHOHXV=mm6=fD5aMhqcH09Li3Tg@mail.gmail.com  

commit   : 8f65db5ecf9ba235f2b5804ab6c788a310e31452    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:37:57 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:37:57 -0400    

While we've always allowed such cases, the documentation didn't  
say you could do it.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 2033d108ee6b7328a17175645bc86c127279bca0    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:10:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 15:10:06 -0400    

Mention specifically that you can't call aggregates, window functions,  
or procedures this way (the inability to call SRFs was already  
mentioned).  
  
Also, the claim that PQfn doesn't support NULL arguments or results  
has been a lie since we invented protocol 3.0.  Not sure why this  
text was never updated for that, but do it now.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 73bad52a93d61785d20a511f1e0ef9e0c2a518e6    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 14:10:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 30 Apr 2021 14:10:26 -0400    

Reject aggregates, window functions, and procedures.  Aggregates  
failed anyway, though with a somewhat obscure error message.  
Window functions would hit an Assert or null-pointer dereference.  
Procedures seemed to work as long as you didn't try to do  
transaction control, but (a) transaction control is sort of the  
point of a procedure, and (b) it's not entirely clear that no  
bugs lurk in that path.  Given the lack of testing of this area,  
it seems safest to be conservative in what we support.  
  
Also reject proretset functions, as the fastpath protocol can't  
support returning a set.  
  
Also remove an easily-triggered assertion that the given OID  
isn't 0; the subsequent lookups can handle that case themselves.  
  
Per report from Theodor-Arsenij Larionov-Trichkin.  
Back-patch to all supported branches.  (The procedure angle  
only applies in v11+, of course.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 54a23307193c30197ba89d680bb25a72c1fd8384    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 29 Apr 2021 15:24:38 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 29 Apr 2021 15:24:38 -0400    

Commits 29aeda6e4 et al closed up some oversights involving not checking  
for non-upgradable types within container types, such as arrays and  
ranges.  However, I only looked at version.c, failing to notice that  
there were substantially-equivalent tests in check.c.  (The division  
of responsibility between those files is less than clear...)  
  
In addition, because genbki.pl does not guarantee that auto-generated  
rowtype OIDs will hold still across versions, we need to consider that  
the composite type associated with a system catalog or view is  
non-upgradable.  It seems unlikely that someone would have a user  
column declared that way, but if they did, trying to read it in another  
PG version would likely draw "no such pg_type OID" failures, thanks  
to the type OID embedded in composite Datums.  
  
To support the composite and reg*-type cases, extend the recursive  
query that does the search to allow any base query that returns  
a column of pg_type OIDs, rather than limiting it to exactly one  
starting type.  
  
As before, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : f6171e6843f028aff27edd8f5e54bb7777d80045    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 28 Apr 2021 10:03:28 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 28 Apr 2021 10:03:28 -0400    

Somehow I'd convinced myself that rotating to UTC-12 was the way  
to do this, but upon further review, it's definitely UTC+12.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0d05a3a1df4f0f6ab3341dae77da633f2df75cec    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:59:00 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:59:00 +0900    

Spotted by buildfarm member prion, with -DRELCACHE_FORCE_RELEASE.  
  
Introduced in f7aab36.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : 6e41ff0562d3e1a5a88c826b0aa0b8f20bc92763    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:18:33 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 28 Apr 2021 11:18:33 +0900    

Attempting to use this function with event triggers failed, as, since  
its introduction in a676201, this code has never associated an object  
name with event triggers.  This addresses the failure by adding the  
event trigger name to the set defining its object address.  
  
Note that regression tests are added within event_trigger and not  
object_address to avoid issues with concurrent connections in parallel  
schedules.  
  
Author: Joel Jacobson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : b391db4943dce0688635b9e6550af22a4dac96ed    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 26 Apr 2021 11:50:35 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 26 Apr 2021 11:50:35 -0400    

For some reason, the "julian" option for extract()/date_part() has  
never gotten listed in the manual.  Also, while Appendix B mentioned  
in passing that we don't conform to the usual astronomical definition  
that a Julian date starts at noon UTC, it was kind of vague about what  
we do instead.  Clarify that, and add an example showing how to get  
the astronomical definition if you want it.  
  
It's been like this for ages, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 6cb7dcdb03d76f93411e0e810aba13b6fe4c9a43    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 11:12:04 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 11:12:04 -0400    

commit   : 1d997cb3753253f2527aff398c2badbcda830588    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 10:21:22 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 21 Apr 2021 10:21:22 -0400    

This has been found to cause hangs where tcp usage is forced.  
  
Alexey Kodratov  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch to all live branches  

commit   : d48212c45202ba9ae1ff1a68daad67b53da99dc4    
  
author   : Andrew Dunstan <[email protected]>    
date     : Fri, 16 Apr 2021 16:54:04 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Fri, 16 Apr 2021 16:54:04 -0400    

In order to avoid getting old logfile contents certain functions in  
PostgresNode were doing one of two things. On Windows it rotated the  
logfile and restarted the server, while elsewhere it truncated the log  
file. Both of these are unnecessary. We borrow from the buildfarm which  
does this instead: note the size of the logfile before we start, and  
then when fetching the logfile skip to that position before accumulating  
contents. This is spelled differently on Windows but the effect is the  
same. This is largely centralized in TestLib's slurp_file function,  
which has a new optional parameter, the offset to skip to before  
starting to reading the file. Code in the client becomes much neater.  
  
Backpatch to all live branches.  
  
Michael Paquier, slightly modified by me.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 041f4efd25e2438c323d298d797ff0a48f27eed0    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 15:10:18 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 13 Apr 2021 15:10:18 -0400    

Most GUC check hooks that inspect database state have special checks  
that prevent them from throwing hard errors for state-dependent issues  
when source == PGC_S_TEST.  This allows, for example,  
"ALTER DATABASE d SET default_text_search_config = foo" when the "foo"  
configuration hasn't been created yet.  Without this, we have problems  
during dump/reload or pg_upgrade, because pg_dump has no idea about  
possible dependencies of GUC values and can't ensure a safe restore  
ordering.  
  
However, check_role() and check_session_authorization() hadn't gotten  
the memo about that, and would throw hard errors anyway.  It's not  
entirely clear what is the use-case for "ALTER ROLE x SET role = y",  
but we've now heard two independent complaints about that bollixing  
an upgrade, so apparently some people are doing it.  
  
Hence, fix these two functions to act more like other check hooks  
with similar needs.  (But I did not change their insistence on  
being inside a transaction, as it's still not apparent that setting  
either GUC from the configuration file would be wise.)  
  
Also fix check_temp_buffers, which had a different form of the disease  
of making state-dependent checks without any exception for PGC_S_TEST.  
A cursory survey of other GUC check hooks did not find any more issues  
of this ilk.  (There are a lot of interdependencies among  
PGC_POSTMASTER and PGC_SIGHUP GUCs, which may be a bad idea, but  
they're not relevant to the immediate concern because they can't be  
set via ALTER ROLE/DATABASE.)  
  
Per reports from Charlie Hornsby and Nathan Bossart.  Back-patch  
to all supported branches.  
  
Discussion: https://postgr.es/m/HE1P189MB0523B31598B0C772C908088DB7709@HE1P189MB0523.EURP189.PROD.OUTLOOK.COM  
Discussion: https://postgr.es/m/[email protected]  

commit   : f488d19f3e32bc626693af488b3a48c1d480bb95    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:41 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:41 -0700    

With the Oracle Developer Studio 12.6 compiler, #line directives alter  
the current source file location for purposes of #include "..."  
directives.  Hence, a VPATH build failed with 'cannot find include file:  
"specscanner.c"'.  With two exceptions, parser-containing directories  
already add "-I. -I$(srcdir)"; eliminate the exceptions.  Back-patch to  
9.6 (all supported versions).  

commit   : 14652a19feab4d1cf0c637b896f89427d05b9c91    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:21 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 12 Apr 2021 19:24:21 -0700    

It doesn't support "\(foo\)*" like a POSIX "sed" implementation does;  
see the Autoconf manual.  Back-patch to 9.6 (all supported versions).  

commit   : a6158a4d9c2e6b8af0d4217b8d80afeff8adec0e    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 12 Apr 2021 14:37:22 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 12 Apr 2021 14:37:22 -0400    

There are hacks in parse_coerce.c to push down a requested coercion  
to below any CollateExpr that may appear.  However, we did that even  
if the requested data type is non-collatable, leading to an invalid  
expression tree in which CollateExpr is applied to a non-collatable  
type.  The fix is just to drop the CollateExpr altogether, reasoning  
that it's useless.  
  
This bug is ten years old, dating to the original addition of  
COLLATE support.  The lack of field complaints suggests that there  
aren't a lot of user-visible consequences.  We noticed the problem  
because it would trigger an assertion in DefineVirtualRelation if  
the invalid structure appears as an output column of a view; however,  
in a non-assert build, you don't see a crash just a (subtly incorrect)  
complaint about applying collation to a non-collatable type.  I found  
that by putting the incorrect structure further down in a view, I could  
make a view definition that would fail dump/reload, per the added  
regression test case.  But CollateExpr doesn't do anything at run-time,  
so this likely doesn't lead to any really exciting consequences.  
  
Per report from Yulin Pei.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com  

commit   : 6540322fad8bc32e4d4eb38023d7b6cd8ef1b597    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 12 Apr 2021 11:31:46 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 12 Apr 2021 11:31:46 +0900    

Using Roman numbers (via "RM" or "rm") for a conversion to calculate a  
number of months has never considered the case of negative numbers,  
where a conversion could easily cause out-of-bound memory accesses.  The  
conversions in themselves were not completely consistent either, as  
specifying 12 would result in NULL, but it should mean XII.  
  
This commit reworks the conversion calculation to have a more  
consistent behavior:  
- If the number of months and years is 0, return NULL.  
- If the number of months is positive, return the exact month number.  
- If the number of months is negative, do a backward calculation, with  
-1 meaning December, -2 November, etc.  
  
Reported-by: Theodor Arsenij Larionov-Trichkin  
Author: Julien Rouhaud  
Discussion: https://postgr.es/m/[email protected]  
backpatch-through: 9.6  

commit   : c777a1fcc6ba1d61bbb763f6fbd3ac8a5994446d    
  
author   : Magnus Hagander <[email protected]>    
date     : Fri, 9 Apr 2021 12:40:14 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Fri, 9 Apr 2021 12:40:14 +0200    

Author: Daniel Westermann  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/GV0P278MB0483A7AA85BAFCC06D90F453D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM  

commit   : 1cdbf7f0d256b9147859643b20e812c0e9d10263    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 9 Apr 2021 13:53:38 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 9 Apr 2021 13:53:38 +0900    

Comment fixes are applied on HEAD, and documentation improvements are  
applied on back-branches where needed.  
  
Author: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

commit   : df97ee6ea02264279b38c7b9b83ed849d8f9642d    
  
author   : Tomas Vondra <[email protected]>    
date     : Wed, 7 Apr 2021 15:58:35 +0200    
  
committer: Tomas Vondra <[email protected]>    
date     : Wed, 7 Apr 2021 15:58:35 +0200    

The code in bringetbitmap() simply added the whole matching page range  
to the TID bitmap, as determined by pages_per_range, even if some of the  
pages were beyond the end of the heap. The query then might fail with  
an error like this:  
  
  ERROR:  could not open file "base/20176/20228.2" (target block  
          262144): previous segment is only 131021 blocks  
  
In this case, the relation has 262093 pages (131072 and 131021 pages),  
but we're trying to acess block 262144, i.e. first block of the 3rd  
segment. At that point _mdfd_getseg() notices the preceding segment is  
incomplete, and fails.  
  
Hitting this in practice is rather unlikely, because:  
  
* Most indexes use power-of-two ranges, so segments and page ranges  
  align perfectly (segment end is also a page range end).  
  
* The table size has to be just right, with the last segment being  
  almost full - less than one page range from full segment, so that the  
  last page range actually crosses the segment boundary.  
  
* Prefetch has to be enabled. The regular page access checks that  
  pages are not beyond heap end, but prefetch does not. On older  
  releases (before 12) the execution stops after hitting the first  
  non-existent page, so the prefetch distance has to be sufficient  
  to reach the first page in the next segment to trigger the issue.  
  Since 12 it's enough to just have prefetch enabled, the prefetch  
  distance does not matter.  
  
Fixed by not adding non-existent pages to the TID bitmap. Backpatch  
all the way back to 9.6 (BRIN indexes were introduced in 9.5, but that  
release is EOL).  
  
Backpatch-through: 9.6  

commit   : e3bf9621632cc369fbedd15743daf17e55f8db62    
  
author   : Fujii Masao <[email protected]>    
date     : Tue, 6 Apr 2021 02:25:37 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Tue, 6 Apr 2021 02:25:37 +0900    

Maxim Orlov reported that the shutdown of standby server could result in  
the following assertion failure. The cause of this issue was that,  
when the shutdown caused the startup process to exit, recovery-time  
transaction tracking was not shut down even if it's already initialized,  
and some locks the tracked transactions were holding could not be released.  
At this situation, if other process was invoked and the PGPROC entry that  
the startup process used was assigned to it, it found such unreleased locks  
and caused the assertion failure, during the initialization of it.  
  
    TRAP: FailedAssertion("SHMQueueEmpty(&(MyProc->myProcLocks[i]))"  
  
This commit fixes this issue by making the startup process shut down  
transaction tracking and release all locks, at the exit of it.  
  
Back-patch to all supported branches.  
  
Reported-by: Maxim Orlov  
Author: Fujii Masao  
Reviewed-by: Maxim Orlov  
Discussion: https://postgr.es/m/[email protected]  

commit   : 605ef23c7c0fb44f841ec8f1f588325dd8fc951e    
  
author   : Bruce Momjian <[email protected]>    
date     : Fri, 2 Apr 2021 16:42:29 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Fri, 2 Apr 2021 16:42:29 -0400    

All other places already use MONTHS_PER_YEAR appropriately.  
  
Backpatch-through: 9.6  

commit   : 9e67a9440736be87d4013ce2d23faa131523c07e    
  
author   : Joe Conway <[email protected]>    
date     : Fri, 2 Apr 2021 13:48:56 -0400    
  
committer: Joe Conway <[email protected]>    
date     : Fri, 2 Apr 2021 13:48:56 -0400    

Command-line options, or previous "ALTER (ROLE|DATABASE) ...  
SET ROLE ..." commands, can change the value of the default role  
for a session. In the presence of one of these, RESET ROLE will  
change the current user identifier to the default role rather  
than the session user identifier. Fix the documentation to  
reflect this reality. Backpatch to all supported versions.  
  
Author: Nathan Bossart  
Reviewed-By: Laurenz Albe, David G. Johnston, Joe Conway  
Reported by: Nathan Bossart  
Discussion: https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com  
Backpatch-through: 9.6  

commit   : cc7416f3b679decdf2d073fc35402f6e2235720e    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 2 Apr 2021 16:37:28 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 2 Apr 2021 16:37:28 +0900    

The current instructions describing how to write the backup_label and  
tablespace_map files are confusing.  For example, opening a file in text  
mode on Windows and copy-pasting the file's contents would result in a  
failure at recovery because of the extra CRLF characters generated.  The  
documentation was not stating that clearly, and per discussion this is  
not considered as a supported scenario.  
  
This commit extends a bit the documentation to mention that it may be  
required to open the file in binary mode before writing its data.  
  
Reported-by: Wang Shenhao  
Author: David Steele  
Reviewed-by: Andrew Dunstan, Magnus Hagander  
Discussion: https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local  
Backpatch-through: 9.6  

commit   : ab88525ad29cc23f99eb615258621caadbb5127e    
  
author   : Bruce Momjian <[email protected]>    
date     : Thu, 1 Apr 2021 21:17:24 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Thu, 1 Apr 2021 21:17:24 -0400    

Also mention that you should read the intervening major releases notes.  
This change was also applied to the website.  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

commit   : 2c9b857afbcf5c563d84824d32cd55e167ae0f6d    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 1 Apr 2021 13:34:16 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 1 Apr 2021 13:34:16 -0400    

Despite the clear comments pointing out that the duplicative code  
segments in ReadHead() and _discoverArchiveFormat() needed to be  
in sync, they were not: the latter did not bother to apply any of  
the sanity checks in the former.  We'd missed noticing this partly  
because none of those checks would fail in scenarios we customarily  
test, and partly because the oversight would be masked if both  
segments execute, which they would in cases other than needing to  
autodetect the format of a non-seekable stdin source.  However,  
in a case meeting all these requirements --- for example, trying  
to read a newer-than-supported archive format from non-seekable  
stdin --- pg_restore missed applying the version check and would  
likely dump core or otherwise misbehave.  
  
The whole thing is silly anyway, because there seems little reason  
to duplicate the logic beyond the one-line verification that the  
file starts with "PGDMP".  There seems to have been an undocumented  
assumption that multiple major formats (major enough to require  
separate reader modules) would nonetheless share the first half-dozen  
fields of the custom-format header.  This seems unlikely, so let's  
fix it by just nuking the duplicate logic in _discoverArchiveFormat().  
  
Also get rid of the pointless attempt to seek back to the start of  
the file after successful autodetection.  That wastes cycles and  
it means we have four behaviors to verify not two.  
  
Per bug #16951 from Sergey Koposov.  This has been broken for  
decades, so back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 5ae07439db9e35120cd1c24fa96533307c9c09ec    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 1 Apr 2021 15:29:12 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 1 Apr 2021 15:29:12 +0900    

Some sections of the documentation used "exclusive lock" to describe  
that an ACCESS EXCLUSIVE lock is taken during a given operation.  This  
can be confusing to the reader as ACCESS SHARE is allowed with an  
EXCLUSIVE lock is used, but that would not be the case with what is  
described on those parts of the documentation.  
  
Author: Greg Rychlewski  
Discussion: https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com  
Backpatch-through: 9.6  

commit   : 72ceeab9c5ad0a78d1ee04c680b11a55ae13e94f    
  
author   : Etsuro Fujita <[email protected]>    
date     : Tue, 30 Mar 2021 13:00:08 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Tue, 30 Mar 2021 13:00:08 +0900    

Back-patch to all supported branches.  
  
Author: Etsuro Fujita  
Discussion: https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com  

commit   : ef924828fd84b3ebaaae97f8dd8349112cb5d8fa    
  
author   : Stephen Frost <[email protected]>    
date     : Sun, 28 Mar 2021 11:28:22 -0400    
  
committer: Stephen Frost <[email protected]>    
date     : Sun, 28 Mar 2021 11:28:22 -0400    

Commit c6763156589 added an acronym reference for "TLS" but the definition  
was never added.  
  
Author: Daniel Gustafsson  
Reviewed-by: Michael Paquier  
Backpatch-through: 9.6  
Discussion: https://postgr.es/m/[email protected]  

commit   : c29bab799c82c3640ec93cf14d0b4f174ae33fe0    
  
author   : Fujii Masao <[email protected]>    
date     : Thu, 25 Mar 2021 11:23:30 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Thu, 25 Mar 2021 11:23:30 +0900    

Previously the WAL replay of COMMIT_TS_SETTS record called  
TransactionTreeSetCommitTsData() with the argument write_xlog=true,  
which generated and wrote new COMMIT_TS_SETTS record.  
This should not be acceptable because it's during recovery.  
  
This commit fixes the WAL replay of COMMIT_TS_SETTS record  
so that it calls TransactionTreeSetCommitTsData() with write_xlog=false  
and doesn't generate new WAL during recovery.  
  
Back-patch to all supported branches.  
  
Reported-by: lx zou <[email protected]>  
Author: Fujii Masao  
Reviewed-by: Alvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  

commit   : 4f670c64efd972e8869117d8324a1379070c9793    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 23 Mar 2021 14:27:50 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 23 Mar 2021 14:27:50 -0400    

Jasen Betts reported yet another unintended side effect of commit  
85c54287a: reconnecting with "\c service=whatever" did not have the  
expected results.  The reason is that starting from the output of  
PQconndefaults() effectively allows environment variables (such  
as PGPORT) to override entries in the service file, whereas the  
normal priority is the other way around.  
  
Not using PQconndefaults at all would require yet a third main code  
path in do_connect's parameter setup, so I don't really want to fix  
it that way.  But we can have the logic effectively ignore all the  
default values for just a couple more lines of code.  
  
This patch doesn't change the behavior for "\c -reuse-previous=on  
service=whatever".  That remains significantly different from before  
85c54287a, because many more parameters will be re-used, and thus  
not be possible for service entries to replace.  But I think this  
is (mostly?) intentional.  In any case, since libpq does not report  
where it got parameter values from, it's hard to do differently.  
  
Per bug #16936 from Jasen Betts.  As with the previous patches,  
back-patch to all supported branches.  (9.5 is unfortunately now  
out of support, so this won't get fixed there.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : e73068b0710bfd651893e1a954e3ec95e533c118    
  
author   : Fujii Masao <[email protected]>    
date     : Tue, 23 Mar 2021 09:53:08 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Tue, 23 Mar 2021 09:53:08 +0900    

pg_waldump --stats=record identifies a record by a combination  
of the RmgrId and the four bits of the xl_info field of the record.  
But XACT records use the first bit of those four bits for an optional  
flag variable, and the following three bits for the opcode to  
identify a record. So previously the same type of XACT record  
could have different four bits (three bits are the same but the  
first one bit is different), and which could cause  
pg_waldump --stats=record to show two lines of per-record statistics  
for the same XACT record. This is a bug.  
  
This commit changes pg_waldump --stats=record so that it processes  
only XACT record differently, i.e., filters the opcode out of xl_info  
and uses a combination of the RmgrId and those three bits as  
the identifier of a record, only for XACT record. For other records,  
the four bits of the xl_info field are still used.  
  
Back-patch to all supported branches.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Shinya Kato, Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7e25217701cc54d16a2f07df135da3ba43066449    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 22:09:41 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 22:09:41 -0400    

Because guc.c prefers to keep all its string values in malloc'd  
not palloc'd storage, it has to be more careful than usual to  
avoid leaks.  Error exits out of string GUC hook checks failed  
to clear the proposed value string, and error exits out of  
ProcessGUCArray() failed to clear the malloc'd results of  
ParseLongOption().  
  
Found via valgrind testing.  
This problem is ancient, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 09e961929614d211405721f070b3db9f1b3f25f4    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 21:44:43 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Mar 2021 21:44:43 -0400    

The text search cache mechanisms assume that we can clean up  
an invalidated dictionary cache entry simply by resetting the  
associated long-lived memory context.  However, that does not work  
for ispell affixes that make use of regular expressions, because  
the regex library deals in plain old malloc.  Hence, we leaked  
compiled regex(es) any time we dropped such a cache entry.  That  
could quickly add up, since even a fairly trivial regex can use up  
tens of kB, and a large one can eat megabytes.  Add a memory context  
callback to ensure that a regex gets freed when its owning cache  
entry is cleared.  
  
Found via valgrind testing.  
This problem is ancient, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 19b32bd53f90577a130879ddfeb4f9d75d08c1c5    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 17 Mar 2021 16:10:38 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 17 Mar 2021 16:10:38 -0400    

Robert Foggia of Trustwave reported that read_tablespace_map()  
fails to prevent an overrun of its on-stack input buffer.  
Since the tablespace map file is presumed trustworthy, this does  
not seem like an interesting security vulnerability, but still  
we should fix it just in the name of robustness.  
  
While here, document that pg_basebackup's --tablespace-mapping option  
doesn't work with tar-format output, because it doesn't.  To make it  
work, we'd have to modify the tablespace_map file within the tarball  
sent by the server, which might be possible but I'm not volunteering.  
(Less-painful solutions would require changing the basebackup protocol  
so that the source server could adjust the map.  That's not very  
appetizing either.)  

commit   : a42c4438b058e3ab5ce79c27deed3ea12aa794e5    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 12 Mar 2021 12:20:15 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 12 Mar 2021 12:20:15 -0500    

psql's editing commands decide whether the user has edited the file  
by checking for change of modification timestamp.  This is probably  
fine for a pre-existing file, but with a temporary file that is  
created within the command, it's possible for a fast typist to  
save-and-exit in less than the one-second granularity of stat(2)  
timestamps.  On Windows FAT filesystems the granularity is even  
worse, 2 seconds, making the race a bit easier to hit.  
  
To fix, try to set the temp file's mod time to be two seconds ago.  
It's unlikely this would fail, but then again the race condition  
itself is unlikely, so just ignore any error.  
  
Also, we might as well check the file size as well as its mod time.  
  
While this is a difficult bug to hit, it still seems worth  
back-patching, to ensure that users' edits aren't lost.  
  
Laurenz Albe, per gripe from Jacob Champion; based on fix suggestions  
from Jacob and myself  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a98e53e10dd5e9278c62b49406f30c1d7e28ab0a    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 11 Mar 2021 14:43:45 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 11 Mar 2021 14:43:45 -0500    

Commit 92785dac2 copied some logic related to advancement of inStart  
from pqParseInput3 into getRowDescriptions and getAnotherTuple,  
because it wanted to allow user-defined row processor callbacks to  
potentially longjmp out of the library, and inStart would have to be  
updated before that happened to avoid an infinite loop.  We later  
decided that that API was impossibly fragile and reverted it, but  
we didn't undo all of the related code changes, and this bit of  
messiness survived.  Undo it now so that there's just one place in  
pqParseInput3's processing where inStart is advanced; this will  
simplify addition of better tracing support.  
  
getParamDescriptions had grown similar processing somewhere along  
the way (not in 92785dac2; I didn't track down just when), but it's  
actually buggy because its handling of corrupt-message cases seems to  
have been copied from the v2 logic where we lacked a known message  
length.  The cases where we "goto not_enough_data" should not simply  
return EOF, because then we won't consume the message, potentially  
creating an infinite loop.  That situation now represents a  
definitively corrupt message, and we should report it as such.  
  
Although no field reports of getParamDescriptions getting stuck in  
a loop have been seen, it seems appropriate to back-patch that fix.  
I chose to back-patch all of this to keep the logic looking more alike  
in supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : b6b1d72247ee390b471b68cdedd632b8530e88e5    
  
author   : Bruce Momjian <[email protected]>    
date     : Wed, 10 Mar 2021 20:25:18 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Wed, 10 Mar 2021 20:25:18 -0500    

This is a follow-on patch to 92c12e46d5.  In that patch, we renamed  
"altitude" to "elevation" in the docs, based on these details:  
  
   https://mapscaping.com/blogs/geo-candy/what-is-the-difference-between-elevation-relief-and-altitude  
  
This renames the tutorial SQL files to match the documentation.  
  
Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

commit   : c7a4fc3dd001646d5938687ad59ab84545d5d043    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 24 Feb 2021 16:14:13 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 24 Feb 2021 16:14:13 +0900    

The portions fixing the documentation are backpatched where needed.  
  
Author: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
backpatch-through: 9.6  

commit   : 0ccebe779dc0ed1c5f37370f2d5bc59779cf9073    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 23 Feb 2021 17:30:21 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 23 Feb 2021 17:30:21 -0300    

Commit 866e24d47db1 added an assert that HEAP_XMAX_LOCK_ONLY and  
HEAP_KEYS_UPDATED cannot appear together, on the faulty assumption that  
the latter necessarily referred to an update and not a tuple lock; but  
that's wrong, because SELECT FOR UPDATE can use precisely that  
combination, as evidenced by the amcheck test case added here.  
  
Remove the Assert(), and also patch amcheck's verify_heapam.c to not  
complain if the combination is found.  Also, out of overabundance of  
caution, update (across all branches) README.tuplock to be more explicit  
about this.  
  
Author: Julien Rouhaud <[email protected]>  
Reviewed-by: Mahendra Singh Thalor <[email protected]>  
Reviewed-by: Dilip Kumar <[email protected]>  
Discussion: https://postgr.es/m/20210124061758.GA11756@nol  

commit   : ab208421eeb785fc72a3e7fc1122127029165392    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 18 Feb 2021 22:38:55 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 18 Feb 2021 22:38:55 -0500    

While poking at the regex code, I happened to notice that the bug  
squashed in commit afcc8772e had a sibling: next() failed to return  
a specific value associated with the '}' token for a "\{m,n\}"  
quantifier when parsing in basic RE mode.  Again, this could result  
in treating the quantifier as non-greedy, which it never should be in  
basic mode.  For that to happen, the last character before "\}" that  
sets "nextvalue" would have to set it to zero, or it'd have to have  
accidentally been zero from the start.  The failure can be provoked  
repeatably with, for example, a bound ending in digit "0".  
  
Like the previous patch, back-patch all the way.  

commit   : 618d139f82d4222df796e989de3c4c4de0359d7b    
  
author   : Thomas Munro <[email protected]>    
date     : Tue, 16 Feb 2021 13:10:37 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Tue, 16 Feb 2021 13:10:37 +1300    

Back-patch a tiny bit of commit fbb2e9a0 into 9.6 and 10, to silence an  
uninitialized variable warning from GCC 10.2.  Seen on buildfarm member  
handfish, and my own development workflow where I like to use -Werror.  
  
Discussion: https://postgr.es/m/CA%2BhUKGJRcwvK86Uf5t-FrTekZjqHtpv3u%3D3MuBg8Zw8R933Mqg%40mail.gmail.com  

commit   : 09a3b19e38ee09ce1f12cee9b9537ae66d729ead    
  
author   : Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 15:43:39 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 15:43:39 +1300    

FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to  
choose open_datasync as its default value.  That may not be a good  
choice for all systems, and performs worse than fdatasync in some  
scenarios.  Let's preserve the existing default behavior for now.  
  
Like commit 576477e73c4, which did the same for Linux, back-patch to all  
supported releases.  
  
Discussion: https://postgr.es/m/CA%2BhUKGLsAMXBQrCxCXoW-JsUYmdOL8ALYvaX%3DCrHqWxm-nWbGA%40mail.gmail.com  

commit   : 8a07e5bd82486b024e58417c09da193c275b13d3    
  
author   : Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 13:32:58 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Mon, 15 Feb 2021 13:32:58 +1300    

While cleaning up after a parallel query or parallel index creation that  
created temporary files, we could be interrupted by a statement timeout.  
The error handling path would then fail to clean up the files when it  
ran dsm_detach() again, because the callback was already popped off the  
list.  Prevent this hazard by holding interrupts while the cleanup code  
runs.  
  
Thanks to Heikki Linnakangas for this suggestion, and also to Kyotaro  
Horiguchi, Masahiko Sawada, Justin Pryzby and Tom Lane for discussion of  
this and earlier ideas on how to fix the problem.  
  
Back-patch to all supported releases.  
  
Reported-by: Justin Pryzby <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

commit   : cc7ea0717b128cb23d564b9b7c899eb6164752fc    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 13 Feb 2021 17:49:08 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 13 Feb 2021 17:49:08 -0500    

Modern gcc and clang compilers offer alignment sanitizers, which help to detect  
pointer misalignment.  However, our codebase already contains x86-specific  
crc32 computation code, which uses unalignment access.  Thankfully, those  
compilers also support the attribute, which disables alignment sanitizers at  
the function level.  This commit adds pg_attribute_no_sanitize_alignment(),  
which wraps this attribute, and applies it to pg_comp_crc32c_sse42() function.  
  
Back-patch of commits 993bdb9f9 and ad2ad698a, to enable doing  
alignment testing in all supported branches.  
  
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com  
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us  
Author: Alexander Korotkov, revised by Tom Lane  
Reviewed-by: Tom Lane  

commit   : f4d781dae6b60424304b141165943185b907990b    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 12 Feb 2021 16:26:47 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 12 Feb 2021 16:26:47 -0500    

Given a regex pattern with a very long fixed prefix (approaching 500  
characters), the result of pow(FIXED_CHAR_SEL, fixed_prefix_len) can  
underflow to zero.  Typically the preceding selectivity calculation  
would have underflowed as well, so that we compute 0/0 and get NaN.  
In released branches this leads to an assertion failure later on.  
That doesn't happen in HEAD, for reasons I've not explored yet,  
but it's surely still a bug.  
  
To fix, just skip the division when the pow() result is zero, so  
that we'll (most likely) return a zero selectivity estimate.  In  
the edge cases where "sel" didn't yet underflow, perhaps this  
isn't desirable, but I'm not sure that the case is worth spending  
a lot of effort on.  The results of regex_selectivity_sub() are  
barely worth the electrons they're written on anyway :-(  
  
Per report from Alexander Lakhin.  Back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/[email protected]