PostgreSQL 12.4 (upcoming) commit log

Fix yet another issue with step generation in partition pruning.

commit   : 4f26932296057721d7668ec13cfd6af2bb8410ec    
  
author   : Etsuro Fujita <efujita@postgresql.org>    
date     : Fri, 7 Aug 2020 14:45:02 +0900    
  
committer: Etsuro Fujita <efujita@postgresql.org>    
date     : Fri, 7 Aug 2020 14:45:02 +0900    

Click here for diff

Commit 13838740f fixed some issues with step generation in partition  
pruning, but there was yet another one: get_steps_using_prefix() assumes  
that clauses in the passed-in prefix list are sorted in ascending order  
of their partition key numbers, but the caller failed to ensure this for  
range partitioning, which led to an assertion failure in debug builds.  
Adjust the caller function to arrange the clauses in the prefix list in  
the required order for range partitioning.  
  
Back-patch to v11, like the previous commit.  
  
Patch by me, reviewed by Amit Langote.  
  
Discussion: https://postgr.es/m/CAPmGK16jkXiFG0YqMbU66wte-oJTfW6D1HaNvQf%3D%2B5o9%3Dm55wQ%40mail.gmail.com  

M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql

First-draft release notes for 12.4.

commit   : a2e0cf45c21afbcbc544d1aca8d51d90004aa5d9    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 6 Aug 2020 15:49:45 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 6 Aug 2020 15:49:45 -0400    

Click here for diff

As usual, the release notes for other branches will be made by cutting  
these down, but put them up for community review first.  

M doc/src/sgml/release-12.sgml

Fix typo.

commit   : 7c78040f6c46e3db8cfa45fc31add9bc511cc2b2    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Thu, 6 Aug 2020 14:55:00 -0400    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Thu, 6 Aug 2020 14:55:00 -0400    

Click here for diff

Per report from Tom Lane. Previously fixed in master by  
commit f057980149ddccd4b862d2c6b3920ed498b0d7ec.  

M src/backend/access/transam/xlog.c

Fix minor problems with non-exclusive backup cleanup.

commit   : bcbc27251d35336a6442761f59638138a772b839    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Thu, 6 Aug 2020 13:58:23 -0400    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Thu, 6 Aug 2020 13:58:23 -0400    

Click here for diff

The previous coding imagined that it could call before_shmem_exit()  
when a non-exclusive backup began and then remove the previously-added  
handler by calling cancel_before_shmem_exit() when that backup  
ended. However, this only works provided that nothing else in the  
system has registered a before_shmem_exit() hook in the interim,  
because cancel_before_shmem_exit() is documented to remove a callback  
only if it is the latest callback registered. It also only works  
if nothing can ERROR out between the time that sessionBackupState  
is reset and the time that cancel_before_shmem_exit(), which doesn't  
seem to be strictly true.  
  
To fix, leave the handler installed for the lifetime of the session,  
arrange to install it just once, and teach it to quietly do nothing if  
there isn't a non-exclusive backup in process.  
  
This was originally committed to master as  
303640199d0436c5e7acdf50b837a027b5726594, but I did not back-patch  
at the time because the consequences were minor. However, now  
there's been a second report of this causing trouble with a slightly  
different test case than the one I reported originally, so now  
I'm back-patching as far as v11 where JIT was introduced.  
  
Patch by me, reviewed by Kyotaro Horiguchi, Michael Paquier (who  
preferred a different approach, but got outvoted), Fujii Masao,  
and Tom Lane, and with comments by various others. New problem  
report from Bharath Rupireddy.  
  
Discussion: http://postgr.es/m/CA+TgmobMjnyBfNhGTKQEDbqXYE3_rXWpc4CM63fhyerNCes3mA@mail.gmail.com  
Discussion: http://postgr.es/m/CALj2ACWk7j4F2v2fxxYfrroOF=AdFNPr1WsV+AGtHAFQOqm_pw@mail.gmail.com  

M src/backend/access/transam/xlog.c
M src/backend/access/transam/xlogfuncs.c
M src/backend/replication/basebackup.c
M src/include/access/xlog.h

doc: clarify "state" table reference in tutorial

commit   : f87f77ec8adf65370c6d35919cd576eae60a5528    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Wed, 5 Aug 2020 17:12:10 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Wed, 5 Aug 2020 17:12:10 -0400    

Click here for diff

Reported-by: Vyacheslav Shablistyy  
  
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/advanced.sgml

Fix matching of sub-partitions when a partitioned plan is stale.

commit   : f992da210ff84d2d8f0bf9b703881c880cc74d9f    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 5 Aug 2020 15:38:55 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 5 Aug 2020 15:38:55 -0400    

Click here for diff

Since we no longer require AccessExclusiveLock to add a partition,  
the executor may see that a partitioned table has more partitions  
than the planner saw.  ExecCreatePartitionPruneState's code for  
matching up the partition lists in such cases was faulty, and would  
misbehave if the planner had successfully pruned any partitions from  
the query.  (Thus, trouble would occur only if a partition addition  
happens concurrently with a query that uses both static and dynamic  
partition pruning.)  This led to an Assert failure in debug builds,  
and probably to crashes or query misbehavior in production builds.  
  
To repair the bug, just explicitly skip zeroes in the plan's  
relid_map[] list.  I also made some cosmetic changes to make the code  
more readable (IMO anyway).  Also, convert the cross-checking Assert  
to a regular test-and-elog, since it's now apparent that this logic  
is more fragile than one would like.  
  
Currently, there's no way to repeatably exercise this code, except  
with manual use of a debugger to stop the backend between planning  
and execution.  Hence, no test case in this patch.  We oughta do  
something about that testability gap, but that's for another day.  
  
Amit Langote and Tom Lane, per report from Justin Pryzby.  Oversight  
in commit 898e5e329; backpatch to v12 where that appeared.  
  
Discussion: https://postgr.es/m/20200802181131.GA27754@telsasoft.com  

M src/backend/executor/execPartition.c

Increase hard-wired timeout values in ecpg regression tests.

commit   : 55ffd6140c3e2e9cacc76925daa094e2ed979947    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 4 Aug 2020 15:20:31 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 4 Aug 2020 15:20:31 -0400    

Click here for diff

A couple of test cases had connect_timeout=14, a value that seems  
to have been plucked from a hat.  While it's more than sufficient  
for normal cases, slow/overloaded buildfarm machines can get a timeout  
failure here, as per recent report from "sungazer".  Increase to 180  
seconds, which is in line with our typical timeouts elsewhere in  
the regression tests.  
  
Back-patch to 9.6; the code looks different in 9.5, and this doesn't  
seem to be quite worth the effort to adapt to that.  
  
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sungazer&dt=2020-08-04%2007%3A12%3A22  

M src/interfaces/ecpg/test/connect/test1.pgc
M src/interfaces/ecpg/test/connect/test5.pgc
M src/interfaces/ecpg/test/expected/connect-test1-minGW32.stderr
M src/interfaces/ecpg/test/expected/connect-test1.c
M src/interfaces/ecpg/test/expected/connect-test1.stderr
M src/interfaces/ecpg/test/expected/connect-test5.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr

Doc: fix obsolete info about allowed range of TZ offsets in timetz.

commit   : 8d5c632e9606fd757326888a2eade1990f7fe048    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 3 Aug 2020 13:11:16 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 3 Aug 2020 13:11:16 -0400    

Click here for diff

We've allowed UTC offsets up to +/- 15:59 since commit cd0ff9c0f, but  
that commit forgot to fix the documentation about timetz.  
  
Per bug #16571 from osdba.  
  
Discussion: https://postgr.es/m/16571-eb7501598de78c8a@postgresql.org  

M doc/src/sgml/datatype.sgml

Fix rare failure in LDAP tests.

commit   : 76b2b3e72438dda2ec0566ee85feadde3224ff2f    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 3 Aug 2020 12:39:15 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 3 Aug 2020 12:39:15 +1200    

Click here for diff

Instead of writing a query to psql's stdin, use -c.  This avoids a  
failure where psql exits before we write, seen a few times on the build  
farm.  Thanks to Tom Lane for the suggestion.  
  
Back-patch to 11, where the LDAP tests arrived.  
  
Reviewed-by: Noah Misch <noah@leadboat.com>  
Discussion: https://postgr.es/m/CA%2BhUKGLFmW%2BHQYPeKiwSp5sdFFHtFViCpw4Mh6yAgEx74r5-Cw%40mail.gmail.com  

M src/test/ldap/t/001_auth.pl

Restore lost amcheck TOAST test coverage.

commit   : 16c977906e92673b22a05bc80e9a53d023cdd79b    
  
author   : Peter Geoghegan <pg@bowt.ie>    
date     : Fri, 31 Jul 2020 15:34:25 -0700    
  
committer: Peter Geoghegan <pg@bowt.ie>    
date     : Fri, 31 Jul 2020 15:34:25 -0700    

Click here for diff

Commit eba77534 fixed an amcheck false positive bug involving  
inconsistencies in TOAST input state between table and index.  A test  
case was added that verified that such an inconsistency didn't result in  
a spurious corruption related error.  
  
Test coverage from the test was accidentally lost by commit 501e41dd,  
which propagated ALTER TABLE ...  SET STORAGE attstorage state to  
indexes.  This broke the test because the test specifically relied on  
attstorage not being propagated.  This artificially forced there to be  
index tuples whose datums were equivalent to the datums in the heap  
without the datums actually being bitwise equal.  
  
Fix this by updating pg_attribute directly instead.  Commit 501e41dd  
made similar changes to a test_decoding TOAST-related test case which  
made the same assumption, but overlooked the amcheck test case.  
  
Backpatch: 11-, just like commit eba77534 (and commit 501e41dd).  

M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/sql/check_btree.sql

Fix recently-introduced performance problem in ts_headline().

commit   : 70248d8f5b533e0fa045cfc67f623366cdaec9e0    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 31 Jul 2020 11:43:12 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 31 Jul 2020 11:43:12 -0400    

Click here for diff

The new hlCover() algorithm that I introduced in commit c9b0c678d  
turns out to potentially take O(N^2) or worse time on long documents,  
if there are many occurrences of individual query words but few or no  
substrings that actually satisfy the query.  (One way to hit this  
behavior is with a "common_word & rare_word" type of query.)  This  
seems unavoidable given the original goal of checking every substring  
of the document, so we have to back off that idea.  Fortunately, it  
seems unlikely that anyone would really want headlines spanning all of  
a long document, so we can avoid the worse-than-linear behavior by  
imposing a maximum length of substring that we'll consider.  
  
For now, just hard-wire that maximum length as a multiple of max_words  
times max_fragments.  Perhaps at some point somebody will argue for  
exposing it as a ts_headline parameter, but I'm hesitant to make such  
a feature addition in a back-patched bug fix.  
  
I also noted that the hlFirstIndex() function I'd added in that  
commit was unnecessarily stupid: it really only needs to check whether  
a HeadlineWordEntry's item pointer is null or not.  This wouldn't make  
all that much difference in typical cases with queries having just  
a few terms, but a cycle shaved is a cycle earned.  
  
In addition, add a CHECK_FOR_INTERRUPTS call in TS_execute_recurse.  
This ensures that hlCover's loop is cancellable if it manages to take  
a long time, and it may protect some other TS_execute callers as well.  
  
Back-patch to 9.6 as the previous commit was.  I also chose to add the  
CHECK_FOR_INTERRUPTS call to 9.5.  The old hlCover() algorithm seems  
to avoid the O(N^2) behavior, at least on the test case I tried, but  
nonetheless it's not very quick on a long document.  
  
Per report from Stephen Frost.  
  
Discussion: https://postgr.es/m/20200724160535.GW12375@tamriel.snowman.net  

M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/tsvector_op.c

Doc: fix high availability solutions comparison.

commit   : a63fbd34813ea61a3d293e1143e509378aa9d6e7    
  
author   : Tatsuo Ishii <ishii@postgresql.org>    
date     : Fri, 31 Jul 2020 07:48:23 +0900    
  
committer: Tatsuo Ishii <ishii@postgresql.org>    
date     : Fri, 31 Jul 2020 07:48:23 +0900    

Click here for diff

In "High Availability, Load Balancing, and Replication" chapter,  
certain descriptions of Pgpool-II were not correct at this point.  It  
does not need conflict resolution. Also "Multiple-Server Parallel  
Query Execution" is not supported anymore.  
  
Discussion: https://postgr.es/m/20200726.230128.53842489850344110.t-ishii%40sraoss.co.jp  
Author: Tatsuo Ishii  
Reviewed-by: Bruce Momjian  
Backpatch-through: 9.5  

M doc/src/sgml/high-availability.sgml

doc: Mention index references in pg_inherits

commit   : 7de22d228aae5183161546be7cb874deb7389019    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 30 Jul 2020 15:48:56 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 30 Jul 2020 15:48:56 +0900    

Click here for diff

Partitioned indexes are also registered in pg_inherits, but the  
description of this catalog did not reflect that.  
  
Author: Dagfinn Ilmari Mannsåker  
Discussion: https://postgr.es/m/87k0ynj35y.fsf@wibble.ilmari.org  
Backpatch-through: 11  

M doc/src/sgml/catalogs.sgml

Doc: Improve documentation for pg_jit_available()

commit   : 6ed346499c30162a81fdfd84556284c77030a17e    
  
author   : David Rowley <drowley@postgresql.org>    
date     : Tue, 28 Jul 2020 22:52:51 +1200    
  
committer: David Rowley <drowley@postgresql.org>    
date     : Tue, 28 Jul 2020 22:52:51 +1200    

Click here for diff

Per complaint from Scott Ribe. Based on wording suggestion from Tom Lane.  
  
Discussion: https://postgr.es/m/1956E806-1468-4417-9A9D-235AE1D5FE1A@elevated-dev.com  
Backpatch-through: 11, where pg_jit_available() was added  

M doc/src/sgml/func.sgml

Fix some issues with step generation in partition pruning.

commit   : 62c4a77295ec7bf3f7209ae8cfa92a3a2b87b9c4    
  
author   : Etsuro Fujita <efujita@postgresql.org>    
date     : Tue, 28 Jul 2020 11:00:02 +0900    
  
committer: Etsuro Fujita <efujita@postgresql.org>    
date     : Tue, 28 Jul 2020 11:00:02 +0900    

Click here for diff

In the case of range partitioning, get_steps_using_prefix() assumes that  
the passed-in prefix list contains at least one clause for each of the  
partition keys earlier than one specified in the passed-in  
step_lastkeyno, but the caller (ie, gen_prune_steps_from_opexps())  
didn't take it into account, which led to a server crash or incorrect  
results when the list contained no clauses for such partition keys, as  
reported in bug #16500 and #16501 from Kobayashi Hisanori.  Update the  
caller to call that function only when the list created there contains  
at least one clause for each of the earlier partition keys in the case  
of range partitioning.  
  
While at it, fix some other issues:  
  
* The list to pass to get_steps_using_prefix() is allowed to contain  
  multiple clauses for the same partition key, as described in the  
  comment for that function, but that function actually assumed that the  
  list contained just a single clause for each of middle partition keys,  
  which led to an assertion failure when the list contained multiple  
  clauses for such partition keys.  Update that function to match the  
  comment.  
* In the case of hash partitioning, partition keys are allowed to be  
  NULL, in which case the list to pass to get_steps_using_prefix()  
  contains no clauses for NULL partition keys, but that function treats  
  that case as like the case of range partitioning, which led to the  
  assertion failure.  Update the assertion test to take into account  
  NULL partition keys in the case of hash partitioning.  
* Fix a typo in a comment in get_steps_using_prefix_recurse().  
* gen_partprune_steps() failed to detect self-contradiction from  
  strict-qual clauses and an IS NULL clause for the same partition key  
  in some cases, producing incorrect partition-pruning steps, which led  
  to incorrect results of partition pruning, but didn't cause any  
  user-visible problems fortunately, as the self-contradiction is  
  detected later in the query planning.  Update that function to detect  
  the self-contradiction.  
  
Per bug #16500 and #16501 from Kobayashi Hisanori.  Patch by me, initial  
diagnosis for the reported issue and review by Dmitry Dolgov.  
Back-patch to v11, where partition pruning was introduced.  
  
Discussion: https://postgr.es/m/16500-d1613f2a78e1e090%40postgresql.org  
Discussion: https://postgr.es/m/16501-5234a9a0394f6754%40postgresql.org  

M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql

Fix corner case with 16kB-long decompression in pgcrypto, take 2

commit   : 5bd087eb5d770624ea5ea903af7a7ec6c460d060    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 27 Jul 2020 15:58:59 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 27 Jul 2020 15:58:59 +0900    

Click here for diff

A compressed stream may end with an empty packet.  In this case  
decompression finishes before reading the empty packet and the  
remaining stream packet causes a failure in reading the following  
data.  This commit makes sure to consume such extra data, avoiding a  
failure when decompression the data.  This corner case was reproducible  
easily with a data length of 16kB, and existed since e94dd6a.  A cheap  
regression test is added to cover this case based on a random,  
incompressible string.  
  
The first attempt of this patch has allowed to find an older failure  
within the compression logic of pgcrypto, fixed by b9b6105.  This  
involved SLES 15 with z390 where a custom flavor of libz gets used.  
Bonus thanks to Mark Wong for providing access to the specific  
environment.  
  
Reported-by: Frank Gagnepain  
Author: Kyotaro Horiguchi, Michael Paquier  
Reviewed-by: Tom Lane  
Discussion: https://postgr.es/m/16476-692ef7b84e5fb893@postgresql.org  
Backpatch-through: 9.5  

M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql

Fix handling of structure for bytea data type in ECPG

commit   : 61a4a3a62a98bb7508a86d00900e80e23ebf2b63    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 27 Jul 2020 10:29:11 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 27 Jul 2020 10:29:11 +0900    

Click here for diff

Some code paths dedicated to bytea used the structure for varchar.  This  
did not lead to any actual bugs, as bytea and varchar have the same  
definition, but it could become a trap if one of these definitions  
changes for a new feature or a bug fix.  
  
Issue introduced by 050710b.  
  
Author: Shenhao Wang  
Reviewed-by: Vignesh C, Michael Paquier  
Discussion: https://postgr.es/m/07ac7dee1efc44f99d7f53a074420177@G08CNEXMBPEKD06.g08.fujitsu.local  
Backpatch-through: 12  

M src/interfaces/ecpg/ecpglib/data.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/execute.c

Fix buffer usage stats for nodes above Gather Merge.

commit   : bdaa84e38932de43fca89b20a5078bf92ea356e4    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Sat, 25 Jul 2020 10:38:46 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Sat, 25 Jul 2020 10:38:46 +0530    

Click here for diff

Commit 85c9d347 addressed a similar problem for Gather and Gather  
Merge nodes but forgot to account for nodes above parallel nodes.  This  
still works for nodes above Gather node because we shut down the workers  
for Gather node as soon as there are no more tuples.  We can do a similar  
thing for Gather Merge as well but it seems better to account for stats  
during nodes shutdown after completing the execution.  
  
Reported-by: Stéphane Lorek, Jehan-Guillaume de Rorthais  
Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>  
Reviewed-by: Amit Kapila  
Backpatch-through: 10, where it was introduced  
Discussion: https://postgr.es/m/20200718160206.584532a2@firost  

M src/backend/executor/execProcnode.c

Fix ancient violation of zlib's API spec.

commit   : 3d4a778152ecd8dc6815ce8b5525925bac446a99    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 23 Jul 2020 17:19:37 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 23 Jul 2020 17:19:37 -0400    

Click here for diff

contrib/pgcrypto mishandled the case where deflate() does not consume  
all of the offered input on the first try.  It reset the next_in pointer  
to the start of the input instead of leaving it alone, causing the wrong  
data to be fed to the next deflate() call.  
  
This has been broken since pgcrypto was committed.  The reason for the  
lack of complaints seems to be that it's fairly hard to get stock zlib  
to not consume all the input, so long as the output buffer is big enough  
(which it normally would be in pgcrypto's usage; AFAICT the input is  
always going to be packetized into packets no larger than ZIP_OUT_BUF).  
However, IBM's zlibNX implementation for AIX evidently will do it  
in some cases.  
  
I did not add a test case for this, because I couldn't find one that  
would fail with stock zlib.  When we put back the test case for  
bug #16476, that will cover the zlibNX situation well enough.  
  
While here, write deflate()'s second argument as Z_NO_FLUSH per its  
API spec, instead of hard-wiring the value zero.  
  
Per buildfarm results and subsequent investigation.  
  
Discussion: https://postgr.es/m/16476-692ef7b84e5fb893@postgresql.org  

M contrib/pgcrypto/pgp-compress.c

doc: Document that ssl_ciphers does not affect TLS 1.3

commit   : 63b2297a33ebca87942e55643339c08fbdd25526    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Thu, 23 Jul 2020 17:13:00 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Thu, 23 Jul 2020 17:13:00 +0200    

Click here for diff

TLS 1.3 uses a different way of specifying ciphers and a different  
OpenSSL API.  PostgreSQL currently does not support setting those  
ciphers.  For now, just document this.  In the future, support for  
this might be added somehow.  
  
Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org>  
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>  

M doc/src/sgml/config.sgml

Fix error message.

commit   : 8bf4e69a7f7b076fae3b8e2361977f590d095b5a    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Thu, 23 Jul 2020 21:10:49 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Thu, 23 Jul 2020 21:10:49 +1200    

Click here for diff

Remove extra space.  Back-patch to all releases, like commit 7897e3bb.  
  
Author: Lu, Chenyang <lucy.fnst@cn.fujitsu.com>  
Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local  

M src/backend/storage/file/buffile.c

Revert "Fix corner case with PGP decompression in pgcrypto"

commit   : e30a63f258f5cf3e7e797be131890ecd1144b44f    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 23 Jul 2020 08:29:18 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 23 Jul 2020 08:29:18 +0900    

Click here for diff

This reverts commit 9e10898, after finding out that buildfarm members  
running SLES 15 on z390 complain on the compression and decompression  
logic of the new test: pipistrelles, barbthroat and steamerduck.  
  
Those hosts are visibly using hardware-specific changes to improve zlib  
performance, requiring more investigation.  
  
Thanks to Tom Lane for the discussion.  
  
Discussion: https://postgr.es/m/20200722093749.GA2564@paquier.xyz  
Backpatch-through: 9.5  

M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql

Fix corner case with PGP decompression in pgcrypto

commit   : bba2e66aec4c8af4ad8b3be0d7cde1a97c974a00    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 22 Jul 2020 14:52:46 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 22 Jul 2020 14:52:46 +0900    

Click here for diff

A compressed stream may end with an empty packet, and PGP decompression  
finished before reading this empty packet in the remaining stream.  This  
caused a failure in pgcrypto, handling this case as corrupted data.  
This commit makes sure to consume such extra data, avoiding a failure  
when decompression the entire stream.  This corner case was reproducible  
with a data length of 16kB, and existed since its introduction in  
e94dd6a.  A cheap regression test is added to cover this case.  
  
Thanks to Jeff Janes for the extra investigation.  
  
Reported-by: Frank Gagnepain  
Author: Kyotaro Horiguchi, Michael Paquier  
Discussion: https://postgr.es/m/16476-692ef7b84e5fb893@postgresql.org  
Backpatch-through: 9.5  

M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql

neqjoinsel must now pass through collation to eqjoinsel.

commit   : 171633ff5d8e21ceda5d824f3c9f13aa18a78505    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 21 Jul 2020 19:40:44 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 21 Jul 2020 19:40:44 -0400    

Click here for diff

Since commit 044c99bc5, eqjoinsel passes the passed-in collation  
to any operators it invokes.  However, neqjoinsel failed to pass  
on whatever collation it got, so that if we invoked a  
collation-dependent operator via that code path, we'd get "could not  
determine which collation to use for string comparison" or the like.  
  
Per report from Justin Pryzby.  Back-patch to v12, like the previous  
commit.  
  
Discussion: https://postgr.es/m/20200721191606.GL5748@telsasoft.com  

M src/backend/utils/adt/selfuncs.c

Assert that we don't insert nulls into attnotnull catalog columns.

commit   : 43ef3c4c360a10226732b62ab3b2097cdf089dce    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 21 Jul 2020 12:38:08 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 21 Jul 2020 12:38:08 -0400    

Click here for diff

The executor checks for this error, and so does the bootstrap catalog  
loader, but we never checked for it in retail catalog manipulations.  
The folly of that has now been exposed, so let's add assertions  
checking it.  Checking in CatalogTupleInsert[WithInfo] and  
CatalogTupleUpdate[WithInfo] should be enough to cover this.  
  
Back-patch to v10; the aforesaid functions didn't exist before that,  
and it didn't seem worth adapting the patch to the oldest branches.  
But given the risk of JIT crashes, I think we certainly need this  
as far back as v11.  
  
Pre-v13, we have to explicitly exclude pg_subscription.subslotname  
and pg_subscription_rel.srsublsn from the checks, since they are  
mismarked.  (Even if we change our mind about applying BKI_FORCE_NULL  
in the branch tips, it doesn't seem wise to have assertions that  
would fire in existing databases.)  
  
Discussion: https://postgr.es/m/298837.1595196283@sss.pgh.pa.us  

M doc/src/sgml/bki.sgml
M src/backend/catalog/indexing.c

Avoid direct C access to possibly-null pg_subscription_rel.srsublsn.

commit   : b7103bbe34aa3d66f4618d0abdee5d3107ea8f91    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 21 Jul 2020 11:40:46 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 21 Jul 2020 11:40:46 -0400    

Click here for diff

This coding technique is unsafe, since we'd be accessing off the end  
of the tuple if the field is null.  SIGSEGV is pretty improbable, but  
perhaps not impossible.  Also, returning garbage for the LSN doesn't  
seem like a great idea, even if callers aren't looking at it today.  
  
Also update docs to point out explicitly that  
pg_subscription.subslotname and pg_subscription_rel.srsublsn  
can be null.  
  
Perhaps we should mark these two fields BKI_FORCE_NULL, so that  
they'd be correctly labeled in databases that are initdb'd in the  
future.  But we can't force that for existing databases, and on  
balance it's not too clear that having a mix of different catalog  
contents in the field would be wise.  
  
Apply to v10 (where this code came in) through v12.  Already  
fixed in v13 and HEAD.  
  
Discussion: https://postgr.es/m/732838.1595278439@sss.pgh.pa.us  

M doc/src/sgml/catalogs.sgml
M src/backend/catalog/pg_subscription.c
M src/include/catalog/pg_subscription_rel.h

Kluge slot_compile_deform() to ignore incorrect attnotnull markings.

commit   : 798b4faefd205a8527afb82c9a87a419d2e06098    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Jul 2020 15:54:24 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Jul 2020 15:54:24 -0400    

Click here for diff

Since we mustn't force an initdb in released branches, there is no  
simple way to correct the markings of pg_subscription.subslotname  
and pg_subscription_rel.srsublsn as attnotnull in existing pre-v13  
installations.  
  
Fortunately, released branches don't rely on attnotnull being correct  
for much.  The planner looks at it in relation_excluded_by_constraints,  
but it'd be difficult to get that to matter for a query on a system  
catalog.  The only place where it's really problematic is in JIT's  
slot_compile_deform(), which can produce incorrect code that crashes  
if there are NULLs in an allegedly not-null column.  
  
Hence, hack up slot_compile_deform() to be specifically aware of  
these two incorrect markings and not trust them.  
  
This applies to v11 and v12; the JIT code didn't exist before that,  
and we've fixed the markings in v13.  
  
Discussion: https://postgr.es/m/229396.1595191345@sss.pgh.pa.us  

M src/backend/jit/llvm/llvmjit_deform.c
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql

Fix construction of updated-columns bitmap in logical replication.

commit   : 71e561bd4bc2d6e85f9a1ae8064d8b190dcad2b4    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Jul 2020 13:40:16 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Jul 2020 13:40:16 -0400    

Click here for diff

Commit b9c130a1f failed to apply the publisher-to-subscriber column  
mapping while checking which columns were updated.  Perhaps less  
significantly, it didn't exclude dropped columns either.  This could  
result in an incorrect updated-columns bitmap and thus wrong decisions  
about whether to fire column-specific triggers on the subscriber while  
applying updates.  In HEAD (since commit 9de77b545), it could also  
result in accesses off the end of the colstatus array, as detected by  
buildfarm member skink.  Fix the logic, and adjust 003_constraints.pl  
so that the problem is exposed in unpatched code.  
  
In HEAD, also add some assertions to check that we don't access off  
the ends of these newly variable-sized arrays.  
  
Back-patch to v10, as b9c130a1f was.  
  
Discussion: https://postgr.es/m/CAH2-Wz=79hKQ4++c5A060RYbjTHgiYTHz=fw6mptCtgghH2gJA@mail.gmail.com  

M src/backend/replication/logical/worker.c
M src/test/subscription/t/003_constraints.pl

doc: Refresh more URLs in the docs

commit   : ab5ad0c71a0f4428d6ecb874b7eaa7a95fe4c17e    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sat, 18 Jul 2020 22:43:45 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sat, 18 Jul 2020 22:43:45 +0900    

Click here for diff

This updates some URLs that are redirections, mostly to an equivalent  
using https.  One URL referring to generalized partial indexes was  
outdated.  
  
Author: Kyotaro Horiguchi  
Discussion: https://postgr.es/m/20200717.121308.1369606287593685396.horikyota.ntt@gmail.com  
Backpatch-through: 9.5  

M doc/src/sgml/acronyms.sgml
M doc/src/sgml/biblio.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/cube.sgml
M doc/src/sgml/dfunc.sgml
M doc/src/sgml/earthdistance.sgml
M doc/src/sgml/external-projects.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/intro.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/nls.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml

doc: Fix description of \copy for psql

commit   : aee67299306e717986f054d149f6cb2f1c447656    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sat, 18 Jul 2020 10:42:50 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sat, 18 Jul 2020 10:42:50 +0900    

Click here for diff

The WHERE clause introduced by 31f3817 was not described.  While on it,  
split the grammar of \copy FROM and TO into two distinct parts for  
clarity as they support different set of options.  
  
Author: Vignesh C  
Discussion: https://postgr.es/m/CALDaNm3zWr=OmxeNqOqfT=uZTSdam_j-gkX94CL8eTNfgUtf6A@mail.gmail.com  
Backpatch-through: 12  

M doc/src/sgml/ref/psql-ref.sgml

Cope with data-offset-less archive files during out-of-order restores.

commit   : 66232220ee6e35aefd9abfb9a551db142c1604d8    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Jul 2020 13:03:50 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Jul 2020 13:03:50 -0400    

Click here for diff

pg_dump produces custom-format archive files that lack data offsets  
when it is unable to seek its output.  Up to now that's been a hazard  
for pg_restore.  But if pg_restore is able to seek in the archive  
file, there is no reason to throw up our hands when asked to restore  
data blocks out of order.  Instead, whenever we are searching for a  
data block, record the locations of the blocks we passed over (that  
is, fill in the missing data-offset fields in our in-memory copy of  
the TOC data).  Then, when we hit a case that requires going  
backwards, we can just seek back.  
  
Also track the furthest point that we've searched to, and seek back  
to there when beginning a search for a new data block.  This avoids  
possible O(N^2) time consumption, by ensuring that each data block  
is examined at most twice.  (On Unix systems, that's at most twice  
per parallel-restore job; but since Windows uses threads here, the  
threads can share block location knowledge, reducing the amount of  
duplicated work.)  
  
We can also improve the code a bit by using fseeko() to skip over  
data blocks during the search.  
  
This is all of some use even in simple restores, but it's really  
significant for parallel pg_restore.  In that case, we require  
seekability of the input already, and we will very probably need  
to do out-of-order restores.  
  
Back-patch to v12, as this fixes a regression introduced by commit  
548e50976.  Before that, parallel restore avoided requesting  
out-of-order restores, so it would work on a data-offset-less  
archive.  Now it will again.  
  
Ideally this patch would include some test coverage, but there are  
other open bugs that need to be fixed before we can extend our  
coverage of parallel restore very much.  Plan to revisit that later.  
  
David Gilman and Tom Lane; reviewed by Justin Pryzby  
  
Discussion: https://postgr.es/m/CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com  

M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_dump/pg_backup_custom.c

Remove manual tracking of file position in pg_dump/pg_backup_custom.c.

commit   : 39a068ce66758c2c2138754e75bebc2f59160edc    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Jul 2020 12:14:28 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Jul 2020 12:14:28 -0400    

Click here for diff

We do not really need to track the file position by hand.  We were  
already relying on ftello() whenever the archive file is seekable,  
while if it's not seekable we don't need the file position info  
anyway because we're not going to be able to re-write the TOC.  
  
Moreover, that tracking was buggy since it failed to account for  
the effects of fseeko().  Somewhat remarkably, that seems not to  
have made for any live bugs up to now.  We could fix the oversights,  
but it seems better to just get rid of the whole error-prone mess.  
  
In itself this is merely code cleanup.  However, it's necessary  
infrastructure for an upcoming bug-fix patch (because that code  
*does* need valid file position after fseeko).  The bug fix  
needs to go back as far as v12; hence, back-patch that far.  
  
Discussion: https://postgr.es/m/CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com  

M src/bin/pg_dump/pg_backup_custom.c

Ensure that distributed timezone abbreviation files are plain ASCII.

commit   : a40733d04707f8d1a6a7dca6430f98b3cab64c63    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Jul 2020 11:03:55 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Jul 2020 11:03:55 -0400    

Click here for diff

We had two occurrences of "Mitteleuropäische Zeit" in Europe.txt,  
though the corresponding entries in Default were spelled  
"Mitteleuropaeische Zeit".  Standardize on the latter spelling to  
avoid questions of which encoding to use.  
  
While here, correct a couple of other trivial inconsistencies between  
the Default file and the supposedly-matching entries in the *.txt  
files, as exposed by some checking with comm(1).  Also, add BDST to  
the Europe.txt file; it previously was only listed in Default.  
None of this has any direct functional effect.  
  
Per complaint from Christoph Berg.  As usual for timezone data patches,  
apply to all branches.  
  
Discussion: https://postgr.es/m/20200716100743.GE3534683@msg.df7cb.de  

M src/timezone/tznames/Antarctica.txt
M src/timezone/tznames/Australia.txt
M src/timezone/tznames/Default
M src/timezone/tznames/Europe.txt

Fix whitespace

commit   : dafa153dda82d335a46e2a3b48247b7fcb308d4e    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Fri, 17 Jul 2020 15:16:13 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Fri, 17 Jul 2020 15:16:13 +0200    

Click here for diff

M src/backend/utils/adt/genfile.c

Switch pg_test_fsync to use binary mode on Windows

commit   : cd113a0b4741b16624a0ffad326538de2f806cca    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 16 Jul 2020 15:52:58 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 16 Jul 2020 15:52:58 +0900    

Click here for diff

pg_test_fsync has always opened files using the text mode on Windows, as  
this is the default mode used if not enforced by _setmode().  
  
This fixes a failure when running pg_test_fsync down to 12 because  
O_DSYNC and the text mode are not able to work together nicely.  We  
fixed the handling of O_DSYNC in 12~ for the tool by switching to the  
concurrent-safe version of fopen() in src/port/ with 0ba06e0.  And  
40cfe86, by enforcing the text mode for compatibility reasons if O_TEXT  
or O_BINARY are not specified by the caller, broke pg_test_fsync.  For  
all versions, this avoids any translation overhead, and pg_test_fsync  
should test binary writes, so it is a gain in all cases.  
  
Note that O_DSYNC is still not handled correctly in ~11, leading to  
pg_test_fsync to show insanely high numbers for open_datasync() (using  
this property it is easy to notice that the binary mode is much  
faster).  This would require a backpatch of 0ba06e0 and 40cfe86, which  
could potentially break existing applications, so this is left out.  
  
There are no TAP tests for this tool yet, so I have checked all builds  
manually using MSVC.  We could invent a new option to run a single  
transaction instead of using a duration of 1s to make the tests a  
maximum short, but this is left as future work.  
  
Thanks to Bruce Momjian for the discussion.  
  
Reported-by: Jeff Janes  
Author: Michael Paquier  
Discussion: https://postgr.es/m/16526-279ded30a230d275@postgresql.org  
Backpatch-through: 9.5  

M src/bin/pg_test_fsync/pg_test_fsync.c

Fix handling of missing files when using pg_rewind with online source

commit   : 92927477ff7062aba67b8139036acdc28603a76a    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 15 Jul 2020 15:17:36 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 15 Jul 2020 15:17:36 +0900    

Click here for diff

When working with an online source cluster, pg_rewind gets a list of all  
the files in the source data directory using a WITH RECURSIVE query,  
returning a NULL result for a file's metadata if it gets removed between  
the moment it is listed in a directory and the moment its metadata is  
obtained with pg_stat_file() (say a recycled WAL segment).  The query  
result was processed in such a way that for each tuple we checked only  
that the first file's metadata was NULL.  This could have two  
consequences, both resulting in a failure of the rewind:  
- If the first tuple referred to a removed file, all files from the  
source would be ignored.  
- Any file actually missing would not be considered as such.  
  
While on it, rework slightly the code so as no values are saved if we  
know that a file is going to be skipped.  
  
Issue introduced by b36805f, so backpatch down to 9.5.  
  
Author: Justin Pryzby, Michael Paquier  
Reviewed-by: Daniel Gustafsson, Masahiko Sawada  
Discussion: https://postgr.es/m/20200713061010.GC23581@telsasoft.com  
Backpatch-through: 9.5  

M src/bin/pg_rewind/libpq_fetch.c

Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.

commit   : de797e823591c8f123dc15d5f18b4c098a5bab12    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 14 Jul 2020 18:56:49 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 14 Jul 2020 18:56:49 -0400    

Click here for diff

reparameterize_path_by_child() failed to reparameterize BitmapAnd  
and BitmapOr paths.  This matters only if such a path is chosen as  
the inside of a nestloop partition-wise join, where we have to pass  
in parameters from the outside of the nestloop.  If that did happen,  
we generated a bad plan that would likely lead to crashes at execution.  
  
This is not entirely reparameterize_path_by_child()'s fault though;  
it's the victim of an ancient decision (my ancient decision, I think)  
to not bother filling in param_info in BitmapAnd/Or path nodes.  That  
caused the function to believe that such nodes and their children  
contain no parameter references and so need not be processed.  
  
In hindsight that decision looks pretty penny-wise and pound-foolish:  
while it saves a few cycles during path node setup, we do commonly  
need the information later.  In particular, by reversing the decision  
and requiring valid param_info data in all nodes of a bitmap path  
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()  
function, which computed the data on-demand.  It's not unlikely that  
that nets out as a savings of cycles in many scenarios.  A couple  
of other things in indxpath.c can be simplified as well.  
  
While here, get rid of some cases in reparameterize_path_by_child()  
that are visibly dead or useless, given that we only care about  
reparameterizing paths that can be on the inside of a parameterized  
nestloop.  This case reminds one of the maxim that untested code  
probably does not work, so I'm unwilling to leave unreachable code  
in this function.  (I did leave the T_Gather case in place even  
though it's not reached in the regression tests.  It's not very  
clear to me when the planner might prefer to put Gather below  
rather than above a nestloop, but at least in principle the case  
might be interesting.)  
  
Per bug #16536, originally from Arne Roland but with a test case  
by Andrew Gierth.  Back-patch to v11 where this code came in.  
  
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org  

M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/partition_join.sql

Fix timing issue with ALTER TABLE's validate constraint

commit   : 1231a0b0ea66df231875a7470a74ac060f5b9b97    
  
author   : David Rowley <drowley@postgresql.org>    
date     : Tue, 14 Jul 2020 17:03:12 +1200    
  
committer: David Rowley <drowley@postgresql.org>    
date     : Tue, 14 Jul 2020 17:03:12 +1200    

Click here for diff

An ALTER TABLE to validate a foreign key in which another subcommand  
already caused a pending table rewrite could fail due to ALTER TABLE  
attempting to validate the foreign key before the actual table rewrite  
takes place.  This situation could result in an error such as:  
  
ERROR:  could not read block 0 in file "base/nnnnn/nnnnn": read only 0 of 8192 bytes  
  
The failure here was due to the SPI call which validates the foreign key  
trying to access an index which is yet to be rebuilt.  
  
Similarly, we also incorrectly tried to validate CHECK constraints before  
the heap had been rewritten.  
  
The fix for both is to delay constraint validation until phase 3, after  
the table has been rewritten.  For CHECK constraints this means a slight  
behavioral change.  Previously ALTER TABLE VALIDATE CONSTRAINT on  
inheritance tables would be validated from the bottom up.  This was  
different from the order of evaluation when a new CHECK constraint was  
added.  The changes made here aligns the VALIDATE CONSTRAINT evaluation  
order for inheritance tables to be the same as ADD CONSTRAINT, which is  
generally top-down.  
  
Reported-by: Nazli Ugur Koyluoglu, using SQLancer  
Discussion: https://postgr.es/m/CAApHDvp%3DZXv8wiRyk_0rWr00skhGkt8vXDrHJYXRMft3TjkxCA%40mail.gmail.com  
Backpatch-through: 9.5 (all supported versions)  

M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql

commit   : 330410ecadfb406c99d08c24efde651e18f653bf    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Tue, 14 Jul 2020 13:17:34 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Tue, 14 Jul 2020 13:17:34 +0900    

Click here for diff

Incorrect function names were referenced.  As this fixes some portions  
of tableam.h, that is mentioned in the docs as something to look at when  
implementing a table AM, backpatch down to 12 where this has been  
introduced.  
  
Author: Hironobu Suzuki  
Discussion: https://postgr.es/m/8fe6d672-28dd-3f1d-7aed-ac2f6d599d3f@interdb.jp  
Backpatch-through: 12  

M src/backend/access/heap/heapam.c
M src/include/access/tableam.h

Cope with lateral references in the quals of a subquery RTE.

commit   : d3b642ad99e958475bab4592db0341badfacee56    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 13 Jul 2020 20:38:21 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 13 Jul 2020 20:38:21 -0400    

Click here for diff

The qual pushdown logic assumed that all Vars in a restriction clause  
must be Vars referencing subquery outputs; but since we introduced  
LATERAL, it's possible for such a Var to be a lateral reference instead.  
This led to an assertion failure in debug builds.  In a non-debug  
build, there might be no ill effects (if qual_is_pushdown_safe decided  
the qual was unsafe anyway), or we could get failures later due to  
construction of an invalid plan.  I've not gone to much length to  
characterize the possible failures, but at least segfaults in the  
executor have been observed.  
  
Given that this has been busted since 9.3 and it took this long for  
anybody to notice, I judge that the case isn't worth going to great  
lengths to optimize.  Hence, fix by just teaching qual_is_pushdown_safe  
that such quals are unsafe to push down, matching the previous behavior  
when it accidentally didn't fail.  
  
Per report from Tom Ellis.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/20200713175124.GQ8220@cloudinit-builder  

M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql

Fix bugs in libpq's management of GSS encryption state.

commit   : e8b22a8a2b65a678d38a120bf7b8cadc144cad17    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 13 Jul 2020 11:57:55 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 13 Jul 2020 11:57:55 -0400    

Click here for diff

GSS-related resources should be cleaned up in pqDropConnection,  
not freePGconn, else the wrong things happen when resetting  
a connection or trying to switch to a different server.  
It's also critical to reset conn->gssenc there.  
  
During connection setup, initialize conn->try_gss at the correct  
place, else switching to a different server won't work right.  
  
Remove now-redundant cleanup of GSS resources around one (and, for  
some reason, only one) pqDropConnection call in connectDBStart.  
  
Per report from Kyotaro Horiguchi that psql would freeze up,  
rather than successfully resetting a GSS-encrypted connection  
after a server restart.  
  
This is YA oversight in commit b0b39f72b, so back-patch to v12.  
  
Discussion: https://postgr.es/m/20200710.173803.435804731896516388.horikyota.ntt@gmail.com  

M src/interfaces/libpq/fe-connect.c

Avoid trying to restore table ACLs and per-column ACLs in parallel.

commit   : 5fea14f4b2c5ca4e4d9def92b17e50fcfbaf3468    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 11 Jul 2020 13:36:50 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 11 Jul 2020 13:36:50 -0400    

Click here for diff

Parallel pg_restore has always supposed that ACL items for different  
objects are independent and can be restored in parallel without  
conflicts.  However, there is one case where this fails: because  
REVOKE on a table is defined to also revoke the privilege(s) at  
column level, we can't restore per-column ACLs till after we restore  
any table-level privileges on their table.  Failure to honor this  
restriction can lead to "tuple concurrently updated" errors during  
parallel restore, or even to the per-column ACLs silently disappearing  
because the table-level REVOKE is executed afterwards.  
  
To fix, add a dependency from each column-level ACL item to its table's  
ACL item, if there is one.  Note that this doesn't fix the hazard  
for pre-existing archive files, only for ones made with a corrected  
pg_dump.  Given that the bug's been there quite awhile without  
field reports, I think this is acceptable.  
  
This requires changing the API of pg_dump's dumpACL() function.  
To keep its argument list from getting even longer, I removed the  
"CatalogId objCatId" argument, which has been unused for ages.  
  
Per report from Justin Pryzby.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/20200706050129.GW4107@telsasoft.com  

M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c

Forbid numeric NaN in jsonpath

commit   : f4ae676e3178124c2bc2b3a3530efe8f3cdbc442    
  
author   : Alexander Korotkov <akorotkov@postgresql.org>    
date     : Sat, 11 Jul 2020 03:21:00 +0300    
  
committer: Alexander Korotkov <akorotkov@postgresql.org>    
date     : Sat, 11 Jul 2020 03:21:00 +0300    

Click here for diff

SQL standard doesn't define numeric Inf or NaN values.  It appears even more  
ridiculous to support then in jsonpath assuming JSON doesn't support these  
values as well.  This commit forbids returning NaN from .double(), which was  
previously allowed.  NaN can't be result of inner-jsonpath computation over  
non-NaNs.  So, we can not expect NaN in the jsonpath output.  
  
Reported-by: Tom Lane  
Discussion: https://postgr.es/m/203949.1591879542%40sss.pgh.pa.us  
Author: Alexander Korotkov  
Reviewed-by: Tom Lane  
Backpatch-through: 12  

M src/backend/utils/adt/jsonb_util.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out

Improve error reporting for jsonpath .double() method

commit   : 3ec5f6b53dfb6acfe66a952edef1b6887da91777    
  
author   : Alexander Korotkov <akorotkov@postgresql.org>    
date     : Sat, 11 Jul 2020 03:20:46 +0300    
  
committer: Alexander Korotkov <akorotkov@postgresql.org>    
date     : Sat, 11 Jul 2020 03:20:46 +0300    

Click here for diff

When jsonpath .double() method detects that numeric or string can't be  
converted to double precision, it throws an error.  This commit makes these  
errors explicitly express the reason of failure.  
  
Discussion: https://postgr.es/m/CAPpHfdtqJtiSXkP7tOXez18NxhLUH_-75bL8%3DOce4Ki%2Bbv7V6Q%40mail.gmail.com  
Author: Alexander Korotkov  
Reviewed-by: Tom Lane  
Backpatch-through: 12  

M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql

Doc: update or remove dead external links.

commit   : a5f18e8c2180ccfff07380efe52f93e9491dd9b5    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 10 Jul 2020 13:16:00 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 10 Jul 2020 13:16:00 -0400    

Click here for diff

Re-point comp.ai.genetic FAQ link to a more stable address.  
  
Remove stale links to AIX documentation; we don't really need to  
tell AIX users how to use their systems.  
  
Remove stale links to HP documentation about SSL.  We've had to  
update those twice before, making it increasingly obvious that  
HP does not intend them to be stable landing points.  They're  
not particularly authoritative, either.  (This change effectively  
reverts bbd3bdba3.)  
  
Daniel Gustafsson and Álvaro Herrera, per a gripe from  
Kyotaro Horiguchi.  Back-patch, since these links are  
just as dead in the back branches.  
  
Discussion: https://postgr.es/m/20200709.161226.204639179120026914.horikyota.ntt@gmail.com  

M doc/src/sgml/geqo.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml

Remove WARNING message from brin_desummarize_range

commit   : ca5001a36188ecaa311c58ddca192117cad9820d    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Thu, 9 Jul 2020 20:13:25 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Thu, 9 Jul 2020 20:13:25 -0400    

Click here for diff

This message was being emitted on the grounds that only crashed  
summarization could cause it, but in reality even an aborted vacuum  
could do it ... which makes it way too noisy, particularly since it  
shows up in regression tests and makes them die.  
  
Reported by Tom Lane.  
Discussion: https://postgr.es/m/489091.1593534251@sss.pgh.pa.us  

M src/backend/access/brin/brin_revmap.c

Tighten up Windows CRLF conversion in our TAP test scripts.

commit   : a3cfb10764bb4fb20d8d57afc9f02c8eca3f570d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Jul 2020 17:38:52 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Jul 2020 17:38:52 -0400    

Click here for diff

Back-patch commits 91bdf499b and ffb4cee43, so that all branches  
agree on when and how to do Windows CRLF conversion.  
  
This should close the referenced thread.  Thanks to Andrew Dunstan  
for discussion/review.  
  
Discussion: https://postgr.es/m/412ae8da-76bb-640f-039a-f3513499e53d@gmx.net  

M src/bin/pg_rewind/t/RewindTest.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm

Fix pg_current_logfile() to not emit a carriage return on Windows.

commit   : 2564e2d08546dfedc1482e10205be58a20ffe3cc    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Jul 2020 16:02:23 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Jul 2020 16:02:23 -0400    

Click here for diff

Due to not having our signals straight about CRLF vs. LF line  
termination, the output of pg_current_logfile() included a trailing  
\r on Windows.  To fix, force the file descriptor it uses into text  
mode.  
  
While here, move a couple of local variable declarations to make  
the function's logic clearer.  
  
In v12 and v13, also back-patch the test added by 1c4e88e2f so that  
this function has some test coverage.  However, the 004_logrotate.pl  
test script doesn't exist before v12, and it didn't seem worth adding  
to older branches just for this.  
  
Per report from Thomas Kellerer.  Back-patch to v10 where this  
function was added.  
  
Discussion: https://postgr.es/m/412ae8da-76bb-640f-039a-f3513499e53d@gmx.net  

M src/backend/utils/adt/misc.c
M src/bin/pg_ctl/t/004_logrotate.pl

doc: Correct the description about the length of pg_stat_activity.query.

commit   : 0df3f9cb92deb81cdb12367d641c0a7855507e5b    
  
author   : Fujii Masao <fujii@postgresql.org>    
date     : Thu, 9 Jul 2020 13:31:33 +0900    
  
committer: Fujii Masao <fujii@postgresql.org>    
date     : Thu, 9 Jul 2020 13:31:33 +0900    

Click here for diff

pg_stat_activity.query text is truncated at 1024 bytes. But previously  
the document described that it's truncated at 1024 characters.  
This was not accurate when considering multibyte characters.  
  
Back-patch to v10 where this inaccurate description was added.  
  
Author: Atsushi Torikoshi  
Reviewed-by: Daniel Gustafsson, Fujii Masao  
Discussion: https://postgr.es/m/cd5b49a5a14e887542f5f569c1c6bde2@oss.nttdata.com  

M doc/src/sgml/monitoring.sgml

doc: Fix incorrect reference to textout in plpgsql examples

commit   : 939db2c1c85879c147a943ad4eb13411c54607b0    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sun, 5 Jul 2020 19:36:18 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sun, 5 Jul 2020 19:36:18 +0900    

Click here for diff

This error has survived for 22 years, and has been introduced by  
da63386.  
  
Reported-by: Erwin Brandstetter  
Discussion: https://postgr.es/m/CAGHENJ57wogGOvGXo5LgWYcqswxafLck8ELqHDR+zrkTPgs_OQ@mail.gmail.com  
Backpatch-through: 9.5  

M doc/src/sgml/plpgsql.sgml

Fix "ignoring return value" complaints from commit 96d1f423f9

commit   : b615b236569055f3b69df85f21b3b909735343e5    
  
author   : Joe Conway <mail@joeconway.com>    
date     : Sat, 4 Jul 2020 13:47:21 -0400    
  
committer: Joe Conway <mail@joeconway.com>    
date     : Sat, 4 Jul 2020 13:47:21 -0400    

Click here for diff

The cfbot and some BF animals are complaining about the previous  
read_binary_file commit because of ignoring return value of ‘fread’.  
So let's make everyone happy by testing the return value even though  
not strictly needed.  
  
Reported by Justin Pryzby, and suggested patch by Tom Lane. Backpatched  
to v11 same as the previous commit.  
  
Reported-By: Justin Pryzby  
Reviewed-By: Tom Lane  
Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com  
Backpatch-through: 11  

M src/backend/utils/adt/genfile.c

Read until EOF vice stat-reported size in read_binary_file

commit   : 015e899a7a671a42f11468afb12e1415415a1da8    
  
author   : Joe Conway <mail@joeconway.com>    
date     : Sat, 4 Jul 2020 06:28:44 -0400    
  
committer: Joe Conway <mail@joeconway.com>    
date     : Sat, 4 Jul 2020 06:28:44 -0400    

Click here for diff

read_binary_file(), used by SQL functions pg_read_file() and friends,  
uses stat to determine file length to read, when not passed an explicit  
length as an argument. This is problematic, for example, if the file  
being read is a virtual file with a stat-reported length of zero.  
Arrange to read until EOF, or StringInfo data string lenth limit, is  
reached instead.  
  
Original complaint and patch by me, with significant review, corrections,  
advice, and code optimizations by Tom Lane. Backpatched to v11. Prior to  
that only paths relative to the data and log dirs were allowed for files,  
so no "zero length" files were reachable anyway.  
  
Reviewed-By: Tom Lane  
Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com  
Backpatch-through: 11  

M contrib/adminpack/expected/adminpack.out
M src/backend/utils/adt/genfile.c

Clamp total-tuples estimates for foreign tables to ensure planner sanity.

commit   : 153c14cdd714381b186d3e3a2272d29f4717be9e    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Jul 2020 19:01:21 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Jul 2020 19:01:21 -0400    

Click here for diff

After running GetForeignRelSize for a foreign table, adjust rel->tuples  
to be at least as large as rel->rows.  This prevents bizarre behavior  
in estimate_num_groups() and perhaps other places, especially in the  
scenario where rel->tuples is zero because pg_class.reltuples is  
(suggesting that ANALYZE has never been run for the table).  As things  
stood, we'd end up estimating one group out of any GROUP BY on such a  
table, whereas the default group-count estimate is more likely to result  
in a sane plan.  
  
Also, clarify in the documentation that GetForeignRelSize has the option  
to override the rel->tuples value if it has a better idea of what to use  
than what is in pg_class.reltuples.  
  
Per report from Jeff Janes.  Back-patch to all supported branches.  
  
Patch by me; thanks to Etsuro Fujita for review  
  
Discussion: https://postgr.es/m/CAMkU=1xNo9cnan+Npxgz0eK7394xmjmKg-QEm8wYG9P5-CcaqQ@mail.gmail.com  

M doc/src/sgml/fdwhandler.sgml
M src/backend/optimizer/path/allpaths.c

Fix temporary tablespaces for shared filesets some more.

commit   : 8b83e01607a85c1099dc08f0f95ce22f715bfa26    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Jul 2020 17:01:34 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 3 Jul 2020 17:01:34 -0400    

Click here for diff

Commit ecd9e9f0b fixed the problem in the wrong place, causing unwanted  
side-effects on the behavior of GetNextTempTableSpace().  Instead,  
let's make SharedFileSetInit() responsible for subbing in the value  
of MyDatabaseTableSpace when the default tablespace is called for.  
  
The convention about what is in the tempTableSpaces[] array is  
evidently insufficiently documented, so try to improve that.  
  
It also looks like SharedFileSetInit() is doing the wrong thing in the  
case where temp_tablespaces is empty.  It was hard-wiring use of the  
pg_default tablespace, but it seems like using MyDatabaseTableSpace  
is more consistent with what happens for other temp files.  
  
Back-patch the reversion of PrepareTempTablespaces()'s behavior to  
9.5, as ecd9e9f0b was.  The changes in SharedFileSetInit() go back  
to v11 where that was introduced.  (Note there is net zero code change  
before v11 from these two patch sets, so nothing to release-note.)  
  
Magnus Hagander and Tom Lane  
  
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com  

M src/backend/commands/tablespace.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/sharedfileset.c

Fix temporary tablespaces for shared filesets

commit   : 28a862e9dcc4559e7c41c8f7d84dc7b3a0c928a2    
  
author   : Magnus Hagander <magnus@hagander.net>    
date     : Fri, 3 Jul 2020 15:09:06 +0200    
  
committer: Magnus Hagander <magnus@hagander.net>    
date     : Fri, 3 Jul 2020 15:09:06 +0200    

Click here for diff

A likely copy/paste error in 98e8b480532 from  back in 2004 would  
cause temp tablespace to be reset to InvalidOid if temp_tablespaces  
was set to the same value as the primary tablespace in the database.  
This would cause shared filesets (such as for parallel hash joins)  
to ignore them, putting the temporary files in the default tablespace  
instead of the configured one. The bug is in the old code, but it  
appears to have been exposed only once we had shared filesets.  
  
Reviewed-By: Daniel Gustafsson  
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com  
Backpatch-through: 9.5  

M src/backend/commands/tablespace.c

Fix ecpg crash with bytea and cursor variables.

commit   : 6425d6f6aa1af8b2db94ade64107fb268a7f114c    
  
author   : Michael Meskes <meskes@postgresql.org>    
date     : Tue, 30 Jun 2020 17:31:08 +0200    
  
committer: Michael Meskes <meskes@postgresql.org>    
date     : Tue, 30 Jun 2020 17:31:08 +0200    

Click here for diff

Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>  

M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/test/expected/sql-bytea.c
M src/interfaces/ecpg/test/expected/sql-bytea.stderr
M src/interfaces/ecpg/test/expected/sql-bytea.stdout
M src/interfaces/ecpg/test/sql/bytea.pgc

doc: clarify that storage parameter values are optional

commit   : d7fe24e34d608fcc0b530c139abfa30576d1db89    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Tue, 30 Jun 2020 12:26:51 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Tue, 30 Jun 2020 12:26:51 -0400    

Click here for diff

In a few cases, the documented syntax specified storage parameter values  
as required.  
  
Reported-by: galiev_mr@taximaxim.ru  
  
Discussion: https://postgr.es/m/159283163235.684.4482737698910467437@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_index.sgml

doc: change pg_upgrade wal_level to be not minimal

commit   : 35d5f698396043fbb252a96b5841d02638a22390    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Tue, 30 Jun 2020 11:55:53 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Tue, 30 Jun 2020 11:55:53 -0400    

Click here for diff

Previously it was specified to be only replica.  
  
Discussion: https://postgr.es/m/20200618180058.GK7349@momjian.us  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/pgupgrade.sgml

Fix documentation of "must be vacuumed within" warning.

commit   : a5e169ad80e1d0651499099df700d709d5a4ef2f    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 27 Jun 2020 22:05:04 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 27 Jun 2020 22:05:04 -0700    

Click here for diff

Warnings start 10M transactions before xidStopLimit, which is 11M  
transactions before wraparound.  The sample WARNING output showed a  
value greater than 11M, and its HINT message predated commit  
25ec228ef760eb91c094cc3b6dea7257cc22ffb5.  Hence, the sample was  
impossible.  Back-patch to 9.5 (all supported versions).  

M doc/src/sgml/maintenance.sgml

doc: mention trigger helper functions in CREATE TRIGGER docs

commit   : 257ee34e733aba0fc371177f1a7fd97f3cb98d74    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Thu, 25 Jun 2020 18:33:28 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Thu, 25 Jun 2020 18:33:28 -0400    

Click here for diff

Reported-by: petermpallesen@gmail.com  
  
Discussion: https://postgr.es/m/159195294959.673.5752624528747900508@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/create_trigger.sgml

docs: clarify that CREATE DATABASE does not copy db permissions

commit   : ed3af2ced97c8d7181ee6c79e3c2c2a7dc00e4d2    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Thu, 25 Jun 2020 18:22:44 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Thu, 25 Jun 2020 18:22:44 -0400    

Click here for diff

That is, those database permissions set by GRANT.  
  
Diagnosed-by: Joseph Nahmias  
  
Discussion: https://postgr.es/m/20200614072613.GA21852@nahmias.net  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/create_database.sgml

Fix compiler warning induced by commit d8b15eeb8.

commit   : ff28f35d2ffb3a485c45d99039a1f17c3bd2a475    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 24 Jun 2020 15:47:30 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 24 Jun 2020 15:47:30 -0400    

Click here for diff

I forgot that INT64_FORMAT can't be used with sscanf on Windows.  
Use the same trick of sscanf'ing into a temp variable as we do in  
some other places in zic.c.  
  
The upstream IANA code avoids the portability problem by relying on  
<inttypes.h>'s SCNdFAST64 macro.  Once we're requiring C99 in all  
branches, we should do likewise and drop this set of diffs from  
upstream.  For now, though, a hack seems fine, since we do not  
actually care about leapseconds anyway.  
  
Discussion: https://postgr.es/m/4e5d1a5b-143e-e70e-a99d-a3b01c1ae7c3@2ndquadrant.com  

M src/timezone/zic.c

Add parens to ConvertToXSegs macro

commit   : 013d324d51ef55cbfff50d4302261b5199cd3198    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 24 Jun 2020 14:00:37 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 24 Jun 2020 14:00:37 -0400    

Click here for diff

The current definition is dangerous.  No bugs exist in our code at  
present, but backpatch to 11 nonetheless where it was introduced.  
  
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>  

M src/backend/access/transam/xlog.c

Undo double-quoting of index names in non-text EXPLAIN output formats.

commit   : d3d8755180b68b6d0d385e8f6989bfbc969a891a    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 22 Jun 2020 11:46:41 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 22 Jun 2020 11:46:41 -0400    

Click here for diff

explain_get_index_name() applied quote_identifier() to the index name.  
This is fine for text output, but the non-text output formats all have  
their own quoting conventions and would much rather start from the  
actual index name.  For example in JSON you'd get something like  
  
       "Index Name": "\"My Index\"",  
  
which is surely not desirable, especially when the same does not  
happen for table names.  Hence, move the responsibility for applying  
quoting out to the callers, where it can go into already-existing  
special code paths for text format.  
  
This changes the API spec for users of explain_get_index_name_hook:  
before, they were supposed to apply quote_identifier() if necessary,  
now they should not.  Research suggests that the only publicly  
available user of the hook is hypopg, and it actually forgot to  
apply quoting anyway, so it's fine.  (In any case, there's no  
behavioral change for the output of a hook as seen in non-text  
EXPLAIN formats, so this won't break any case that programs should  
be relying on.)  
  
Digging in the commit logs, it appears that quoting was included in  
explain_get_index_name's duties when commit 604ffd280 invented it;  
and that was fine at the time because we only had text output format.  
This should have been rethought when non-text formats were invented,  
but it wasn't.  
  
This is a fairly clear bug for users of non-text EXPLAIN formats,  
so back-patch to all supported branches.  
  
Per bug #16502 from Maciek Sakrejda.  Patch by me (based on  
investigation by Euler Taveira); thanks to Julien Rouhaud for review.  
  
Discussion: https://postgr.es/m/16502-57bd1c9f913ed1d1@postgresql.org  

M src/backend/commands/explain.c

Fix masking of SP-GiST pages during xlog consistency check

commit   : 396a44a0153657bd4f5385b184b66b7ada8682ca    
  
author   : Alexander Korotkov <akorotkov@postgresql.org>    
date     : Sat, 20 Jun 2020 17:34:51 +0300    
  
committer: Alexander Korotkov <akorotkov@postgresql.org>    
date     : Sat, 20 Jun 2020 17:34:51 +0300    

Click here for diff

spg_mask() didn't take into account that pd_lower equal to SizeOfPageHeaderData  
is still valid value.  This commit fixes that.  Backpatch to 11, where  
spg_mask() pg_lower check was introduced.  
  
Reported-by: Michael Paquier  
Discussion: https://postgr.es/m/20200615131405.GM52676%40paquier.xyz  
Backpatch-through: 11  

M src/backend/access/spgist/spgxlog.c

Ensure write failure reports no-disk-space

commit   : 5b52008a64414cd161db11f4c0d52fc0d2ee304a    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 19 Jun 2020 16:46:07 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 19 Jun 2020 16:46:07 -0400    

Click here for diff

A few places calling fwrite and gzwrite were not setting errno to ENOSPC  
when reporting errors, as is customary; this led to some failures being  
reported as  
"could not write file: Success"  
which makes us look silly.  Make a few of these places in pg_dump and  
pg_basebackup use our customary pattern.  
  
Backpatch-to: 9.5  
Author: Justin Pryzby <pryzby@telsasoft.com>  
Author: Tom Lane <tgl@sss.pgh.pa.us>  
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>  
Discussion: https://postgr.es/m/20200611153753.GU14879@telsasoft.com  

M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_dump/pg_backup_directory.c

Future-proof regression tests against possibly-missing posixrules file.

commit   : b22ca7648b6fafea51978d206a5989cfe0900211    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 19 Jun 2020 13:55:21 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 19 Jun 2020 13:55:21 -0400    

Click here for diff

The IANA time zone folk have deprecated use of a "posixrules" file in  
the tz database.  While for now it's our choice whether to keep  
supplying one in our own builds, installations built with  
--with-system-tzdata will soon be needing to cope with that file not  
being present, at least on some platforms.  
  
This causes a problem for the horology test, which expected the  
nonstandard POSIX zone spec "CST7CDT" to apply pre-2007 US daylight  
savings rules.  That does happen if the posixrules file supplies such  
information, but otherwise the test produces undesired results.  
To fix, add an explicit transition date rule that matches 2005 practice.  
(We could alternatively have switched the test to use some real time  
zone, but it seems useful to have coverage of this type of zone spec.)  
  
While at it, update a documentation example that also relied on  
"CST7CDT"; use a real-world zone name instead.  Also, document why  
the zone names EST5EDT, CST6CDT, MST7MDT, PST8PDT aren't subject to  
similar failures when "posixrules" is missing.  
  
Back-patch to all supported branches, since the hazard is the same  
for all.  
  
Discussion: https://postgr.es/m/1665379.1592581287@sss.pgh.pa.us  

M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql

Fix deadlock danger when atomic ops are done under spinlock.

commit   : 6cc2866c4cdf6a167be1ed9cb0fff9fce55758d6    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 16:50:37 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 16:50:37 -0700    

Click here for diff

This was a danger only for --disable-spinlocks in combination with  
atomic operations unsupported by the current platform.  
  
While atomics.c was careful to signal that a separate semaphore ought  
to be used when spinlock emulation is active, spin.c didn't actually  
implement that mechanism. That's my (Andres') fault, it seems to have  
gotten lost during the development of the atomic operations support.  
  
Fix that issue and add test for nesting atomic operations inside a  
spinlock.  
  
Author: Andres Freund  
Discussion: https://postgr.es/m/20200605023302.g6v3ydozy5txifji@alap3.anarazel.de  
Backpatch: 9.5-  

M src/backend/storage/lmgr/spin.c
M src/test/regress/regress.c

Add basic spinlock tests to regression tests.

commit   : 008c119928a3eec4e55560562d872117e08f6df3    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 16:36:51 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 16:36:51 -0700    

Click here for diff

As s_lock_test, the already existing test for spinlocks, isn't run in  
an automated fashion (and doesn't test a normal backend environment),  
adding tests that are run as part of a normal regression run is a good  
idea. Particularly in light of several recent and upcoming spinlock  
related fixes.  
  
Currently the new tests are run as part of the pre-existing  
test_atomic_ops() test. That perhaps can be quibbled about, but for  
now seems ok.  
  
The only operations that s_lock_test tests but the new tests don't are  
the detection of a stuck spinlock and S_LOCK_FREE (which is otherwise  
unused, not implemented on all platforms, and will be removed).  
  
This currently contains a test for more than INT_MAX spinlocks (only  
run with --disable-spinlocks), to ensure the recent commit fixing a  
bug with more than INT_MAX spinlock initializations is correct. That  
test is somewhat slow, so we might want to disable it after a few  
days.  
  
It might be worth retiring s_lock_test after this. The added coverage  
of a stuck spinlock probably isn't worth the added complexity?  
  
Author: Andres Freund  
Discussion: https://postgr.es/m/20200606023103.avzrctgv7476xj7i@alap3.anarazel.de  

M src/test/regress/regress.c

Doc: document POSIX-style time zone specifications in full.

commit   : 3b8210da32c4aeca4fe0ba9fdb5a0112b313b7a8    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 18 Jun 2020 16:27:18 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 18 Jun 2020 16:27:18 -0400    

Click here for diff

We'd glossed over most of this complexity for years, but it's hard  
to avoid writing it all down now, so that we can explain what happens  
when there's no "posixrules" file in the IANA time zone database.  
That was at best a tiny minority situation till now, but it's likely  
to become quite common in the future, so we'd better explain it.  
  
Nonetheless, we don't really encourage people to use POSIX zone specs;  
picking a named zone is almost always what you really want, unless  
perhaps you're stuck with an out-of-date zone database.  Therefore,  
let's shove all this detail into an appendix.  
  
Patch by me; thanks to Robert Haas for help with some awkward wording.  
  
Discussion: https://postgr.es/m/1390.1562258309@sss.pgh.pa.us  

M doc/src/sgml/datatype.sgml
M doc/src/sgml/datetime.sgml

Fix oldest xmin and LSN computation across repslots after advancing

commit   : 16f43122d46fc1f39af3f46024495de0b27ba751    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 18 Jun 2020 16:35:32 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 18 Jun 2020 16:35:32 +0900    

Click here for diff

Advancing a replication slot did not recompute the oldest xmin and LSN  
values across replication slots, preventing resource removal like  
segments not recycled at checkpoint time.  The original commit that  
introduced the slot advancing in 9c7d06d never did the update of those  
oldest values, and b0afdca removed this code.  
  
This commit adds a TAP test to check segment recycling with advancing  
for physical slots, enforcing an extra segment switch before advancing  
to check if the segment gets correctly recycled after a checkpoint.  
  
Reported-by: Andres Freund  
Reviewed-by: Alexey Kondratov, Kyptaro Horiguchi  
Discussion: https://postgr.es/m/20200609171904.kpltxxvjzislidks@alap3.anarazel.de  
Backpatch-through: 11  

M src/backend/replication/slotfuncs.c
M src/test/recovery/t/001_stream_rep.pl

Sync our copy of the timezone library with IANA release tzcode2020a.

commit   : 812a84d1cca1f8e43adc2e6a7dfe2ae83e2c1a29    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 17 Jun 2020 18:29:29 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 17 Jun 2020 18:29:29 -0400    

Click here for diff

This absorbs a leap-second-related bug fix in localtime.c, and  
teaches zic to handle an expiration marker in the leapseconds file.  
Neither are of any interest to us (for the foreseeable future  
anyway), but we need to stay more or less in sync with upstream.  
  
Also adjust some over-eager changes in the README from commit 957338418.  
I have no intention of making changes that require C99 in this code,  
until such time as all the live back branches require C99.  Otherwise  
back-patching will get too exciting.  
  
For the same reason, absorb assorted whitespace and other cosmetic  
changes from HEAD into the back branches; mostly this reflects use of  
improved versions of pgindent.  
  
All in all then, quite a boring update.  But I figured I'd get it  
done while I was looking at this code.  

M src/timezone/README
M src/timezone/localtime.c
M src/timezone/pgtz.c
M src/timezone/strftime.c
M src/timezone/zic.c

spinlock emulation: Fix bug when more than INT_MAX spinlocks are initialized.

commit   : b91cfaa34f44386786c64bbf80d296b84ecc811e    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 15:25:49 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 15:25:49 -0700    

Click here for diff

Once the counter goes negative we ended up with spinlocks that errored  
out on first use (due to check in tas_sema).  
  
Author: Andres Freund  
Reviewed-By: Robert Haas  
Discussion: https://postgr.es/m/20200606023103.avzrctgv7476xj7i@alap3.anarazel.de  
Backpatch: 9.5-  

M src/backend/storage/lmgr/spin.c

Doc: fix copy-and-pasteo in ecpg docs.

commit   : dab8561cded852179a3f5fd87c8f31416d5cfdaf    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 16 Jun 2020 16:41:11 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 16 Jun 2020 16:41:11 -0400    

Click here for diff

The synopsis for PGTYPESinterval_free() used the wrong name.  
  
Discussion: https://postgr.es/m/159231203030.679.3061023914894071953@wrigleys.postgresql.org  

M doc/src/sgml/ecpg.sgml

Fix buffile.c error handling.

commit   : 28ee126699446e5eb9ef4d83a9d195203e944a18    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Tue, 16 Jun 2020 13:50:56 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Tue, 16 Jun 2020 13:50:56 +1200    

Click here for diff

Convert buffile.c error handling to use ereport.  This fixes cases where  
I/O errors were indistinguishable from EOF or not reported.  Also remove  
"%m" from error messages where errno would be bogus.  While we're  
modifying those strings, add block numbers and short read byte counts  
where appropriate.  
  
Back-patch to all supported releases.  
  
Reported-by: Amit Khandekar <amitdkhan.pg@gmail.com>  
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>  
Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com>  
Reviewed-by: Robert Haas <robertmhaas@gmail.com>  
Reviewed-by: Ibrar Ahmed <ibrar.ahmad@gmail.com>  
Reviewed-by: Michael Paquier <michael@paquier.xyz>  
Discussion: https://postgr.es/m/CA%2BhUKGJE04G%3D8TLK0DLypT_27D9dR8F1RQgNp0jK6qR0tZGWOw%40mail.gmail.com  

M src/backend/access/gist/gistbuildbuffers.c
M src/backend/executor/nodeHashjoin.c
M src/backend/storage/file/buffile.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/sharedtuplestore.c
M src/backend/utils/sort/tuplestore.c

pg_upgrade: set vacuum_defer_cleanup_age to zero

commit   : 8e933596c9e425ead61cb2800d7b889cdcf83b6a    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 15 Jun 2020 20:59:40 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 15 Jun 2020 20:59:40 -0400    

Click here for diff

Non-zero vacuum_defer_cleanup_age values cause pg_upgrade freezing of  
the system catalogs to be incomplete, or do nothing.  This will cause  
the upgrade to fail in confusing ways.  
  
Reported-by: Laurenz Albe  
  
Discussion: https://postgr.es/m/7d6f6c22ba05ce0c526e9e8b7bfa8105e7da45e6.camel@cybertec.at  
  
Backpatch-through: 9.5  

M src/bin/pg_upgrade/server.c

Doc: Add references for SI and SSI.

commit   : eb1286413c0abfabf51788215186841f5dbb345b    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 15 Jun 2020 11:33:13 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 15 Jun 2020 11:33:13 +1200    

Click here for diff

Our documentation failed to point out that REPEATABLE READ is really  
snapshot isolation, which might be important to some users.  Point to  
the standard reference paper for this complicated topic.  
  
Likewise, add a reference to the VLDB paper about PostgreSQL SSI, for  
technical information about our SSI implementation and how it compares  
to S2PL.  
  
While here, add a note about catalog access using a lower isolation  
level, per recent user complaint.  
  
Back-patch to all releases.  
  
Reported-by: Kyle Kingsbury <aphyr@jepsen.io>  
Reviewed-by: Andres Freund <andres@anarazel.de>  
Reviewed-by: Peter Geoghegan <pg@bowt.ie>  
Reviewed-by: Tatsuo Ishii <ishii@sraoss.co.jp>  
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443%40jepsen.io  
Discussion: https://postgr.es/m/16454-9408996bb1750faf%40postgresql.org  

M doc/src/sgml/biblio.sgml
M doc/src/sgml/mvcc.sgml

Fix behavior of float aggregates for single Inf or NaN inputs.

commit   : 874372a941a6ee31ba4034db0367e0929e5f49bc    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 13 Jun 2020 13:43:24 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 13 Jun 2020 13:43:24 -0400    

Click here for diff

When there is just one non-null input value, and it is infinity or NaN,  
aggregates such as stddev_pop and covar_pop should produce a NaN  
result, because the calculation is not well-defined.  They used to do  
so, but since we adopted Youngs-Cramer aggregation in commit e954a727f,  
they produced zero instead.  That's an oversight, so fix it.  Add tests  
exercising these edge cases.  
  
Affected aggregates are  
  
 var_pop(double precision)  
 stddev_pop(double precision)  
 var_pop(real)  
 stddev_pop(real)  
 regr_sxx(double precision,double precision)  
 regr_syy(double precision,double precision)  
 regr_sxy(double precision,double precision)  
 regr_r2(double precision,double precision)  
 regr_slope(double precision,double precision)  
 regr_intercept(double precision,double precision)  
 covar_pop(double precision,double precision)  
 corr(double precision,double precision)  
  
Back-patch to v12 where the behavior change was accidentally introduced.  
  
Report and patch by me; thanks to Dean Rasheed for review.  
  
Discussion: https://postgr.es/m/353062.1591898766@sss.pgh.pa.us  

M src/backend/utils/adt/float.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql

doc: remove xreflabels from commits 75fcdd2ae2 and 85af628da5

commit   : 81cd796b825cf8d09bb6acea75482f9583e6a951    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Thu, 11 Jun 2020 18:19:25 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Thu, 11 Jun 2020 18:19:25 -0400    

Click here for diff

xreflabels prevent references to the chapter numbers of sections id's.  
It should only be used in specific cases.  
  
Discussion: https://postgr.es/m/8315c0ca-7758-8823-fcb6-f37f9413e6b6@2ndquadrant.com  
  
Backpatch-through: 9.5  

M doc/src/sgml/ecpg.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/spi.sgml
M doc/src/sgml/storage.sgml
M doc/src/sgml/vacuumlo.sgml

Fix mishandling of NaN counts in numeric_[avg_]combine.

commit   : 4284e1184692019da656a0f78748363fd2b40f47    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 11 Jun 2020 17:38:42 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 11 Jun 2020 17:38:42 -0400    

Click here for diff

When merging two NumericAggStates, the code missed adding the new  
state's NaNcount unless its N was also nonzero; since those counts  
are independent, this is wrong.  
  
This would only have visible effect if some partial aggregate scans  
found only NaNs while earlier ones found only non-NaNs; then we could  
end up falsely deciding that there were no NaNs and fail to return a  
NaN final result as expected.  That's pretty improbable, so it's no  
surprise this hasn't been reported from the field.  Still, it's a bug.  
  
I didn't try to produce a regression test that would show the bug,  
but I did notice that these functions weren't being reached at all  
in our regression tests, so I improved the tests to at least  
exercise them.  With these additions, I see pretty complete code  
coverage on the aggregation-related functions in numeric.c.  
  
Back-patch to 9.6 where this code was introduced.  (I only added  
the improved test case as far back as v10, though, since the  
relevant part of aggregates.sql isn't there at all in 9.6.)  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql

Avoid update conflict out serialization anomalies.

commit   : e620a38c23b9f7338d93ffd4c51bacc31973ac35    
  
author   : Peter Geoghegan <pg@bowt.ie>    
date     : Thu, 11 Jun 2020 10:09:43 -0700    
  
committer: Peter Geoghegan <pg@bowt.ie>    
date     : Thu, 11 Jun 2020 10:09:43 -0700    

Click here for diff

SSI's HeapCheckForSerializableConflictOut() test failed to correctly  
handle conditions involving a concurrently inserted tuple which is later  
concurrently updated by a separate transaction .  A SELECT statement  
that called HeapCheckForSerializableConflictOut() could end up using the  
same XID (updater's XID) for both the original tuple, and the successor  
tuple, missing the XID of the xact that created the original tuple  
entirely.  This only happened when neither tuple from the chain was  
visible to the transaction's MVCC snapshot.  
  
The observable symptoms of this bug were subtle.  A pair of transactions  
could commit, with the later transaction failing to observe the effects  
of the earlier transaction (because of the confusion created by the  
update to the non-visible row).  This bug dates all the way back to  
commit dafaa3ef, which added SSI.  
  
To fix, make sure that we check the xmin of concurrently inserted tuples  
that happen to also have been updated concurrently.  
  
Author: Peter Geoghegan  
Reported-By: Kyle Kingsbury  
Reviewed-By: Thomas Munro  
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443@jepsen.io  
Backpatch: All supported versions  

M src/backend/storage/lmgr/predicate.c
A src/test/isolation/expected/update-conflict-out.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/update-conflict-out.spec

Fix typos.

commit   : b7ed1d9944005ec443c94d3536be42473309b786    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Thu, 11 Jun 2020 14:10:43 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Thu, 11 Jun 2020 14:10:43 +0530    

Click here for diff

Reported-by: John Naylor  
Author: John Naylor  
Backpatch-through: 9.5  
Discussion: https://postgr.es/m/CACPNZCtRuvs6G+EYqejhVJgBq2AKeZdXRVJsbX4syhO9gn5SNQ@mail.gmail.com  

M src/test/regress/input/largeobject.source
M src/test/regress/output/largeobject.source
M src/test/regress/output/largeobject_1.source

Update description of parameter password_encryption

commit   : 17d8cf2897443874fb66d2e4895a049cf961ec58    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Wed, 10 Jun 2020 11:57:41 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Wed, 10 Jun 2020 11:57:41 +0200    

Click here for diff

The previous description string still described the pre-PostgreSQL  
10 (pre eb61136dc75a76caef8460fa939244d8593100f2) behavior of  
selecting between encrypted and unencrypted, but it is now choosing  
between encryption algorithms.  

M src/backend/utils/misc/guc.c

Avoid need for valgrind suppressions for pg_atomic_init_u64 on some platforms.

commit   : 980a3cd48a08689f0526a7ed0537899e155bd9f4    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 19:52:19 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Mon, 8 Jun 2020 19:52:19 -0700    

Click here for diff

Previously we used pg_atomic_write_64_impl inside  
pg_atomic_init_u64. That works correctly, but on platforms without  
64bit single copy atomicity it could trigger spurious valgrind errors  
about uninitialized memory, because we use compare_and_swap for atomic  
writes on such platforms.  
  
I previously suppressed one instance of this problem (6c878edc1df),  
but as Tom reports that wasn't enough. As the atomic variable cannot  
yet be concurrently accessible during initialization, it seems better  
to have pg_atomic_init_64_impl set the value directly.  
  
Change pg_atomic_init_u32_impl for symmetry.  
  
Reported-By: Tom Lane  
Author: Andres Freund  
Discussion: https://postgr.es/m/1714601.1591503815@sss.pgh.pa.us  
Backpatch: 9.5-  

M src/include/port/atomics/generic.h
M src/tools/valgrind.supp

Fix locking bugs that could corrupt pg_control.

commit   : 72766ad6397a1795f2739bda95370b02d5738a3f    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 8 Jun 2020 13:57:24 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 8 Jun 2020 13:57:24 +1200    

Click here for diff

The redo routines for XLOG_CHECKPOINT_{ONLINE,SHUTDOWN} must acquire  
ControlFileLock before modifying ControlFile->checkPointCopy, or the  
checkpointer could write out a control file with a bad checksum.  
  
Likewise, XLogReportParameters() must acquire ControlFileLock before  
modifying ControlFile and calling UpdateControlFile().  
  
Back-patch to all supported releases.  
  
Author: Nathan Bossart <bossartn@amazon.com>  
Author: Fujii Masao <masao.fujii@oss.nttdata.com>  
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>  
Reviewed-by: Michael Paquier <michael@paquier.xyz>  
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>  
Discussion: https://postgr.es/m/70BF24D6-DC51-443F-B55A-95735803842A%40amazon.com  

M src/backend/access/transam/xlog.c

Doc: Update example symptom of systemd misconfiguration.

commit   : b944b1d1a9ac4a08b790833522d5edd165f740d1    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 8 Jun 2020 13:20:46 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Mon, 8 Jun 2020 13:20:46 +1200    

Click here for diff

In PostgreSQL 10, we stopped using System V semaphores on Linux  
systems.  Update the example we give of an error message from a  
misconfigured system to show what people are most likely to see these  
days.  
  
Back-patch to 10, where PREFERRED_SEMAPHORES=UNNAMED_POSIX arrived.  
  
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>  
Discussion: https://postgr.es/m/CA%2BhUKGLmJUSwybaPQv39rB8ABpqJq84im2UjZvyUY4feYhpWMw%40mail.gmail.com  

M doc/src/sgml/runtime.sgml

MSVC: Avoid warning when testing a TAP suite without PROVE_FLAGS.

commit   : fb2641f8aa7f8544a2d00909e19fa83d42394154    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sun, 7 Jun 2020 16:27:13 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sun, 7 Jun 2020 16:27:13 -0700    

Click here for diff

Commit 7be5d8df1f74b78620167d3abf32ee607e728919 surfaced the logic  
error, which had no functional implications, by adding "use warnings".  
The buildfarm always customizes PROVE_FLAGS, so the warning did not  
appear there.  Back-patch to 9.5 (all supported versions).  

M src/tools/msvc/vcregress.pl

Try to read data from the socket in pqSendSome's write_failed paths.

commit   : 2edf14f5ac346873e80d06bbaf1b4840bd9d4f6c    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 7 Jun 2020 13:44:13 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 7 Jun 2020 13:44:13 -0400    

Click here for diff

Even when we've concluded that we have a hard write failure on the  
socket, we should continue to try to read data.  This gives us an  
opportunity to collect any final error message that the backend might  
have sent before closing the connection; moreover it is the job of  
pqReadData not pqSendSome to close the socket once EOF is detected.  
  
Due to an oversight in 1f39a1c06, pqSendSome failed to try to collect  
data in the case where we'd already set write_failed.  The problem was  
masked for ordinary query operations (which really only make one write  
attempt anyway), but COPY to the server would continue to send data  
indefinitely after a mid-COPY connection loss.  
  
Hence, add pqReadData calls into the paths where pqSendSome drops data  
because of write_failed.  If we've lost the connection, this will  
eventually result in closing the socket and setting CONNECTION_BAD,  
which will cause PQputline and siblings to report failure, allowing  
the application to terminate the COPY sooner.  (Basically this restores  
what happened before 1f39a1c06.)  
  
There are related issues that this does not solve; for example, if the  
backend sends an error but doesn't drop the connection, we did and  
still will keep pumping COPY data as long as the application sends it.  
Fixing that will require application-visible behavior changes though,  
and anyway it's an ancient behavior that we've had few complaints about.  
For now I'm just trying to fix the regression from 1f39a1c06.  
  
Per a complaint from Andres Freund.  Back-patch into v12 where  
1f39a1c06 came in.  
  
Discussion: https://postgr.es/m/20200603201242.ofvm4jztpqytwfye@alap3.anarazel.de  

M src/interfaces/libpq/fe-misc.c

doc: Clean up title case use

commit   : a00222f07b142484fb2d36e6e2343db7c9f96d69    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Sun, 7 Jun 2020 13:18:36 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Sun, 7 Jun 2020 13:18:36 +0200    

Click here for diff

M doc/src/sgml/features.sgml

commit   : 8414f41d3485aaed2985cf06cea9a07c50eec262    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Sun, 7 Jun 2020 11:16:51 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Sun, 7 Jun 2020 11:16:51 +0200    

Click here for diff

M doc/src/sgml/func.sgml

Refresh function name in CRC-associated Valgrind suppressions.

commit   : 0b70f0302b674f006122953ff14bf23a53bc9b5a    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Fri, 5 Jun 2020 20:10:53 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Fri, 5 Jun 2020 20:10:53 -0700    

Click here for diff

Back-patch to 9.5, where commit 4f700bcd20c087f60346cb8aefd0e269be8e2157  
first appeared.  
  
Reviewed by Tom Lane.  Reported by Andrew Dunstan.  
  
Discussion: https://postgr.es/m/4dfabec2-a3ad-0546-2d62-f816c97edd0c@2ndQuadrant.com  

M src/tools/valgrind.supp

Add unlikely() to CHECK_FOR_INTERRUPTS()

commit   : 8401ad52350e8a5f0bb71942f2ac7636969bf8f3    
  
author   : Joe Conway <mail@joeconway.com>    
date     : Fri, 5 Jun 2020 16:49:28 -0400    
  
committer: Joe Conway <mail@joeconway.com>    
date     : Fri, 5 Jun 2020 16:49:28 -0400    

Click here for diff

Add the unlikely() branch hint macro to CHECK_FOR_INTERRUPTS().  
Backpatch to REL_10_STABLE where we first started using unlikely().  
  
Discussion: https://www.postgresql.org/message-id/flat/8692553c-7fe8-17d9-cbc1-7cddb758f4c6%40joeconway.com  

M src/include/miscadmin.h

Use query collation, not column's collation, while examining statistics.

commit   : 022cd0bfd33968f2b004106cfeaa3b2951e7f322    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 5 Jun 2020 16:18:50 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 5 Jun 2020 16:18:50 -0400    

Click here for diff

Commit 5e0928005 changed the planner so that, instead of blindly using  
DEFAULT_COLLATION_OID when invoking operators for selectivity estimation,  
it would use the collation of the column whose statistics we're  
considering.  This was recognized as still being not quite the right  
thing, but it seemed like a good incremental improvement.  However,  
shortly thereafter we introduced nondeterministic collations, and that  
creates cases where operators can fail if they're passed the wrong  
collation.  We don't want planning to fail in cases where the query itself  
would work, so this means that we *must* use the query's collation when  
invoking operators for estimation purposes.  
  
The only real problem this creates is in ineq_histogram_selectivity, where  
the binary search might produce a garbage answer if we perform comparisons  
using a different collation than the column's histogram is ordered with.  
However, when the query's collation is significantly different from the  
column's default collation, the estimate we previously generated would be  
pretty irrelevant anyway; so it's not clear that this will result in  
noticeably worse estimates in practice.  (A follow-on patch will improve  
this situation in HEAD, but it seems too invasive for back-patch.)  
  
The patch requires changing the signatures of mcv_selectivity and allied  
functions, which are exported and very possibly are used by extensions.  
In HEAD, I just did that, but an API/ABI break of this sort isn't  
acceptable in stable branches.  Therefore, in v12 the patch introduces  
"mcv_selectivity_ext" and so on, with signatures matching HEAD, and makes  
the old functions into wrappers that assume DEFAULT_COLLATION_OID should  
be used.  That does not match the prior behavior, but it should avoid risk  
of failure in most cases.  (In practice, I think most extension datatypes  
aren't collation-aware, so the change probably doesn't matter to them.)  
  
Per report from James Lucas.  Back-patch to v12 where the problem was  
introduced.  
  
Discussion: https://postgr.es/m/CAAFmbbOvfi=wMM=3qRsPunBSLb8BFREno2oOzSBS=mzfLPKABw@mail.gmail.com  

M src/backend/utils/adt/like_support.c
M src/backend/utils/adt/network_selfuncs.c
M src/backend/utils/adt/selfuncs.c
M src/include/utils/selfuncs.h

Preserve pg_index.indisreplident across REINDEX CONCURRENTLY

commit   : 75f14792408211f2ef6bf2dac31796a8a1ec3efb    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Fri, 5 Jun 2020 10:29:27 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Fri, 5 Jun 2020 10:29:27 +0900    

Click here for diff

If the flag value is lost, logical decoding would work the same way as  
REPLICA IDENTITY NOTHING, meaning that no old tuple values would be  
included in the changes anymore produced by logical decoding.  
  
Author: Michael Paquier  
Reviewed-by: Euler Taveira  
Discussion: https://postgr.es/m/20200603065340.GK89559@paquier.xyz  
Backpatch-through: 12  

M src/backend/catalog/index.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql

Reject "23:59:60.nnn" in datetime input.

commit   : a958b07bc4533d8c80b0f10cc4a3b209002b387f    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 4 Jun 2020 16:42:08 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 4 Jun 2020 16:42:08 -0400    

Click here for diff

It's intentional that we don't allow values greater than 24 hours,  
while we do allow "24:00:00" as well as "23:59:60" as inputs.  
However, the range check was miscoded in such a way that it would  
accept "23:59:60.nnn" with a nonzero fraction.  For time or timetz,  
the stored result would then be greater than "24:00:00" which would  
fail dump/reload, not to mention possibly confusing other operations.  
  
Fix by explicitly calculating the result and making sure it does not  
exceed 24 hours.  (This calculation is redundant with what will happen  
later in tm2time or tm2timetz.  Maybe someday somebody will find that  
annoying enough to justify refactoring to avoid the duplication; but  
that seems too invasive for a back-patched bug fix, and the cost is  
probably unmeasurable anyway.)  
  
Note that this change also rejects such input as the time portion  
of a timestamp(tz) value.  
  
Back-patch to v10.  The bug is far older, but to change this pre-v10  
we'd need to ensure that the logic behaves sanely with float timestamps,  
which is possibly nontrivial due to roundoff considerations.  
Doesn't really seem worth troubling with.  
  
Per report from Christoph Berg.  
  
Discussion: https://postgr.es/m/20200520125807.GB296739@msg.df7cb.de  

M src/backend/utils/adt/date.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/date.h
M src/test/regress/expected/time.out
M src/test/regress/expected/timetz.out
M src/test/regress/sql/time.sql
M src/test/regress/sql/timetz.sql

Fix instance of elog() called while holding a spinlock

commit   : 03aa25b6e34b5f5184cfd0df71f1096c66523b72    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 4 Jun 2020 10:18:02 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 4 Jun 2020 10:18:02 +0900    

Click here for diff

This broke the project rule to not call any complex code while a  
spinlock is held.  Issue introduced by b89e151.  
  
Discussion: https://postgr.es/m/20200602.161518.1399689010416646074.horikyota.ntt@gmail.com  
Backpatch-through: 9.5  

M src/backend/replication/logical/logical.c

Don't call palloc() while holding a spinlock, either.

commit   : 3d474a07934c6da9f1adca3fb7010663e6249810    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 3 Jun 2020 12:36:00 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 3 Jun 2020 12:36:00 -0400    

Click here for diff

Fix some more violations of the "only straight-line code inside a  
spinlock" rule.  These are hazardous not only because they risk  
holding the lock for an excessively long time, but because it's  
possible for palloc to throw elog(ERROR), leaving a stuck spinlock  
behind.  
  
copy_replication_slot() had two separate places that did pallocs  
while holding a spinlock.  We can make the code simpler and safer  
by copying the whole ReplicationSlot struct into a local variable  
while holding the spinlock, and then referencing that copy.  
(While that's arguably more cycles than we really need to spend  
holding the lock, the struct isn't all that big, and this way seems  
far more maintainable than copying fields piecemeal.  Anyway this  
is surely much cheaper than a palloc.)  That bug goes back to v12.  
  
InvalidateObsoleteReplicationSlots() not only did a palloc while  
holding a spinlock, but for extra sloppiness then leaked the memory  
--- probably for the lifetime of the checkpointer process, though  
I didn't try to verify that.  Fortunately that silliness is new  
in HEAD.  
  
pg_get_replication_slots() had a cosmetic violation of the rule,  
in that it only assumed it's safe to call namecpy() while holding  
a spinlock.  Still, that's a hazard waiting to bite somebody, and  
there were some other cosmetic coding-rule violations in the same  
function, so clean it up.  I back-patched this as far as v10; the  
code exists before that but it looks different, and this didn't  
seem important enough to adapt the patch further back.  
  
Discussion: https://postgr.es/m/20200602.161518.1399689010416646074.horikyota.ntt@gmail.com  

M src/backend/replication/slotfuncs.c
M src/include/replication/slot.h

doc: Move wal_init_zero and wal_recycle descriptions to proper section.

commit   : 53f32ea4d38b07e0669d0c9467a2a174bddc9b26    
  
author   : Fujii Masao <fujii@postgresql.org>    
date     : Wed, 3 Jun 2020 09:59:43 +0900    
  
committer: Fujii Masao <fujii@postgresql.org>    
date     : Wed, 3 Jun 2020 09:59:43 +0900    

Click here for diff

The group of wal_init_zero and wal_recycle is WAL_SETTINGS in guc.c,  
but previously their documents were located in  
"Replication"/"Sending Servers" section. This commit moves them to  
the proper section "Write Ahead Log"/"Settings".  
  
Back-patch to v12 where wal_init_zero and wal_recycle parameters  
were introduced.  
  
Author: Fujii Masao  
Discussion: https://postgr.es/m/b5190ab4-a169-6a42-0e49-aed0807c8976@oss.nttdata.com  

M doc/src/sgml/config.sgml

Fix use-after-release mistake in currtid() and currtid2() for views

commit   : 894041eb262fa3c730db11bbd6a93798a905556c    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 1 Jun 2020 14:41:25 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 1 Jun 2020 14:41:25 +0900    

Click here for diff

This issue has been present since the introduction of this code as of  
a3519a2 from 2002, and has been found by buildfarm member prion that  
uses RELCACHE_FORCE_RELEASE via the tests introduced recently in  
e786be5.  
  
Discussion: https://postgr.es/m/20200601022055.GB4121@paquier.xyz  
Backpatch-through: 9.5  

M src/backend/utils/adt/tid.c

Fix crashes with currtid() and currtid2()

commit   : 95e389b3c2ea35ad60419f285fd5c1d0511142e7    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 1 Jun 2020 10:32:53 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 1 Jun 2020 10:32:53 +0900    

Click here for diff

A relation that has no storage initializes rd_tableam to NULL, which  
caused those two functions to crash because of a pointer dereference.  
Note that in 11 and older versions, this has always failed with a  
confusing error "could not open file".  
  
These two functions are used by the Postgres ODBC driver, which requires  
them only when connecting to a backend strictly older than 8.1.  When  
connected to 8.2 or a newer version, the driver uses a RETURNING clause  
instead whose support has been added in 8.2, so it should be possible to  
just remove both functions in the future.  This is left as an issue to  
address later.  
  
While on it, add more regression tests for those functions as we never  
really had coverage for them, and for aggregates of TIDs.  
  
Reported-by: Jaime Casanova, via sqlsmith  
Author: Michael Paquier  
Reviewed-by: Álvaro Herrera  
Discussion: https://postgr.es/m/CAJGNTeO93u-5APMga6WH41eTZ3Uee9f3s8dCpA-GSSqNs1b=Ug@mail.gmail.com  
Backpatch-through: 12  

M src/backend/utils/adt/tid.c
A src/test/regress/expected/tid.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/tid.sql

Make install-tests target work with vpath builds

commit   : 7f92218b8a0c3b9232dbb37f6026bfeaf2346e7c    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Sun, 31 May 2020 18:33:00 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Sun, 31 May 2020 18:33:00 -0400    

Click here for diff

Also add a top-level install-tests target.  
  
Backpatch to all live branches.  
  
Craig Ringer, tweaked by me.  

M GNUmakefile.in
M src/test/regress/GNUmakefile

Doc: Mention about caveats of --concurrently on reindexdb page

commit   : 38be24228dcdde935e6090ac4d7f85ceea58d914    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sun, 31 May 2020 10:48:28 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sun, 31 May 2020 10:48:28 +0900    

Click here for diff

The documentation of REINDEX includes a complete description of  
CONCURRENTLY and its advantages as well as its disadvantages, but  
reindexdb was not really clear about all that.  
  
From discussion with Tom Lane, based on a report from Andrey Klychkov.  
  
Discussion: https://postgr.es/m/1590486572.205117372@f500.i.mail.ru  
Backpatch-through: 12  

M doc/src/sgml/ref/reindexdb.sgml

llvmjit: Fix building against LLVM 11 by removing unnecessary include.

commit   : 59e390cac97555e54457874ef27092012cb48b18    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Thu, 28 May 2020 15:08:12 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Thu, 28 May 2020 15:08:12 -0700    

Click here for diff

LLVM has removed this header, in the branch that will become llvm  
11. But as it turns out we didn't actually need it, so just remove it.  
  
Author: Jesse Zhang <sbjesse@gmail.com>  
Discussion: https://postgr.es/m/CAGf+fX7bvtP0YXMu7pOsu_NwhxW6dArTkxb=jt7M2-UJkyJ_3g@mail.gmail.com  
Backpatch: 11, where JIT support using llvm was introduced.  

M src/backend/jit/llvm/llvmjit_inline.cpp

commit   : e8eb485954463c2ed1d8dee48ae58063dffec972    
  
author   : Joe Conway <mail@joeconway.com>    
date     : Thu, 28 May 2020 13:44:59 -0400    
  
committer: Joe Conway <mail@joeconway.com>    
date     : Thu, 28 May 2020 13:44:59 -0400    

Click here for diff

Two of the members of rconn were left uninitialized. When  
dblink_open() is called without an outer transaction it  
handles the initialization for us, but with an outer  
transaction it does not. Arrange for initialization  
in all cases. Backpatch to all supported versions.  
  
Reported-by: Alexander Lakhin  
Discussion: https://www.postgresql.org/message-id/flat/9bd0744f-5f04-c778-c5b3-809efe9c30c7%40joeconway.com#c545909a41664991aca60c4d70a10ce7  

M contrib/dblink/dblink.c

Add CHECK_FOR_INTERRUPTS() to the repeat() function

commit   : 3ccae5445cc82ce93c660a2b21a244a51dbe30a6    
  
author   : Joe Conway <mail@joeconway.com>    
date     : Thu, 28 May 2020 13:16:56 -0400    
  
committer: Joe Conway <mail@joeconway.com>    
date     : Thu, 28 May 2020 13:16:56 -0400    

Click here for diff

The repeat() function loops for potentially a long time without  
ever checking for interrupts. This prevents, for example, a query  
cancel from interrupting until the work is all done. Fix by  
inserting a CHECK_FOR_INTERRUPTS() into the loop.  
  
Backpatch to all supported versions.  
  
Discussion: https://www.postgresql.org/message-id/flat/8692553c-7fe8-17d9-cbc1-7cddb758f4c6%40joeconway.com  

M src/backend/utils/adt/oracle_compat.c

Add missing error code to "cannot attach index ..." error.

commit   : 7b009944c2525fcf8ee35db135c4bc560eb208f2    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Thu, 28 May 2020 12:37:00 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Thu, 28 May 2020 12:37:00 +0300    

Click here for diff

ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE was used in an ereport with the  
same message but different errdetail a few lines earlier, so use that  
here as well.  
  
Backpatch-through: 11  

M src/backend/commands/tablecmds.c

Fix typo in test comment.

commit   : 169a6588a71724e7932ece91d844b810b0dbed67    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Thu, 28 May 2020 12:35:18 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Thu, 28 May 2020 12:35:18 +0300    

Click here for diff

The same comment was copied to a few different places, with the same typo.  
Backpatch down to v11, where this typo was introduced.  

M src/test/regress/expected/alter_table.out
M src/test/regress/expected/hash_part.out
M src/test/regress/expected/insert.out
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/hash_part.sql
M src/test/regress/sql/insert.sql
M src/test/regress/sql/partition_prune.sql

Add lcov exclusion markers to jsonpath scanner

commit   : 98cab3ebce8376139d672a39ddeed4fff002e3e5    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Tue, 26 May 2020 14:09:36 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Tue, 26 May 2020 14:09:36 +0200    

Click here for diff

This was done for all scanners in  
421167362242ce1fb46d6d720798787e7cd65aad but not added to the new one.  

M src/backend/utils/adt/jsonpath_scan.l

gss: add missing references to hostgssenc and hostnogssenc

commit   : a286d470c9ba31410141f2ae77e3ca11275783c4    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 25 May 2020 20:19:28 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 25 May 2020 20:19:28 -0400    

Click here for diff

These were missed when these were added to pg_hba.conf in PG 12;  
updates docs and pg_hba.conf.sample.  
  
Reported-by: Arthur Nascimento  
  
Bug: 16380  
  
Discussion: https://postgr.es/m/20200421182736.GG19613@momjian.us  
  
Backpatch-through: 12  

M doc/src/sgml/client-auth.sgml
M src/backend/libpq/pg_hba.conf.sample

Add a temp-install prerequisite to top-level "check-tests".

commit   : adc4682efbac9ec2c48843f9ef9bc61f14c3e39f    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Mon, 25 May 2020 16:21:04 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Mon, 25 May 2020 16:21:04 -0700    

Click here for diff

The target failed, tested $PATH binaries, or tested a stale temporary  
installation.  Commit c66b438db62748000700c9b90b585e756dd54141 missed  
this.  Back-patch to 9.5 (all supported versions).  

M GNUmakefile.in

Fix two typos in a comment

commit   : 8fdde45e32e486903d6866a4b0a761f418463493    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 22 May 2020 17:39:16 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 22 May 2020 17:39:16 -0400    

Click here for diff

They were introduced in 898e5e3290a7; backpatch to 12.  

M src/backend/partitioning/partdesc.c

doc: suggest 1.1 as a random_page_cost value for SSDs

commit   : a87209ce7e1270b3a48e107c981a7035196d662b    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Thu, 21 May 2020 20:28:38 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Thu, 21 May 2020 20:28:38 -0400    

Click here for diff

Reported-by: yigong hu  
  
Discussion: https://postgr.es/m/CAOxFffcourucFqSk+tZA13ErS3XRYkDy6EeaPff4AvHGiEEuug@mail.gmail.com  
  
Backpatch-through: 9.5  

M doc/src/sgml/config.sgml

doc: Simplify mention of unique indexes for NULL control

commit   : 388d7f8c62b407005f56b6262a678d9fb96ea18c    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Thu, 21 May 2020 19:49:30 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Thu, 21 May 2020 19:49:30 -0400    

Click here for diff

Discussion: https://postgr.es/m/2304.1586532634@sss.pgh.pa.us  
  
Backpatch-through: 9.5  

M doc/src/sgml/indices.sgml

Fix MSVC installations with multiple "configure" files detected

commit   : 089baec6fd2097984af92baa9e5aaf0cdd1238e6    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 21 May 2020 14:41:30 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 21 May 2020 14:41:30 +0900    

Click here for diff

When installing binaries and libraries using the MSVC installation  
routines, the operation gets done after moving to the root folder, whose  
location is detected by checking if "configure" exists two times in a  
row.  So, calling the installation script from src/tools/msvc/ with an  
extra "configure" file four levels up the root path of the code tree  
causes the execution to go further up, leading to a failure in finding  
the builds.  This commit fixes the issue by moving to the root folder of  
the code tree only once, when necessary.  
  
Author: Arnold Müller  
Reviewed-by: Daniel Gustafsson  
Discussion: https://postgr.es/m/16343-f638f67e7e52b86c@postgresql.org  
Backpatch-through: 9.5  

M src/tools/msvc/Install.pm

doc: Adding a partition does not require Access Exclusive lock

commit   : e2a19a6d253622318bacaea8faccdbf7ca771ddf    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 20 May 2020 14:35:39 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 20 May 2020 14:35:39 -0400    

Click here for diff

This doc update was missed in 898e5e3290a7.  Backpatch to 12.  
  
Pointed out by Pavel Luzanov  
Discussion: https://postgr.es/m/642e9fbc-b832-698b-9a8f-d626afd7014d@postgrespro.ru  

M doc/src/sgml/ddl.sgml

Doc: Fix description of pg_class.relreplident

commit   : fd7636909101225d65dcb44448dd35dea0731eba    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 20 May 2020 14:21:45 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 20 May 2020 14:21:45 +0900    

Click here for diff

The description missed a comma and lacked an explanation of what happens  
with REPLICA IDENTITY USING INDEX when the dependent index is dropped.  
  
Author: Marina Polyakova  
Reviewed-by: Daniel Gustafsson, Michael Paquier  
Discussion: https://postgr.es/m/ad1a0badc32658b1bbb07aa312346a1d@postgrespro.ru  
Backpatch-through: 9.5  

M doc/src/sgml/catalogs.sgml

Fix comment in slot.c.

commit   : bf3827f29cb07ababd0fc43fa5ea7ac8a9388f5a    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Mon, 18 May 2020 08:15:05 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Mon, 18 May 2020 08:15:05 +0530    

Click here for diff

Reported-by: Sawada Masahiko  
Author: Sawada Masahiko  
Reviewed-by: Amit Kapila  
Backpatch-through: 9.5  
Discussion: https://postgr.es/m/CA+fd4k4Ws7M7YQ8PqSym5WB1y75dZeBTd1sZJUQdfe0KJQ-iSA@mail.gmail.com  

M src/backend/replication/slot.c

Fix assertion with relation using REPLICA IDENTITY FULL in subscriber

commit   : b4ded2f227f3ec1144a4691d2326eb90174447db    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sat, 16 May 2020 18:16:31 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sat, 16 May 2020 18:16:31 +0900    

Click here for diff

In a logical replication subscriber, a table using REPLICA IDENTITY FULL  
which has a primary key would try to use the primary key's index  
available to scan for a tuple, but an assertion only assumed as correct  
the case of an index associated to REPLICA IDENTITY USING INDEX.  This  
commit corrects the assertion so as the use of a primary key index is a  
valid case.  
  
Reported-by: Dilip Kumar  
Analyzed-by: Dilip Kumar  
Author: Euler Taveira  
Reviewed-by: Michael Paquier, Masahiko Sawada  
Discussion: https://postgr.es/m/CAFiTN-u64S5bUiPL1q5kwpHNd0hRnf1OE-bzxNiOs5zo84i51w@mail.gmail.com  
Backpatch-through: 10  

M src/backend/executor/execReplication.c
M src/test/subscription/t/001_rep_changes.pl

Fix bogus initialization of replication origin shared memory state.

commit   : c8b1c953b8be8ede059b104198c5603a251eb19f    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 15 May 2020 19:05:39 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 15 May 2020 19:05:39 -0400    

Click here for diff

The previous coding zeroed out offsetof(ReplicationStateCtl, states)  
more bytes than it was entitled to, as a consequence of starting the  
zeroing from the wrong pointer (or, if you prefer, using the wrong  
calculation of how much to zero).  
  
It's unsurprising that this has not caused any reported problems,  
since it can be expected that the newly-allocated block is at the end  
of what we've used in shared memory, and we always make the shmem  
block substantially bigger than minimally necessary.  Nonetheless,  
this is wrong and it could bite us someday; plus it's a dangerous  
model for somebody to copy.  
  
This dates back to the introduction of this code (commit 5aa235042),  
so back-patch to all supported branches.  

M src/backend/replication/logical/origin.c

Avoid killing btree items that are already dead

commit   : 1d84751c60bf615379f77f0f9d2fa5c5200d4681    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 15 May 2020 16:50:34 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 15 May 2020 16:50:34 -0400    

Click here for diff

_bt_killitems marks btree items dead when a scan leaves the page where  
they live, but it does so with only share lock (to improve concurrency).  
This was historicall okay, since killing a dead item has no  
consequences.  However, with the advent of data checksums and  
wal_log_hints, this action incurs a WAL full-page-image record of the  
page.  Multiple concurrent processes would write the same page several  
times, leading to WAL bloat.  The probability of this happening can be  
reduced by only killing items if they're not already dead, so change the  
code to do that.  
  
The problem could eliminated completely by having _bt_killitems upgrade  
to exclusive lock upon seeing a killable item, but that would reduce  
concurrency so it's considered a cure worse than the disease.  
  
Backpatch all the way back to 9.5, since wal_log_hints was introduced in  
9.4.  
  
Author: Masahiko Sawada <masahiko.sawada@2ndquadrant.com>  
Discussion: https://postgr.es/m/CA+fd4k6PeRj2CkzapWNrERkja5G0-6D-YQiKfbukJV+qZGFZ_Q@mail.gmail.com  

M src/backend/access/nbtree/nbtutils.c

docs: add xreflabel entries for autovacuum, SP-GiST, and TOAST

commit   : 5663844ed73f8ef70c7bb64e30275a9065dbed9e    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 12:38:40 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 12:38:40 -0400    

Click here for diff

This is for use by the PG 13 release notes, but might be used for minor  
release notes in the future.  
  
Backpatch-through: 9.5  

M doc/src/sgml/maintenance.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/storage.sgml

doc: add missing xreflabels to the main docs (not refs)

commit   : 6b1e3919c7bd1b7a02ab23271516583d4e21b578    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 12:05:43 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 12:05:43 -0400    

Click here for diff

Add missing xreflabels for index types, geqo, libpq, spi, server-side  
languages, ecpg, and vaacuumlo.  
  
Backpatch-through: 9.5  

M doc/src/sgml/ecpg.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/spi.sgml
M doc/src/sgml/vacuumlo.sgml

doc: remove extra blank line at the top of SGML files

commit   : 8aeaae4a1d372dc9760787e66ae0697303eea1a5    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 09:55:43 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 09:55:43 -0400    

Click here for diff

Backpatch-through: 9.5  

M doc/src/sgml/ref/copy.sgml

doc: make ref/*.sgml file header comment layout consistent

commit   : 4032178e77d46a8853f1aff50fe3b4b8070da931    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 08:52:24 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 15 May 2020 08:52:24 -0400    

Click here for diff

M doc/src/sgml/ref/checkpoint.sgml
M doc/src/sgml/ref/create_cast.sgml
M doc/src/sgml/ref/create_collation.sgml
M doc/src/sgml/ref/create_conversion.sgml
M doc/src/sgml/ref/create_foreign_table.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/ref/create_transform.sgml
M doc/src/sgml/ref/drop_cast.sgml
M doc/src/sgml/ref/drop_collation.sgml
M doc/src/sgml/ref/drop_conversion.sgml
M doc/src/sgml/ref/drop_foreign_table.sgml
M doc/src/sgml/ref/drop_transform.sgml
M doc/src/sgml/ref/load.sgml
M doc/src/sgml/ref/pg_config-ref.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pgarchivecleanup.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/pgtestfsync.sgml
M doc/src/sgml/ref/pgtesttiming.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/set_constraints.sgml
M doc/src/sgml/ref/set_session_auth.sgml
M doc/src/sgml/ref/set_transaction.sgml

Fix amcheck for page checks concurrent to replay of btree page deletion

commit   : ae1f9b0a9bc3d2d1e46fcd38c8b2e7ba49238b05    
  
author   : Alexander Korotkov <akorotkov@postgresql.org>    
date     : Wed, 6 May 2020 15:35:27 +0300    
  
committer: Alexander Korotkov <akorotkov@postgresql.org>    
date     : Wed, 6 May 2020 15:35:27 +0300    

Click here for diff

amcheck expects at least hikey to always exist on leaf page even if it is  
deleted page.  But replica reinitializes page during replay of page deletion,  
causing deleted page to have no items.  Thus, replay of page deletion can  
cause an error in concurrent amcheck run.  
  
This commit relaxes amcheck expectation making it tolerate deleted page with  
no items.  
  
Reported-by: Konstantin Knizhnik  
Discussion: https://postgr.es/m/CAPpHfdt_OTyQpXaPJcWzV2N-LNeNJseNB-K_A66qG%3DL518VTFw%40mail.gmail.com  
Author: Alexander Korotkov  
Reviewed-by: Peter Geoghegan  
Backpatch-through: 11  

M contrib/amcheck/verify_nbtree.c

Move check for fsync=off so that pendingOps still gets cleared.

commit   : 891a2007e3987545a2543c1f04cd0682576b39fa    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Thu, 14 May 2020 08:39:26 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Thu, 14 May 2020 08:39:26 +0300    

Click here for diff

Commit 3eb77eba5a moved the loop and refactored it, and inadvertently  
changed the effect of fsync=off so that it also skipped removing entries  
from the pendingOps table. That was not intentional, and leads to an  
assertion failure if you turn fsync on while the server is running and  
reload the config.  
  
Backpatch-through: 12-  
Reviewed-By: Thomas Munro  
Discussion: https://www.postgresql.org/message-id/3cbc7f4b-a5fa-56e9-9591-c886deb07513%40iki.fi  

M src/backend/storage/sync/sync.c

Fix the MSVC build for versions 2015 and later.

commit   : 98171e59a6cae02b5efdbf6de12bc630d58373c6    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Thu, 14 May 2020 09:34:46 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Thu, 14 May 2020 09:34:46 +0530    

Click here for diff

Visual Studio 2015 and later versions should still be able to do the same  
as Visual Studio 2012, but the declaration of locale_name is missing in  
_locale_t, causing the code compilation to fail, hence this falls back  
instead on to enumerating all system locales by using EnumSystemLocalesEx  
to find the required locale name.  If the input argument is in Unix-style  
then we can get ISO Locale name directly by using GetLocaleInfoEx() with  
LCType as LOCALE_SNAME.  
  
In passing, change the documentation references of the now obsolete links.  
  
Note that this problem occurs only with NLS enabled builds.  
  
Author: Juan José Santamaría Flecha, Davinder Singh and Amit Kapila  
Reviewed-by: Ranier Vilela and Amit Kapila  
Backpatch-through: 9.5  
Discussion: https://postgr.es/m/CAHzhFSFoJEWezR96um4-rg5W6m2Rj9Ud2CNZvV4NWc9tXV7aXQ@mail.gmail.com  

M src/backend/utils/adt/pg_locale.c

Fix pg_recvlogical avoidance of superfluous Standby Status Update.

commit   : 73a5c0d81ea611ad59051fe52a4b9e72b9e90289    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Wed, 13 May 2020 20:42:09 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Wed, 13 May 2020 20:42:09 -0700    

Click here for diff

The defect suppressed a Standby Status Update message when bytes flushed  
to disk had changed but bytes received had not changed.  If  
pg_recvlogical then exited with no intervening Standby Status Update,  
the next pg_recvlogical repeated already-flushed records.  The defect  
could also cause superfluous messages, which are functionally harmless.  
Back-patch to 9.5 (all supported versions).  
  
Discussion: https://postgr.es/m/20200502221647.GA3941274@rfd.leadboat.com  

M src/bin/pg_basebackup/pg_recvlogical.c

In successful pg_recvlogical, end PGRES_COPY_OUT cleanly.

commit   : 7130be8aa3e022b08245f77cbbe5a71473b36a73    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Wed, 13 May 2020 20:42:09 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Wed, 13 May 2020 20:42:09 -0700    

Click here for diff

pg_recvlogical merely called PQfinish(), so the backend sent messages  
after the disconnect.  When that caused EPIPE in internal_flush(),  
before a LogicalConfirmReceivedLocation(), the next pg_recvlogical would  
repeat already-acknowledged records.  Whether or not the defect causes  
EPIPE, post-disconnect messages could contain an ErrorResponse that the  
user should see.  One properly ends PGRES_COPY_OUT by repeating  
PQgetCopyData() until it returns a negative value.  Augment one of the  
tests to cover the case of WAL past --endpos.  Back-patch to v10, where  
commit 7c030783a5bd07cadffc2a1018bc33119a4c7505 first appeared.  Before  
that commit, pg_recvlogical never reached PGRES_COPY_OUT.  
  
Reported by Thomas Munro.  
  
Discussion: https://postgr.es/m/CAEepm=1MzM2Z_xNe4foGwZ1a+MO_2S9oYDq3M5D11=JDU_+0Nw@mail.gmail.com  

M src/bin/pg_basebackup/pg_recvlogical.c
M src/test/recovery/t/006_logical_decoding.pl