PostgreSQL 10.18 (upcoming) commit log

Further refinement of stuck_on_old_timeline recovery test

commit   : 1dc53c914fe8bace75ed6c227f6c2e6833e3e7b4    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Tue, 15 Jun 2021 15:30:11 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Tue, 15 Jun 2021 15:30:11 -0400    

Click here for diff

TestLib::perl2host can take a file argument as well as a directory  
argument, so that code becomes substantially simpler. Also add comments  
on why we're using forward slashes, and why we're setting  
PERL_BADLANG=0.  
  
Discussion: https://postgr.es/m/e9947bcd-20ee-027c-f0fe-01f736b7e345@dunslane.net  

M src/test/recovery/t/025_stuck_on_old_timeline.pl

Fix decoding of speculative aborts.

commit   : 35f56c124a7afb3d5059390dce15a47701c76d3b    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Tue, 15 Jun 2021 09:10:24 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Tue, 15 Jun 2021 09:10:24 +0530    

Click here for diff

During decoding for speculative inserts, we were relying for cleaning  
toast hash on confirmation records or next change records. But that  
could lead to multiple problems (a) memory leak if there is neither a  
confirmation record nor any other record after toast insertion for a  
speculative insert in the transaction, (b) error and assertion failures  
if the next operation is not an insert/update on the same table.  
  
The fix is to start queuing spec abort change and clean up toast hash  
and change record during its processing. Currently, we are queuing the  
spec aborts for both toast and main table even though we perform cleanup  
while processing the main table's spec abort record. Later, if we have a  
way to distinguish between the spec abort record of toast and the main  
table, we can avoid queuing the change for spec aborts of toast tables.  
  
Reported-by: Ashutosh Bapat  
Author: Dilip Kumar  
Reviewed-by: Amit Kapila  
Backpatch-through: 9.6, where it was introduced  
Discussion: https://postgr.es/m/CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com  

M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h

Work around portability issue with newer versions of mktime().

commit   : 05fccab6af12370aa2ccf32dce32babf57492587    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 13 Jun 2021 14:32:42 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 13 Jun 2021 14:32:42 -0400    

Click here for diff

Recent glibc versions have made mktime() fail if tm_isdst is  
inconsistent with the prevailing timezone; in particular it fails for  
tm_isdst = 1 when the zone is UTC.  (This seems wildly inconsistent  
with the POSIX-mandated treatment of "incorrect" values for the other  
fields of struct tm, so if you ask me it's a bug, but I bet they'll  
say it's intentional.)  This has been observed to cause cosmetic  
problems when pg_restore'ing an archive created in a different  
timezone.  
  
To fix, do mktime() using the field values from the archive, and if  
that fails try again with tm_isdst = -1.  This will give a result  
that's off by the UTC-offset difference from the original zone, but  
that was true before, too.  It's not terribly critical since we don't  
do anything with the result except possibly print it.  (Someday we  
should flush this entire bit of logic and record a standard-format  
timestamp in the archive instead.  That's not okay for a back-patched  
bug fix, though.)  
  
Also, guard our only other use of mktime() by having initdb's  
build_time_t() set tm_isdst = -1 not 0.  This case could only have  
an issue in zones that are DST year-round; but I think some do exist,  
or could in future.  
  
Per report from Wells Oliver.  Back-patch to all supported  
versions, since any of them might need to run with a newer glibc.  
  
Discussion: https://postgr.es/m/CAOC+FBWDhDHO7G-i1_n_hjRzCnUeFO+H-Czi1y10mFhRWpBrew@mail.gmail.com  

M src/bin/initdb/findtimezone.c
M src/bin/pg_dump/pg_backup_archiver.c

Further tweaks to stuck_on_old_timeline recovery test

commit   : 319d616dd0fb06f45545d22820d3acb4154a2e70    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Sun, 13 Jun 2021 07:10:41 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Sun, 13 Jun 2021 07:10:41 -0400    

Click here for diff

Translate path slashes on target directory path. This was confusing old  
branches, but is applied to all branches for the sake of uniformity.  
Perl is perfectly able to understand paths with forward slashes.  
  
Along the way, restore the previous archive_wait query, for the sake of  
uniformity with other tests, per gripe from Tom Lane.  

M src/test/recovery/t/025_stuck_on_old_timeline.pl

Ignore more environment variables in pg_regress.c

commit   : d31c316c74ee3cf2942cb91c2fe91fb6749730c7    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sun, 13 Jun 2021 20:08:00 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sun, 13 Jun 2021 20:08:00 +0900    

Click here for diff

This is similar to the work done in 8279f68 for TestLib.pm, where  
environment variables set may cause unwanted failures if using a  
temporary installation with pg_regress.  The list of variables reset is  
adjusted in each stable branch depending on what is supported.  
  
Comments are added to remember that the lists in TestLib.pm and  
pg_regress.c had better be kept in sync.  
  
Reviewed-by: Álvaro Herrera  
Discussion: https://postgr.es/m/YMNR9GYDn+fHlMta@paquier.xyz  
Backpatch-through: 9.6  

M src/test/perl/TestLib.pm
M src/test/regress/pg_regress.c

Restore robustness of TAP tests that wait for postmaster restart.

commit   : 1b48dd50abd6220f199ba31e54b3c42c7cd729cd    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 12 Jun 2021 15:12:10 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 12 Jun 2021 15:12:10 -0400    

Click here for diff

Several TAP tests use poll_query_until() to wait for the postmaster  
to restart.  They were checking to see if a trivial query  
(e.g. "SELECT 1") succeeds.  However, that's problematic in the wake  
of commit 11e9caff8, because now that we feed said query to psql  
via stdin, we risk IPC::Run whining about a SIGPIPE failure if psql  
quits before reading the query.  Hence, we can't use a nonempty  
query in cases where we need to wait for connection failures to  
stop happening.  
  
Per the precedent of commits c757a3da0 and 6d41dd045, we can pass  
"undef" as the query in such cases to ensure that IPC::Run has  
nothing to write.  However, then we have to say that the expected  
output is empty, and this exposes a deficiency in poll_query_until:  
if psql fails altogether and returns empty stdout, poll_query_until  
will treat that as a success!  That's because, contrary to its  
documentation, it makes no actual check for psql failure, looking  
neither at the exit status nor at stderr.  
  
To fix that, adjust poll_query_until to insist on empty stderr as  
well as a stdout match.  (I experimented with checking exit status  
instead, but it seems that psql often does exit(1) in cases that we  
need to consider successes.  That might be something to fix someday,  
but it would be a non-back-patchable behavior change.)  
  
Back-patch to v10.  The test cases needing this exist only as far  
back as v11, but it seems wise to keep poll_query_until's behavior  
the same in v10, in case we back-patch another such test case in  
future.  (9.6 does not currently need this change, because in that  
branch poll_query_until can't be told to accept empty stdout as  
a success case.)  
  
Per assorted buildfarm failures, mostly on hoverfly.  
  
Discussion: https://postgr.es/m/CAA4eK1+zM6L4QSA1XMvXY_qqWwdUmqkOS1+hWvL8QcYEBGA1Uw@mail.gmail.com  

M src/test/perl/PostgresNode.pm

Ensure pg_filenode_relation(0, 0) returns NULL.

commit   : 26a0535334ce31b12baf146ad0860c10c4947913    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 12 Jun 2021 13:29:24 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 12 Jun 2021 13:29:24 -0400    

Click here for diff

Previously, a zero value for the relfilenode resulted in  
a confusing error message about "unexpected duplicate".  
This function returns NULL for other invalid relfilenode  
values, so zero should be treated likewise.  
  
It's been like this all along, so back-patch to all supported  
branches.  
  
Justin Pryzby  
  
Discussion: https://postgr.es/m/20210612023324.GT16435@telsasoft.com  

M src/backend/utils/adt/dbsize.c

Don't use Asserts to check for violations of replication protocol.

commit   : 4745c119e01a09f927cc67cecab68c24e21b622c    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 12 Jun 2021 12:59:15 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 12 Jun 2021 12:59:15 -0400    

Click here for diff

Using an Assert to check the validity of incoming messages is an  
extremely poor decision.  In a debug build, it should not be that easy  
for a broken or malicious remote client to crash the logrep worker.  
The consequences could be even worse in non-debug builds, which will  
fail to make such checks at all, leading to who-knows-what misbehavior.  
Hence, promote every Assert that could possibly be triggered by wrong  
or out-of-order replication messages to a full test-and-ereport.  
  
To avoid bloating the set of messages the translation team has to cope  
with, establish a policy that replication protocol violation error  
reports don't need to be translated.  Hence, all the new messages here  
use errmsg_internal().  A couple of old messages are changed likewise  
for consistency.  
  
Along the way, fix some non-idiomatic or outright wrong uses of  
hash_search().  
  
Most of these mistakes are new with the "streaming replication"  
patch (commit 464824323), but a couple go back a long way.  
Back-patch as appropriate.  
  
Discussion: https://postgr.es/m/1719083.1623351052@sss.pgh.pa.us  

M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/worker.c

Fix new recovery test for use under msys

commit   : 0cd8a55bda172c704d6a41db51bb255db78a3f9e    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Sat, 12 Jun 2021 08:37:16 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Sat, 12 Jun 2021 08:37:16 -0400    

Click here for diff

Commit caba8f0d43 wasn't quite right for msys, as demonstrated by  
several buildfarm animals, including jacana and fairywren. We need to  
use the msys perl in the archive command, but call it in such a way that  
Windows will understand the path. Furthermore, inside the copy script we  
need to convert a Windows path to an msys path.  

M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/recovery/t/cp_history_files

Remove PGSSLCRLDIR from the list of variables ignored in TAP tests

commit   : 37d94e6b14d41bdfbd1a35ecd391297c938a2abf    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Sat, 12 Jun 2021 10:39:38 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Sat, 12 Jun 2021 10:39:38 +0900    

Click here for diff

This variable was present in the list added by 9d660670, but it is not  
supported by this branch.  Issue noticed while diving into a similar  
change for pg_regress.c.  
  
Backpatch-through: 9.6  

M src/test/perl/TestLib.pm

Adjust new test case to set wal_keep_segments.

commit   : 1c2c6a9a299c4b26c46b48a178a431c0ba21cebd    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Thu, 10 Jun 2021 09:43:35 -0400    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Thu, 10 Jun 2021 09:43:35 -0400    

Click here for diff

Per buildfarm member conchuela and Kyotaro Horiguchi, it's possible  
for the WAL segment that the cascading standby needs to be removed  
too quickly. Hopefully this will prevent that.  
  
Kyotaro Horiguchi  
  
Discussion: http://postgr.es/m/20210610.101240.1270925505780628275.horikyota.ntt@gmail.com  

M src/test/recovery/t/025_stuck_on_old_timeline.pl

Fix corner case failure of new standby to follow new primary.

commit   : 31bda6a22ad11069b1e5f3a206262ee800303e51    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Wed, 9 Jun 2021 16:20:28 -0400    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Wed, 9 Jun 2021 16:20:28 -0400    

Click here for diff

This only happens if (1) the new standby has no WAL available locally,  
(2) the new standby is starting from the old timeline, (3) the promotion  
happened in the WAL segment from which the new standby is starting,  
(4) the timeline history file for the new timeline is available from  
the archive but the WAL files for are not (i.e. this is a race),  
(5) the WAL files for the new timeline are available via streaming,  
and (6) recovery_target_timeline='latest'.  
  
Commit ee994272ca50f70b53074f0febaec97e28f83c4e introduced this  
logic and was an improvement over the previous code, but it mishandled  
this case. If recovery_target_timeline='latest' and restore_command is  
set, validateRecoveryParameters() can change recoveryTargetTLI to be  
different from receiveTLI. If streaming is then tried afterward,  
expectedTLEs gets initialized with the history of the wrong timeline.  
It's supposed to be a list of entries explaining how to get to the  
target timeline, but in this case it ends up with a list of entries  
explaining how to get to the new standby's original timeline, which  
isn't right.  
  
Dilip Kumar and Robert Haas, reviewed by Kyotaro Horiguchi.  
  
Discussion: http://postgr.es/m/CAFiTN-sE-jr=LB8jQuxeqikd-Ux+jHiXyh4YDiZMPedgQKup0g@mail.gmail.com  

M src/backend/access/transam/xlog.c
A src/test/recovery/t/025_stuck_on_old_timeline.pl
A src/test/recovery/t/cp_history_files

Allow PostgresNode.pm's backup method to accept backup_options.

commit   : 9778a191a4c0d814f34160a6a2e595cc842d5abe    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Wed, 9 Jun 2021 12:30:49 -0400    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Wed, 9 Jun 2021 12:30:49 -0400    

Click here for diff

Partial back-port of commit 081876d75ea15c3bd2ee5ba64a794fd8ea46d794.  
A test case for a pending bug fix needs this capability, but the code  
on 9.6 is significantly different, so I'm only back-patching this  
change as far as v10. We'll have to work around the problem another  
way in v9.6.  
  
Discussion: http://postgr.es/m/CAFiTN-tcivNvL0Rg6rD7_CErNfE75H7+gh9WbMxjbgsattja1Q@mail.gmail.com  

M src/test/perl/PostgresNode.pm

Fix inconsistencies in psql --help=commands

commit   : 16a7ed765d6fb1f1026f6ea77629e034d9c1115e    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 9 Jun 2021 16:26:06 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 9 Jun 2021 16:26:06 +0900    

Click here for diff

The set of subcommands supported by \dAp, \do and \dy was described  
incorrectly in psql's --help.  The documentation was already consistent  
with the code.  
  
Reported-by: inoas, from IRC  
Author: Matthijs van der Vleuten  
Reviewed-by: Neil Chen  
Discussion: https://postgr.es/m/6a984e24-2171-4039-9050-92d55e7b23fe@www.fastmail.com  
Backpatch-through: 9.6  

M src/bin/psql/help.c

Fix incautious handling of possibly-miscoded strings in client code.

commit   : 4826fe4c5674b93f3725ce84a091cee521907aae    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 7 Jun 2021 14:15:25 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 7 Jun 2021 14:15:25 -0400    

Click here for diff

An incorrectly-encoded multibyte character near the end of a string  
could cause various processing loops to run past the string's  
terminating NUL, with results ranging from no detectable issue to  
a program crash, depending on what happens to be in the following  
memory.  
  
This isn't an issue in the server, because we take care to verify  
the encoding of strings before doing any interesting processing  
on them.  However, that lack of care leaked into client-side code  
which shouldn't assume that anyone has validated the encoding of  
its input.  
  
Although this is certainly a bug worth fixing, the PG security team  
elected not to regard it as a security issue, primarily because  
any untrusted text should be sanitized by PQescapeLiteral or  
the like before being incorporated into a SQL or psql command.  
(If an app fails to do so, the same technique can be used to  
cause SQL injection, with probably much more dire consequences  
than a mere client-program crash.)  Those functions were already  
made proof against this class of problem, cf CVE-2006-2313.  
  
To fix, invent PQmblenBounded() which is like PQmblen() except it  
won't return more than the number of bytes remaining in the string.  
In HEAD we can make this a new libpq function, as PQmblen() is.  
It seems imprudent to change libpq's API in stable branches though,  
so in the back branches define PQmblenBounded as a macro in the files  
that need it.  (Note that just changing PQmblen's behavior would not  
be a good idea; notably, it would completely break the escaping  
functions' defense against this exact problem.  So we just want a  
version for those callers that don't have any better way of handling  
this issue.)  
  
Per private report from houjingyi.  Back-patch to all supported branches.  

M src/bin/psql/common.c
M src/bin/psql/psqlscanslash.l
M src/bin/psql/stringutils.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/common.c
M src/fe_utils/print.c
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol3.c

Support use of strnlen() in pre-v11 branches.

commit   : 0b116346b08f9b9fc93e3dc9fbf9f839909d6825    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 7 Jun 2021 13:12:35 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 7 Jun 2021 13:12:35 -0400    

Click here for diff

Back-patch a minimal subset of commits fffd651e8 and 46912d9b1,  
to support strnlen() on all platforms without adding any callers.  
This will be needed by a following bug fix.  

M configure
M configure.in
M src/include/pg_config.h.in
M src/include/pg_config.h.win32
M src/include/port.h
M src/interfaces/libpq/.gitignore
M src/interfaces/libpq/Makefile
A src/port/strnlen.c

Fix compiler warning

commit   : 32db0d68be40824b94beb6014f6979e3626b9056    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 7 Jun 2021 11:04:23 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 7 Jun 2021 11:04:23 -0400    

Click here for diff

Introduced by 41306a511c01dd299115cf447858a00e34aebbf6, happens with gcc  
4.7.2.  
  
Forward-port of 1ec36a9eb4c2, which was applied to 9.6 only.  
  
Author: Peter Eisentraut <peter@eisentraut.org>  
Reported-by: Anton Voloshin <a.voloshin@postgrespro.ru>  
Discussion: https://postgr.es/m/be8bbcdf-35f8-a8a6-098f-65c2e9497151@postgrespro.ru  

M src/backend/catalog/objectaddress.c

In PostgresNode.pm, don't pass SQL to psql on the command line

commit   : a17ac975e28e2a83caceb8afddc45cae67d724eb    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Thu, 3 Jun 2021 16:08:33 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Thu, 3 Jun 2021 16:08:33 -0400    

Click here for diff

The Msys shell mangles certain patterns in its command line, so avoid  
handing arbitrary SQL to psql on the command line and instead use  
IPC::Run's redirection facility for stdin. This pattern is already  
mostly whats used, but query_poll_until() was not doing the right thing.  
  
Problem discovered on the buildfarm when a new TAP test failed on msys.  

M src/test/perl/PostgresNode.pm

Reduce risks of conflicts in internal queries of REFRESH MATVIEW CONCURRENTLY

commit   : 20f70f5581726bdb29cf83ce0b5a7e65d257617f    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 3 Jun 2021 15:28:53 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 3 Jun 2021 15:28:53 +0900    

Click here for diff

The internal SQL queries used by REFRESH MATERIALIZED VIEW CONCURRENTLY  
include some aliases for its diff and temporary relations with  
rather-generic names: diff, newdata, newdata2 and mv.  Depending on the  
queries used for the materialized view, using CONCURRENTLY could lead to  
some internal failures if the query and those internal aliases conflict.  
  
Those names have been chosen in 841c29c8.  This commit switches instead  
to a naming pattern which is less likely going to cause conflicts, based  
on an idea from Thomas Munro, by appending _$ to those aliases.  This is  
not perfect as those new names could still conflict, but at least it has  
the advantage to keep the code readable and simple while reducing the  
likelihood of conflicts to be close to zero.  
  
Reported-by: Mathis Rudolf  
Author: Bharath Rupireddy  
Reviewed-by: Bernd Helmle, Thomas Munro, Michael Paquier  
Discussion: https://postgr.es/m/109c267a-10d2-3c53-b60e-720fcf44d9e8@credativ.de  
Backpatch-through: 9.6  

M src/backend/commands/matview.c

Ignore more environment variables in TAP tests

commit   : c63bb6950735103fa6528081014955d065c8ab64    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 3 Jun 2021 11:51:56 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 3 Jun 2021 11:51:56 +0900    

Click here for diff

Various environment variables were not getting reset in the TAP tests,  
which would cause failures depending on the tests or the environment  
variables involved.  For example, PGSSL{MAX,MIN}PROTOCOLVERSION could  
cause failures in the SSL tests.  Even worse, a junk value of  
PGCLIENTENCODING makes a server startup fail.  The list of variables  
reset is adjusted in each stable branch depending on what is supported.  
  
While on it, simplify a bit the code per a suggestion from Andrew  
Dunstan, using a list of variables instead of doing single deletions.  
  
Reviewed-by: Andrew Dunstan, Daniel Gustafsson  
Discussion: https://postgr.es/m/YLbjjRpucIeZ78VQ@paquier.xyz  
Backpatch-through: 9.6  

M src/test/perl/TestLib.pm

Reject SELECT ... GROUP BY GROUPING SETS (()) FOR UPDATE.

commit   : 39862dde4beecdf1c5f84a20fe3bb6f4ba928edb    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 1 Jun 2021 11:12:56 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 1 Jun 2021 11:12:56 -0400    

Click here for diff

This case should be disallowed, just as FOR UPDATE with a plain  
GROUP BY is disallowed; FOR UPDATE only makes sense when each row  
of the query result can be identified with a single table row.  
However, we missed teaching CheckSelectLocking() to check  
groupingSets as well as groupClause, so that it would allow  
degenerate grouping sets.  That resulted in a bad plan and  
a null-pointer dereference in the executor.  
  
Looking around for other instances of the same bug, the only one  
I found was in examine_simple_variable().  That'd just lead to  
silly estimates, but it should be fixed too.  
  
Per private report from Yaoguang Chen.  
Back-patch to all supported branches.  

M src/backend/parser/analyze.c
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/errors.out
M src/test/regress/sql/errors.sql

Raise a timeout to 180s, in test 010_logical_decoding_timelines.pl.

commit   : f9e35ff236c9d12c4c30a17f2c09cad56dbf65bb    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Mon, 31 May 2021 00:29:58 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Mon, 31 May 2021 00:29:58 -0700    

Click here for diff

Per buildfarm member hornet.  Also, update Pod documentation showing the  
lower value.  Back-patch to v10, where the test first appeared.  

M src/test/perl/PostgresNode.pm
M src/test/recovery/t/010_logical_decoding_timelines.pl

fix syntax error

commit   : d5d89629cac9a953aaf1e6f99dd15cb468e747fe    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Fri, 28 May 2021 09:35:11 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Fri, 28 May 2021 09:35:11 -0400    

Click here for diff

M src/tools/msvc/Solution.pm

Report configured port in MSVC built pg_config

commit   : a8c1062a11846327f2d04992558da37a64342c8a    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Fri, 28 May 2021 09:26:30 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Fri, 28 May 2021 09:26:30 -0400    

Click here for diff

This is a long standing omission, discovered when trying to write code  
that relied on it.  
  
Backpatch to all live branches.  

M src/tools/msvc/Solution.pm

Fix MSVC scripts when building with GSSAPI/Kerberos

commit   : 8f507dbc64040691dcb6b76a33b2e7d4d37da97d    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 27 May 2021 20:11:33 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 27 May 2021 20:11:33 +0900    

Click here for diff

The deliverables of upstream Kerberos on Windows are installed with  
paths that do not match our MSVC scripts.  First, the include folder was  
named "inc/" in our scripts, but the upstream MSIs use "include/".  
Second, the build would fail with 64-bit environments as the libraries  
are named differently.  
  
This commit adjusts the MSVC scripts to be compatible with the latest  
installations of upstream, and I have checked that the compilation was  
able to work with the 32-bit and 64-bit installations.  
  
Special thanks to Kondo Yuta for the help in investigating the situation  
in hamerkop, which had an incorrect configuration for the GSS  
compilation.  
  
Reported-by: Brian Ye  
Discussion: https://postgr.es/m/162128202219.27274.12616756784952017465@wrigleys.postgresql.org  
Backpatch-through: 9.6  

M src/tools/msvc/Solution.pm

doc: Fix description of some GUCs in docs and postgresql.conf.sample

commit   : 9395672a52d64067c7c88747b5ced388c88464f4    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 27 May 2021 14:58:19 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 27 May 2021 14:58:19 +0900    

Click here for diff

The following parameters have been imprecise, or incorrect, about their  
description (PGC_POSTMASTER or PGC_SIGHUP):  
- autovacuum_work_mem (docs, as of 9.6~)  
- huge_page_size (docs, as of 14~)  
- max_logical_replication_workers (docs, as of 10~)  
- max_sync_workers_per_subscription (docs, as of 10~)  
- min_dynamic_shared_memory (docs, as of 14~)  
- recovery_init_sync_method (postgresql.conf.sample, as of 14~)  
- remove_temp_files_after_crash (docs, as of 14~)  
- restart_after_crash (docs, as of 9.6~)  
- ssl_min_protocol_version (docs, as of 12~)  
- ssl_max_protocol_version (docs, as of 12~)  
  
This commit adjusts the description of all these parameters to be more  
consistent with the practice used for the others.  
  
Revewed-by: Justin Pryzby  
Discussion: https://postgr.es/m/YK2ltuLpe+FbRXzA@paquier.xyz  
Backpatch-through: 9.6  

M doc/src/sgml/config.sgml

Disallow SSL renegotiation

commit   : c64b5d10d69556635d0ee8096b5ba0bdc49e0e35    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Tue, 25 May 2021 10:11:26 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Tue, 25 May 2021 10:11:26 +0900    

Click here for diff

SSL renegotiation is already disabled as of 48d23c72, however this does  
not prevent the server to comply with a client willing to use  
renegotiation.  In the last couple of years, renegotiation had its set  
of security issues and flaws (like the recent CVE-2021-3449), and it  
could be possible to crash the backend with a client attempting  
renegotiation.  
  
This commit takes one extra step by disabling renegotiation in the  
backend in the same way as SSL compression (f9264d15) or tickets  
(97d3a0b0).  OpenSSL 1.1.0h has added an option named  
SSL_OP_NO_RENEGOTIATION able to achieve that.  In older versions  
there is an option called SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS that  
was undocumented, and could be set within the SSL object created when  
the TLS connection opens, but I have decided not to use it, as it feels  
trickier to rely on, and it is not official.  Note that this option is  
not usable in OpenSSL < 1.1.0h as the internal contents of the *SSL  
object are hidden to applications.  
  
SSL renegotiation concerns protocols up to TLSv1.2.  
  
Per original report from Robert Haas, with a patch based on a suggestion  
by Andres Freund.  
  
Author: Michael Paquier  
Reviewed-by: Daniel Gustafsson  
Discussion: https://postgr.es/m/YKZBXx7RhU74FlTE@paquier.xyz  
Backpatch-through: 9.6  

M src/backend/libpq/be-secure-openssl.c

Clean up cpluspluscheck violation.

commit   : 891358f3c71dc359c05ff7b13726ab9e4caf0e5a    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 20 May 2021 13:03:09 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 20 May 2021 13:03:09 -0400    

Click here for diff

"typename" is a C++ keyword, so pg_upgrade.h fails to compile in C++.  
Fortunately, there seems no likely reason for somebody to need to  
do that.  Nonetheless, it's project policy that all .h files should  
pass cpluspluscheck, so rename the argument to fix that.  
  
Oversight in 57c081de0; back-patch as that was.  (The policy requiring  
pg_upgrade.h to pass cpluspluscheck only goes back to v12, but it  
seems best to keep this code looking the same in all branches.)  

M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c

Fix typo and outdated information in README.barrier

commit   : e71219ca11fabf5a1a184e76a0b1959e06de8e0d    
  
author   : David Rowley <drowley@postgresql.org>    
date     : Tue, 18 May 2021 09:57:43 +1200    
  
committer: David Rowley <drowley@postgresql.org>    
date     : Tue, 18 May 2021 09:57:43 +1200    

Click here for diff

README.barrier didn't seem to get the memo when atomics were added. Fix  
that.  
  
Author: Tatsuo Ishii, David Rowley  
Discussion: https://postgr.es/m/20210516.211133.2159010194908437625.t-ishii%40sraoss.co.jp  
Backpatch-through: 9.6, oldest supported release  

M src/backend/storage/lmgr/README.barrier

Be more careful about barriers when releasing BackgroundWorkerSlots.

commit   : c76ceacbd0c9bc079fb577e7b707ff3940be9e00    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 15 May 2021 12:21:06 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 15 May 2021 12:21:06 -0400    

Click here for diff

ForgetBackgroundWorker lacked any memory barrier at all, while  
BackgroundWorkerStateChange had one but unaccountably did  
additional manipulation of the slot after the barrier.  AFAICS,  
the rule must be that the barrier is immediately before setting  
or clearing slot->in_use.  
  
It looks like back in 9.6 when ForgetBackgroundWorker was first  
written, there might have been some case for not needing a  
barrier there, but I'm not very convinced of that --- the fact  
that the load of bgw_notify_pid is in the caller doesn't seem  
to guarantee no memory ordering problem.  So patch 9.6 too.  
  
It's likely that this doesn't fix any observable bug on Intel  
hardware, but machines with weaker memory ordering rules could  
have problems here.  
  
Discussion: https://postgr.es/m/4046084.1620244003@sss.pgh.pa.us  

M src/backend/postmaster/bgworker.c

Doc: correct erroneous entry in this week's minor release notes.

commit   : e5f982156952c0a9e9380f07a204e808f1fe1fbf    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 17:36:20 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 17:36:20 -0400    

Click here for diff

The patch to disallow a NULL specification in combination with  
GENERATED ... AS IDENTITY applied to both ALWAYS and BY DEFAULT  
variants of that clause, not only the former.  
  
Noted by Shay Rojansky.  
  
Discussion: https://postgr.es/m/CADT4RqAwD3A=RvGiQU9AiTK-6VeuXcycwPHmJPv_OBCJFYOEww@mail.gmail.com  

M doc/src/sgml/release-10.sgml

Prevent infinite insertion loops in spgdoinsert().

commit   : e87a0d2e0aa5a78fea8d672c6883b78124873d6e    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 15:07:34 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 15:07:34 -0400    

Click here for diff

Formerly we just relied on operator classes that assert longValuesOK  
to eventually shorten the leaf value enough to fit on an index page.  
That fails since the introduction of INCLUDE-column support (commit  
09c1c6ab4), because the INCLUDE columns might alone take up more  
than a page, meaning no amount of leaf-datum compaction will get  
the job done.  At least with spgtextproc.c, that leads to an infinite  
loop, since spgtextproc.c won't throw an error for not being able  
to shorten the leaf datum anymore.  
  
To fix without breaking cases that would otherwise work, add logic  
to spgdoinsert() to verify that the leaf tuple size is decreasing  
after each "choose" step.  Some opclasses might not decrease the  
size on every single cycle, and in any case, alignment roundoff  
of the tuple size could obscure small gains.  Therefore, allow  
up to 10 cycles without additional savings before throwing an  
error.  (Perhaps this number will need adjustment, but it seems  
quite generous right now.)  
  
As long as we've developed this logic, let's back-patch it.  
The back branches don't have INCLUDE columns to worry about, but  
this seems like a good defense against possible bugs in operator  
classes.  We already know that an infinite loop here is pretty  
unpleasant, so having a defense seems to outweigh the risk of  
breaking things.  (Note that spgtextproc.c is actually the only  
known opclass with longValuesOK support, so that this is all moot  
for known non-core opclasses anyway.)  
  
Per report from Dilip Kumar.  
  
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com  

M doc/src/sgml/spgist.sgml
M src/backend/access/spgist/spgdoinsert.c

Fix query-cancel handling in spgdoinsert().

commit   : fe64adc9317740569c18e948d0d838d46696c5d3    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 13:26:55 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 13:26:55 -0400    

Click here for diff

Knowing that a buggy opclass could cause an infinite insertion loop,  
spgdoinsert() intended to allow its loop to be interrupted by query  
cancel.  However, that never actually worked, because in iterations  
after the first, we'd be holding buffer lock(s) which would cause  
InterruptHoldoffCount to be positive, preventing servicing of the  
interrupt.  
  
To fix, check if an interrupt is pending, and if so fall out of  
the insertion loop and service the interrupt after we've released  
the buffers.  If it was indeed a query cancel, that's the end of  
the matter.  If it was a non-canceling interrupt reason, make use  
of the existing provision to retry the whole insertion.  (This isn't  
as wasteful as it might seem, since any upper-level index tuples we  
already created should be usable in the next attempt.)  
  
While there's no known instance of such a bug in existing release  
branches, it still seems like a good idea to back-patch this to  
all supported branches, since the behavior is fairly nasty if a  
loop does happen --- not only is it uncancelable, but it will  
quickly consume memory to the point of an OOM failure.  In any  
case, this code is certainly not working as intended.  
  
Per report from Dilip Kumar.  
  
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com  

M src/backend/access/spgist/spgdoinsert.c

Refactor CHECK_FOR_INTERRUPTS() to add flexibility.

commit   : 39b8ccb08619cb3e4c97d90134fcb7fb48c79bac    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 12:54:26 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 May 2021 12:54:26 -0400    

Click here for diff

Split up CHECK_FOR_INTERRUPTS() to provide an additional macro  
INTERRUPTS_PENDING_CONDITION(), which just tests whether an  
interrupt is pending without attempting to service it.  This is  
useful in situations where the caller knows that interrupts are  
blocked, and would like to find out if it's worth the trouble  
to unblock them.  
  
Also add INTERRUPTS_CAN_BE_PROCESSED(), which indicates whether  
CHECK_FOR_INTERRUPTS() can be relied on to clear the pending interrupt.  
  
This commit doesn't actually add any uses of the new macros,  
but a follow-on bug fix will do so.  Back-patch to all supported  
branches to provide infrastructure for that fix.  
  
Alvaro Herrera and Tom Lane  
  
Discussion: https://postgr.es/m/20210513155351.GA7848@alvherre.pgsql  

M src/backend/tcop/postgres.c
M src/include/miscadmin.h

Rename the logical replication global "wrconn"

commit   : b5b7cba7643ca96f0ff882e643ee1ebd8c86003a    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 12 May 2021 19:13:54 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 12 May 2021 19:13:54 -0400    

Click here for diff

The worker.c global wrconn is only meant to be used by logical apply/  
tablesync workers, but there are other variables with the same name. To  
reduce future confusion rename the global from "wrconn" to  
"LogRepWorkerWalRcvConn".  
  
While this is just cosmetic, it seems better to backpatch it all the way  
back to 10 where this code appeared, to avoid future backpatching  
issues.  
  
Author: Peter Smith <smithpb2250@gmail.com>  
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com  

M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/include/replication/worker_internal.h