PostgreSQL 13.4 commit log

Stamp 13.4.

commit   : e849f3f1f884ad140b60a24354c6371cbd2efbb6    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 9 Aug 2021 16:49:05 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 9 Aug 2021 16:49:05 -0400    

Click here for diff

M configure
M configure.in

Last-minute updates for release notes.

commit   : 0145ec9be92b2146e7b94f286cb3dab9eb77ef70    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 9 Aug 2021 14:41:00 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 9 Aug 2021 14:41:00 -0400    

Click here for diff

Security: CVE-2021-3677  

M doc/src/sgml/release-13.sgml

Translation updates

commit   : dc10035ecc8256d4489eb33ba3026efe9bfba082    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 9 Aug 2021 11:56:40 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 9 Aug 2021 11:56:40 +0200    

Click here for diff

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 9bb123c161ac8f773572e112ced524b99e81c1d9  

M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_resetwal/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/psql/po/de.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/libpq/po/es.po

Doc: Fix misleading statement about VACUUM memory limits

commit   : bb08e78972b1d0ec9f11f3d0b18903ec7a216855    
  
author   : David Rowley <[email protected]>    
date     : Mon, 9 Aug 2021 16:47:25 +1200    
  
committer: David Rowley <[email protected]>    
date     : Mon, 9 Aug 2021 16:47:25 +1200    

Click here for diff

In ec34040af I added a mention that there was no point in setting  
maintenance_work_limit to anything higher than 1GB for vacuum, but that  
was incorrect as ginInsertCleanup() also looks at what  
maintenance_work_mem is set to during VACUUM and that's not limited to  
1GB.  
  
Here I attempt to make it more clear that the limitation is only around  
the number of dead tuple identifiers that we can collect during VACUUM.  
  
I've also added a note to autovacuum_work_mem to mention this limitation.  
I didn't do that in ec34040af as I'd had some wrong-headed ideas about  
just limiting the maximum value for that GUC to 1GB.  
  
Author: David Rowley  
Discussion: https://postgr.es/m/CAApHDvpGwOAvunp-E-bN_rbAs3hmxMoasm5pzkYDbf36h73s7w@mail.gmail.com  
Backpatch-through: 9.6, same as ec34040af  

M doc/src/sgml/config.sgml

doc: mention pg_upgrade extension script

commit   : ff18b8d1b169b2bf47daab7d92c5376bc19c0748    
  
author   : Bruce Momjian <[email protected]>    
date     : Sun, 8 Aug 2021 21:05:46 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Sun, 8 Aug 2021 21:05:46 -0400    

Click here for diff

Since commit e462856a7a, pg_upgrade automatically creates a script to  
update extensions, so mention that instead of ALTER EXTENSION.  
  
Backpatch-through: 9.6  

M doc/src/sgml/ref/pgupgrade.sgml

Doc: remove bogus <indexterm> items.

commit   : 410c5a08df4381917cd6da7d94609644e10ca322    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 8 Aug 2021 15:35:30 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 8 Aug 2021 15:35:30 -0400    

Click here for diff

Copy-and-pasteo in 665c5855e, evidently.  The 9.6 docs toolchain  
whined about duplicate index entries, though our modern toolchain  
doesn't.  In any case, these GUCs surely are not about the  
default settings of these values.  

M doc/src/sgml/config.sgml

Release notes for 13.4, 12.8, 11.13, 10.18, 9.6.23.

commit   : 6432cf92658307688be4e241eb16fc9c41a8c0cf    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 8 Aug 2021 14:35:19 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 8 Aug 2021 14:35:19 -0400    

Click here for diff

M doc/src/sgml/release-13.sgml

Really fix the ambiguity in REFRESH MATERIALIZED VIEW CONCURRENTLY.

commit   : ba9f665a4413f855bbf4b544481db326f7b9bd73    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 7 Aug 2021 13:29:32 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 7 Aug 2021 13:29:32 -0400    

Click here for diff

Rather than trying to pick table aliases that won't conflict with  
any possible user-defined matview column name, adjust the queries'  
syntax so that the aliases are only used in places where they can't be  
mistaken for column names.  Mostly this consists of writing "alias.*"  
not just "alias", which adds clarity for humans as well as machines.  
We do have the issue that "SELECT alias.*" acts differently from  
"SELECT alias", but we can use the same hack ruleutils.c uses for  
whole-row variables in SELECT lists: write "alias.*::compositetype".  
  
We might as well revert to the original aliases after doing this;  
they're a bit easier to read.  
  
Like 75d66d10e, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/matview.c
M src/test/regress/expected/matview.out
M src/test/regress/sql/matview.sql

Adjust the integer overflow tests in the numeric code.

commit   : da188b993450c824b03e7dca18c27e9f4c04754f    
  
author   : Dean Rasheed <[email protected]>    
date     : Fri, 6 Aug 2021 21:31:20 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Fri, 6 Aug 2021 21:31:20 +0100    

Click here for diff

Formerly, the numeric code tested whether an integer value of a larger  
type would fit in a smaller type by casting it to the smaller type and  
then testing if the reverse conversion produced the original value.  
That's perfectly fine, except that it caused a test failure on  
buildfarm animal castoroides, most likely due to a compiler bug.  
  
Instead, do these tests by comparing against PG_INT16/32_MIN/MAX. That  
matches existing code in other places, such as int84(), which is more  
widely tested, and so is less likely to go wrong.  
  
While at it, add regression tests covering the numeric-to-int8/4/2  
conversions, and adjust the recently added tests to the style of  
434ddfb79a (on the v11 branch) to make failures easier to diagnose.  
  
Per buildfarm via Tom Lane, reviewed by Tom Lane.  
  
Discussion: https://postgr.es/m/2394813.1628179479%40sss.pgh.pa.us  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql

Fix wording

commit   : d3ad6566a130988af7693267bc8605b86772618c    
  
author   : Peter Eisentraut <[email protected]>    
date     : Fri, 6 Aug 2021 20:55:59 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Fri, 6 Aug 2021 20:55:59 +0200    

Click here for diff

M doc/src/sgml/rules.sgml
M src/backend/utils/adt/rangetypes_selfuncs.c
M src/backend/utils/adt/rangetypes_spgist.c
M src/bin/pg_resetwal/pg_resetwal.c

First-draft release notes for 13.4.

commit   : 2f38ec6a157b60291ece1deb072bfff84f317334    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 6 Aug 2021 14:54:59 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 6 Aug 2021 14:54:59 -0400    

Click here for diff

As usual, the release notes for older branches will be made by cutting  
these down, but put them up for community review first.  

M doc/src/sgml/release-13.sgml

postgres_fdw: Fix issues with generated columns in foreign tables.

commit   : 388a81bf4df4fc86a947368d94094e5276c2f255    
  
author   : Etsuro Fujita <[email protected]>    
date     : Thu, 5 Aug 2021 20:00:02 +0900    
  
committer: Etsuro Fujita <[email protected]>    
date     : Thu, 5 Aug 2021 20:00:02 +0900    

Click here for diff

postgres_fdw imported generated columns from the remote tables as plain  
columns, and caused failures like "ERROR: cannot insert a non-DEFAULT  
value into column "foo"" when inserting into the foreign tables, as it  
tried to insert values into the generated columns.  To fix, we do the  
following under the assumption that generated columns in a postgres_fdw  
foreign table are defined so that they represent generated columns in  
the underlying remote table:  
  
* Send DEFAULT for the generated columns to the foreign server on insert  
  or update, not generated column values computed on the local server.  
* Add to postgresImportForeignSchema() an option "import_generated" to  
  include column generated expressions in the definitions of foreign  
  tables imported from a foreign server.  The option is true by default.  
  
The assumption seems reasonable, because that would make a query of the  
postgres_fdw foreign table return values for the generated columns that  
are consistent with the generated expression.  
  
While here, fix another issue in postgresImportForeignSchema(): it tried  
to include column generated expressions as column default expressions in  
the foreign table definitions when the import_default option was enabled.  
  
Per bug #16631 from Daniel Cherniy.  Back-patch to v12 where generated  
columns were added.  
  
Discussion: https://postgr.es/m/16631-e929fe9db0ffc7cf%40postgresql.org  

M contrib/postgres_fdw/deparse.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql
M doc/src/sgml/postgres-fdw.sgml

Fix division-by-zero error in to_char() with 'EEEE' format.

commit   : a72ad63154256bd3adf1c885f8c3371e9e8ce67a    
  
author   : Dean Rasheed <[email protected]>    
date     : Thu, 5 Aug 2021 09:29:13 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Thu, 5 Aug 2021 09:29:13 +0100    

Click here for diff

This fixes a long-standing bug when using to_char() to format a  
numeric value in scientific notation -- if the value's exponent is  
less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a  
division-by-zero error.  
  
The reason for this error was that get_str_from_var_sci() divides its  
input by 10^exp, which it produced using power_var_int(). However, the  
underflow test in power_var_int() causes it to return zero if the  
result scale is too small. That's not a problem for power_var_int()'s  
only other caller, power_var(), since that limits the rscale to 1000,  
but in get_str_from_var_sci() the exponent can be much smaller,  
requiring a much larger rscale. Fix by introducing a new function to  
compute 10^exp directly, with no rscale limit. This also allows 10^exp  
to be computed more efficiently, without any numeric multiplication,  
division or rounding.  
  
Discussion: https://postgr.es/m/CAEZATCWhojfH4whaqgUKBe8D5jNHB8ytzemL-PnRx+KCTyMXmg@mail.gmail.com  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql

C comment: correct heading of extension query

commit   : 47a573d9113e18864dcb637b72ee3ac1a9ec6b5e    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 12:26:08 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 12:26:08 -0400    

Click here for diff

Reported-by: Justin Pryzby  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

M src/bin/pg_upgrade/version.c

doc: interval spill method for units greater than months

commit   : 1dd84002060cc2cdc20b6cebd0f6bb2b56c7b35b    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 12:17:58 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 12:17:58 -0400    

Click here for diff

Units are _truncated_ to months, but only in back branches since the  
recent commit.  
  
Reported-by: Bryn Llewellyn  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6 to 14  

M doc/src/sgml/datatype.sgml

pg_upgrade: warn about extensions that need updating

commit   : a81c71e3a830aecdfea7a53ae75781e50ba66018    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 11:58:15 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 11:58:15 -0400    

Click here for diff

Also create a script that can be run to update them.  
  
Reported-by: Dave Cramer  
  
Discussion: https://postgr.es/m/CADK3HHKawwbOcGwMGnDuAf3-U8YfvTcS8jqDv3UM=niijs3MMA@mail.gmail.com  
  
Backpatch-through: 9.6  

M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c

pg_upgrade: improve docs about extension upgrades

commit   : 0e6cf3c6f43453f848b41a4f87434d2d9c627a34    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 11:27:33 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 11:27:33 -0400    

Click here for diff

The previous wording was unclear about the steps needed to upgrade  
extensions, and how to update them after pg_upgrade.  
  
Reported-by: Dave Cramer  
  
Discussion: https://postgr.es/m/CADK3HHKawwbOcGwMGnDuAf3-U8YfvTcS8jqDv3UM=niijs3MMA@mail.gmail.com  
  
Backpatch-through: 9.6  

M doc/src/sgml/ref/pgupgrade.sgml

doc: mention inheritance's tableoid can be used in partitioning

commit   : 7134b8cacca445115df7115615c063f5cc119b06    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 11:11:51 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 11:11:51 -0400    

Click here for diff

Previously tableoid was not mentioned in the partition doc section.  We  
only had a link to the "all the normal rules" of inheritance section.  
  
Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 10  

M doc/src/sgml/ddl.sgml

doc: add example of using pg_dump with GNU split and gzip

commit   : 073069075f227f27342e4d935d937141a2ba3910    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 10:57:32 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 3 Aug 2021 10:57:32 -0400    

Click here for diff

This is only possible with GNU split, not other versions like BSD split.  
  
Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

M doc/src/sgml/backup.sgml

Use elog, not Assert, to report failure to provide an outer snapshot.

commit   : 93f99693f9c2a8e0662d847c4ba2abed1628c1d8    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 31 Jul 2021 11:50:14 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 31 Jul 2021 11:50:14 -0400    

Click here for diff

As of commit 84f5c2908, executing SQL commands (via SPI or otherwise)  
requires having either an active Portal, or a caller-established  
active snapshot.  We were simply Assert'ing that that's the case.  
But we've now had a couple different reports of people testing  
extensions that didn't meet this requirement, and were confused by  
the resulting crash.  Let's convert the Assert to a test-and-elog,  
in hopes of making the issue clearer for extension authors.  
  
Per gripes from Liu Huailing and RekGRpth.  Back-patch to v11,  
like the prior commit.  
  
Discussion: https://postgr.es/m/OSZPR01MB6215671E3C5956A034A080DFBEEC9@OSZPR01MB6215.jpnprd01.prod.outlook.com  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/tcop/pquery.c

Fix corner-case errors and loss of precision in numeric_power().

commit   : 053ec4e0c4e8027bc084d25a846e35ba84ae966c    
  
author   : Dean Rasheed <[email protected]>    
date     : Sat, 31 Jul 2021 11:25:39 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Sat, 31 Jul 2021 11:25:39 +0100    

Click here for diff

This fixes a couple of related problems that arise when raising  
numbers to very large powers.  
  
Firstly, when raising a negative number to a very large integer power,  
the result should be well-defined, but the previous code would only  
cope if the exponent was small enough to go through power_var_int().  
Otherwise it would throw an internal error, attempting to take the  
logarithm of a negative number. Fix this by adding suitable handling  
to the general case in power_var() to cope with negative bases,  
checking for integer powers there.  
  
Next, when raising a (positive or negative) number whose absolute  
value is slightly less than 1 to a very large power, the result should  
approach zero as the power is increased. However, in some cases, for  
sufficiently large powers, this would lose all precision and return 1  
instead of 0. This was due to the way that the local_rscale was being  
calculated for the final full-precision calculation:  
  
  local_rscale = rscale + (int) val - ln_dweight + 8  
  
The first two terms on the right hand side are meant to give the  
number of significant digits required in the result ("val" being the  
estimated result weight). However, this failed to account for the fact  
that rscale is clipped to a maximum of NUMERIC_MAX_DISPLAY_SCALE  
(1000), and the result weight might be less then -1000, causing their  
sum to be negative, leading to a loss of precision. Fix this by  
forcing the number of significant digits calculated to be nonnegative.  
It's OK for it to be zero (when the result weight is less than -1000),  
since the local_rscale value then includes a few extra digits to  
ensure an accurate result.  
  
Finally, add additional underflow checks to exp_var() and power_var(),  
so that they consistently return zero for cases like this where the  
result is indistinguishable from zero. Some paths through this code  
already returned zero in such cases, but others were throwing overflow  
errors.  
  
Dean Rasheed, reviewed by Yugo Nagata.  
  
Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUgT5xNoOqce5=6Su0bCR0w@mail.gmail.com  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql

Fix range check in ECPG numeric to int conversion

commit   : 171bf1cea579ce11bc7f0b2986f052022f2b6be1    
  
author   : John Naylor <[email protected]>    
date     : Fri, 30 Jul 2021 13:50:23 -0400    
  
committer: John Naylor <[email protected]>    
date     : Fri, 30 Jul 2021 13:50:23 -0400    

Click here for diff

The previous coding guarded against -INT_MAX instead of INT_MIN,  
leading to -2147483648 being rejected as out of range.  
  
Per bug #17128 from Kevin Sweet  
  
Discussion: https://www.postgresql.org/message-id/flat/17128-55a8a879727a3e3a%40postgresql.org  
Reviewed-by: Tom Lane  
Backpatch to all supported branches  

M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/pgtypeslib/numeric.c
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.c
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.stderr
M src/interfaces/ecpg/test/expected/pgtypeslib-num_test.stdout
M src/interfaces/ecpg/test/pgtypeslib/num_test.pgc

Close yet another race condition in replication slot test code

commit   : 41d27ee7b870c1a1213704d3c020a01eb55799b0    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 29 Jul 2021 17:26:25 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 29 Jul 2021 17:26:25 -0400    

Click here for diff

Buildfarm shows that this test has a further failure mode when a  
checkpoint starts earlier than expected, so we detect a "checkpoint  
completed" line that's not the one we want.  Change the config to try  
and prevent this.  
  
Per buildfarm  
  
Author: Kyotaro Horiguchi <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/test/recovery/t/019_replslot_limit.pl

Add missing exit() in pg_verifybackup when failing to find pg_waldump

commit   : efe169c90090684d2a4ffb0810a0cf2c9b72e19d    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 29 Jul 2021 11:00:00 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 29 Jul 2021 11:00:00 +0900    

Click here for diff

pg_verifybackup needs by default pg_waldump to check after a range of  
WAL segments required for a backup, except if --no-parse-wal is  
specified.  The code checked for the presence of the binary pg_waldump  
in an installation and reported an error, but it forgot to properly  
exit().  This could lead to confusing errors reported.  
  
Reviewed-by: Robert Haas, Fabien Coelho  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 13  

M src/bin/pg_verifybackup/pg_verifybackup.c

Update minimum recovery point on truncation during WAL replay of abort record.

commit   : a66b05b422d74111f2a6407108f4ba7bdddbb6a0    
  
author   : Fujii Masao <[email protected]>    
date     : Thu, 29 Jul 2021 01:34:13 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Thu, 29 Jul 2021 01:34:13 +0900    

Click here for diff

If a file is truncated, we must update minRecoveryPoint. Once a file is  
truncated, there's no going back; it would not be safe to stop recovery  
at a point earlier than that anymore.  
  
Commit 7bffc9b7bf changed xact_redo_commit() so that it updates  
minRecoveryPoint on truncation, but forgot to change xact_redo_abort().  
  
Back-patch to all supported versions.  
  
Reported-by: [email protected]  
Author: Fujii Masao  
Reviewed-by: Heikki Linnakangas  
Discussion: https://postgr.es/m/b029fce3-4fac-4265-968e-16f36ff4d075.mengjuan.cmj@alibaba-inc.com  

M src/backend/access/transam/xact.c

Doc: Clarify lock levels taken during ATTACH PARTITION

commit   : aa1e9211ecf7ef410b4313f0e061de087aed0aa2    
  
author   : David Rowley <[email protected]>    
date     : Wed, 28 Jul 2021 15:01:40 +1200    
  
committer: David Rowley <[email protected]>    
date     : Wed, 28 Jul 2021 15:01:40 +1200    

Click here for diff

It wasn't all that clear which lock levels, if any, would be held on the  
DEFAULT partition during an ATTACH PARTITION operation.  
  
Also, clarify which locks will be taken if the DEFAULT partition or the  
table being attached are themselves partitioned tables.  
  
Here I'm only backpatching to v12 as before then we obtained an ACCESS  
EXCLUSIVE lock on the partitioned table.  It seems much less relevant to  
mention which locks are taken on other tables when the partitioned table  
itself is locked with an ACCESS EXCLUSIVE lock.  
  
Author: Matthias van de Meent, David Rowley  
Discussion: https://postgr.es/m/CAEze2WiTB6iwrV8W_J=fnrnZ7fowW3qu-8iQ8zCHP3FiQ6+o-A@mail.gmail.com  
Backpatch-through: 12  

M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/alter_table.sgml

Set pg_setting.pending_restart when pertinent config lines are removed

commit   : b8f91d7f926368115c27b978c939174c96df1a5f    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 27 Jul 2021 15:44:12 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 27 Jul 2021 15:44:12 -0400    

Click here for diff

This changes the behavior of examining the pg_file_settings view after  
changing a config option that requires restart.  The user needs to know  
that any change of such options does not take effect until a restart,  
and this worked correctly if the line is edited without removing it.  
However, for the case where the line is removed altogether, the flag  
doesn't get set, because a flag was only set in set_config_option, but  
that's not called for lines removed.  Repair.  
  
(Ref.: commits 62d16c7fc561 and a486e35706ea)  
  
Author: Álvaro Herrera <[email protected]>  
Reviewed-by: Daniel Gustafsson <[email protected]>  
Reviewed-by: Tom Lane <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/misc/guc-file.l

Avoid using ambiguous word "non-negative" in error messages.

commit   : 92913fc290f3adb3fe937485474a11ac6708e4b0    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 28 Jul 2021 01:21:52 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 28 Jul 2021 01:21:52 +0900    

Click here for diff

The error messages using the word "non-negative" are confusing  
because it's ambiguous about whether it accepts zero or not.  
This commit improves those error messages by replacing it with  
less ambiguous word like "greater than zero" or  
"greater than or equal to zero".  
  
Also this commit added the note about the word "non-negative" to  
the error message style guide, to help writing the new error messages.  
  
When postgres_fdw option fetch_size was set to zero, previously  
the error message "fetch_size requires a non-negative integer value"  
was reported. This error message was outright buggy. Therefore  
back-patch to all supported versions where such buggy error message  
could be thrown.  
  
Reported-by: Hou Zhijie  
Author: Bharath Rupireddy  
Reviewed-by: Kyotaro Horiguchi, Fujii Masao  
Discussion: https://postgr.es/m/OS0PR01MB5716415335A06B489F1B3A8194569@OS0PR01MB5716.jpnprd01.prod.outlook.com  

M contrib/postgres_fdw/option.c
M doc/src/sgml/sources.sgml
M src/backend/partitioning/partbounds.c
M src/backend/utils/adt/tsquery_op.c
M src/test/modules/test_shm_mq/test.c
M src/test/regress/expected/hash_part.out

doc: for various substring funcs, document if only first match

commit   : d629fcf4b36ca8c1e4e9677745ff7d725433ee19    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 26 Jul 2021 22:54:35 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 26 Jul 2021 22:54:35 -0400    

Click here for diff

Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 13  

M doc/src/sgml/func.sgml

pg_resetxlog: add option to set oldest xid & use by pg_upgrade

commit   : 0a5e708e2bd87adb0779d7c8758e5743cc1c0adf    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 26 Jul 2021 22:38:14 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 26 Jul 2021 22:38:14 -0400    

Click here for diff

Add pg_resetxlog -u option to set the oldest xid in pg_control.  
Previously -x set this value be -2 billion less than the -x value.  
However, this causes the server to immediately scan all relation's  
relfrozenxid so it can advance pg_control's oldest xid to be inside the  
autovacuum_freeze_max_age range, which is inefficient and might disrupt  
diagnostic recovery.  pg_upgrade will use this option to better create  
the new cluster to match the old cluster.  
  
Reported-by: Jason Harvey, Floris Van Nee  
  
Discussion: https://postgr.es/m/[email protected], [email protected]  
  
Author: Bertrand Drouvot  
  
Backpatch-through: 9.6  

M doc/src/sgml/ref/pg_resetwal.sgml
M src/bin/pg_resetwal/pg_resetwal.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/pg_upgrade.h

Harden pg_stat_statements tests against CLOBBER_CACHE_ALWAYS.

commit   : 882d15144bbb21856676ff046faee1e9a580e3e4    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 25 Jul 2021 23:25:15 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 25 Jul 2021 23:25:15 -0400    

Click here for diff

Turns out the buildfarm hasn't been testing this, which will soon change.  
  
Julien Rouhaud, per report from me  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/sql/pg_stat_statements.sql

Fix a couple of memory leaks in src/bin/pg_basebackup/

commit   : 2c7395aad7e849235faac4ead5f69700cfd3c21b    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 26 Jul 2021 11:14:11 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 26 Jul 2021 11:14:11 +0900    

Click here for diff

These have been introduced by 7fbe0c8, and could happen for  
pg_basebackup and pg_receivewal.  
  
Per report from Coverity for the ones in walmethods.c, I have spotted  
the ones in receivelog.c after more review.  
  
Backpatch-through: 10  

M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h

Get rid of artificial restriction on hash table sizes on Windows.

commit   : 2b8f3f5a7c0ede24903782fcffe2553fec01bfbe    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 25 Jul 2021 14:02:27 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 25 Jul 2021 14:02:27 -0400    

Click here for diff

The point of introducing the hash_mem_multiplier GUC was to let users  
reproduce the old behavior of hash aggregation, i.e. that it could use  
more than work_mem at need.  However, the implementation failed to get  
the job done on Win64, where work_mem is clamped to 2GB to protect  
various places that calculate memory sizes using "long int".  As  
written, the same clamp was applied to hash_mem.  This resulted in  
severe performance regressions for queries requiring a bit more than  
2GB for hash aggregation, as they now spill to disk and there's no  
way to stop that.  
  
Getting rid of the work_mem restriction seems like a good idea, but  
it's a big job and could not conceivably be back-patched.  However,  
there's only a fairly small number of places that are concerned with  
the hash_mem value, and it turns out to be possible to remove the  
restriction there without too much code churn or any ABI breaks.  
So, let's do that for now to fix the regression, and leave the  
larger task for another day.  
  
This patch does introduce a bit more infrastructure that should help  
with the larger task, namely pg_bitutils.h support for working with  
size_t values.  
  
Per gripe from Laurent Hasson.  Back-patch to v13 where the  
behavior change came in.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/MN2PR15MB25601E80A9B6D1BA6F592B1985E39@MN2PR15MB2560.namprd15.prod.outlook.com  

M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeHash.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/include/miscadmin.h
M src/include/port/pg_bitutils.h

Make the standby server promptly handle interrupt signals.

commit   : 8d091922ffcc27323ffe2127e228e302b9f153f4    
  
author   : Fujii Masao <[email protected]>    
date     : Mon, 16 Nov 2020 18:27:51 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Mon, 16 Nov 2020 18:27:51 +0900    

Click here for diff

This commit changes the startup process in the standby server so that  
it handles the interrupt signals after waiting for wal_retrieve_retry_interval  
on the latch and resetting it, before entering another wait on the latch.  
This change causes the standby server to promptly handle interrupt signals.  
  
Otherwise, previously, there was the case where the standby needs to  
wait extra five seconds to shutdown when the shutdown request arrived  
while the startup process was waiting for wal_retrieve_retry_interval  
on the latch.  
  
Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty  
Reviewed-by: Soumyadeep Chakraborty  
Discussion: https://postgr.es/m/[email protected]  
  
Per discussion of BUG #17073, back-patch to all supported versions.  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/transam/xlog.c

Fix check for conflicting session- vs transaction-level locks.

commit   : f47408cdc19c2bbe357703afd81037b6d12ed3ba    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 24 Jul 2021 18:35:52 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 24 Jul 2021 18:35:52 -0400    

Click here for diff

We have an implementation restriction that PREPARE TRANSACTION can't  
handle cases where both session-lifespan and transaction-lifespan locks  
are held on the same lockable object.  (That's because we'd otherwise  
need to acquire a new PROCLOCK entry during post-prepare cleanup, which  
is an operation that might fail.  The situation can only arise with odd  
usages of advisory locks, so removing the restriction is probably not  
worth the amount of effort it would take.)  AtPrepare_Locks attempted  
to enforce this, but its logic was many bricks shy of a load, because  
it only detected cases where the session and transaction locks had the  
same lockmode.  Locks of different modes on the same object would lead  
to the rather unhelpful message "PANIC: we seem to have dropped a bit  
somewhere".  
  
To fix, build a transient hashtable with one entry per locktag,  
not one per locktag + mode, and use that to detect conflicts.  
  
Per bug #17122 from Alexander Pyhalov.  This bug is ancient,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/storage/lmgr/lock.c
M src/test/regress/expected/prepared_xacts.out
M src/test/regress/expected/prepared_xacts_1.out
M src/test/regress/sql/prepared_xacts.sql

Make printf("%s", NULL) print "(null)" instead of crashing.

commit   : c0a6f83deba9c55459e52ad4475207f62354a1f1    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 24 Jul 2021 13:41:17 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 24 Jul 2021 13:41:17 -0400    

Click here for diff

We previously took a hard-line attitude that callers should never print  
a null string pointer, and doing so is worthy of an assertion failure  
or crash.  However, we've long since flushed out any easy-to-find bugs  
of that nature.  What remains is a lot of code that perhaps could fail  
that way in hard-to-reach corner cases.  For example, in something as  
simple as  
    ereport(ERROR,  
            (errcode(ERRCODE_UNDEFINED_OBJECT),  
             errmsg("constraint \"%s\" for table \"%s\" does not exist",  
                    conname, get_rel_name(relid))));  
one must wonder whether it's completely guaranteed that get_rel_name  
cannot return NULL in this context.  If such a situation did occur,  
the existing policy converts what might be a pretty minor bug into  
a server crash condition.  This is not good for robustness.  
  
Hence, let's follow the lead of glibc and print "(null)" instead  
of failing.  We should, of course, still consider it a bug if that  
behavior is reachable in ordinary use; but crashing seems less  
desirable than not crashing.  
  
This fix works across-the-board in v12 and up, where we always use  
src/port/snprintf.c.  Before that, on most platforms we're at the mercy  
of the local libc, but it appears that Solaris 10 is the only supported  
platform where we'd still get a crash.  Most other platforms such as  
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some  
point.  (AIX and HPUX just print "" not "(null)", but that's close  
enough.)  I've not checked what Windows' native printf would do, but  
it doesn't matter because we've long used snprintf.c on that platform.  
  
In v12 and up, also const-ify related code so that we're not casting  
away const on the constant string.  This is just neatnik-ism, since  
next to no compilers will warn about that.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/port/snprintf.c

jit: Don't inline functions that access thread-locals.

commit   : 1cfc9f2ef61df81fa54a697081e372672c23e980    
  
author   : Thomas Munro <[email protected]>    
date     : Thu, 22 Jul 2021 14:11:17 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Thu, 22 Jul 2021 14:11:17 +1200    

Click here for diff

Code inlined by LLVM can crash or fail with "Relocation type not  
implemented yet!" if it tries to access thread local variables.  Don't  
inline such code.  
  
Back-patch to 11, where LLVM arrived.  Bug #16696.  
  
Author: Dmitry Marakasov <[email protected]>  
Reviewed-by: Andres Freund <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/jit/llvm/llvmjit_inline.cpp

Doc: improve documentation about exponentiation operator.

commit   : 8344979d5bbc63aa8fb8c804bd933c887a2abb58    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 21 Jul 2021 18:03:33 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 21 Jul 2021 18:03:33 -0400    

Click here for diff

Now that we're not having to wedge this into the straitjacket of  
the old operator table format, we can add another example to  
clarify the point about left-to-right associativity.  
  
Per suggestion from mdione at grulic.org.ar.  
  
https://postgr.es/m/[email protected]  

M doc/src/sgml/func.sgml

Document "B" and "us" as accepted units in postgres.conf.sample

commit   : 0b5dc1f171c4c7433cff2949d7bf64aa912765b3    
  
author   : John Naylor <[email protected]>    
date     : Wed, 21 Jul 2021 10:17:07 -0400    
  
committer: John Naylor <[email protected]>    
date     : Wed, 21 Jul 2021 10:17:07 -0400    

Click here for diff

In postgresql.conf, memory and file size GUCs can be specified with "B"  
(bytes) as of b06d8e58b. Likewise, time GUCs can be specified with "us"  
(microseconds) as of caf626b2c. Update postgres.conf.sample to reflect  
that fact.  
  
Pavel Luzanov  
  
Backpatch to v12, which is the earliest version that allows both of  
these units. A separate commit will document the "B" case for v11.  
  
Discussion: https://www.postgresql.org/message-id/flat/f10d16fc-8fa0-1b3c-7371-cb3a35a13b7a%40postgrespro.ru  

M src/backend/utils/misc/postgresql.conf.sample

doc: Document that only superusers can use pg_import_system_collations().

commit   : df3264e73f6fa99fd04b97905d53050de011f035    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 21 Jul 2021 13:52:37 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 21 Jul 2021 13:52:37 +0900    

Click here for diff

Back-patch to v10 where pg_import_system_collations() was added.  
  
Author: Atsushi Torikoshi  
Reviewed-by: Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/func.sgml

Fix corner-case uninitialized-variable issues in plpgsql.

commit   : 0fce76b99d21e2c359d8355a9e3bfe4c3e3f17c9    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 20 Jul 2021 13:01:48 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 20 Jul 2021 13:01:48 -0400    

Click here for diff

If an error was raised during our initial attempt to check whether  
a successfully-compiled expression is "simple", subsequent calls of  
exec_stmt_execsql would suppose that stmt->mod_stmt was already computed  
when it had not been.  This could lead to assertion failures in debug  
builds; in production builds the effect would typically be to act as  
if INTO STRICT had been specified even when it had not been.  Of course  
that only matters if the subsequent attempt to execute the expression  
succeeds, so that the problem can only be reached by fixing a failure  
in some referenced, inline-able SQL function and then retrying the  
calling plpgsql function in the same session.  
  
(There might be even-more-obscure ways to change the expression's  
behavior without changing the plpgsql function, but that one seems  
like the only one people would be likely to hit in practice.)  
  
The most foolproof way to fix this would be to arrange for  
exec_prepare_plan to not set expr->plan until we've finished the  
subsidiary simple-expression check.  But it seems hard to do that  
without creating reference-count leak issues.  So settle for documenting  
the hazard in a comment and fixing exec_stmt_execsql to test separately  
for whether it's computed stmt->mod_stmt.  (That adds a test-and-branch  
per execution, but hopefully that's negligible in context.)  In v11 and  
up, also fix exec_stmt_call which had a variant of the same issue.  
  
Per bug #17113 from Alexander Lakhin.  Back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plpgsql/src/expected/plpgsql_simple.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
M src/pl/plpgsql/src/sql/plpgsql_simple.sql

Fix some issues with WAL segment opening for pg_receivewal --compress

commit   : fb2b86015a9f3e2e59ddbb59e18cda98363cbd7b    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 20 Jul 2021 12:12:51 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 20 Jul 2021 12:12:51 +0900    

Click here for diff

The logic handling the opening of new WAL segments was fuzzy when using  
--compress if a partial, non-compressed, segment with the same base name  
existed in the repository storing those files.  In this case, using  
--compress would cause the code to first check for the existence and the  
size of a non-compressed segment, followed by the opening of a new  
compressed, partial, segment.  The code was accidentally working  
correctly on most platforms as the buildfarm has proved, except  
bowerbird where gzflush() could fail in this code path.  It is wrong  
anyway to take the code path used pre-padding when creating a new  
partial, non-compressed, segment, so let's fix it.  
  
Note that this issue exists when users mix successive runs of  
pg_receivewal with or without compression, as discovered with the tests  
introduced by ffc9dda.  
  
While on it, this refactors the code so as code paths that need to know  
about the ".gz" suffix are down from four to one in walmethods.c, easing  
a bit the introduction of new compression methods.  This addresses a  
second issue where log messages generated for an unexpected failure  
would not show the compressed segment name involved, which was  
confusing, printing instead the name of the non-compressed equivalent.  
  
Reported-by: Georgios Kokolatos  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 10  

M src/bin/pg_basebackup/receivelog.c
M src/bin/pg_basebackup/walmethods.c
M src/bin/pg_basebackup/walmethods.h

Make new replication slot test code even less racy

commit   : ce413eba411633c5c5c9e326d6d0313c9764c711    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 19 Jul 2021 17:21:07 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 19 Jul 2021 17:21:07 -0400    

Click here for diff

Further fix the test code in ead9e51e8236, this time by waiting until  
the checkpoint has completed before moving on; this ensures that the  
WAL segment removal has already happened when we create the next slot.  
  
Author: Kyotaro Horiguchi <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/test/recovery/t/019_replslot_limit.pl

Don't allow to set replication slot_name as ''.

commit   : bfa2a926daf812bbda2038aec073344693954f16    
  
author   : Amit Kapila <[email protected]>    
date     : Mon, 19 Jul 2021 11:04:21 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Mon, 19 Jul 2021 11:04:21 +0530    

Click here for diff

We don't allow to create replication slot_name as an empty string ('') via  
SQL API pg_create_logical_replication_slot() but it is allowed to be set  
via Alter Subscription command. This will lead to apply worker repeatedly  
keep trying to stream data via slot_name '' and the user is not allowed to  
create the slot with that name.  
  
Author: Japin Li  
Reviewed-By: Ranier Vilela, Amit Kapila  
Backpatch-through: 10, where it was introduced  
Discussion: https://postgr.es/m/MEYP282MB1669CBD98E721C77CA696499B61A9@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM  

M src/backend/commands/subscriptioncmds.c
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql

doc: Mention CASCADE/RESTRICT for DROP STATISTICS

commit   : e159368027e342468bb7f9db0d8d42a76dc42f23    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 19 Jul 2021 12:39:53 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 19 Jul 2021 12:39:53 +0900    

Click here for diff

This grammar has no effect as there are no dependencies on statistics,  
but it is supported by the parser.  This is more consistent with the  
other DROP commands.  
  
Author: Vignesh C  
Discussion: https://postgr.es/m/CALDaNm1LA=yNmzcSfy+0oe6CEAgsxXRf_-UutE3ZncFi8QkFNQ@mail.gmail.com  
Backpatch-through: 10  

M doc/src/sgml/ref/drop_statistics.sgml

Make new replication slot test code less racy

commit   : 7099ba0580356bb455d75de4f6cd0faa046e1a10    
  
author   : Alvaro Herrera <[email protected]>    
date     : Sat, 17 Jul 2021 13:19:17 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Sat, 17 Jul 2021 13:19:17 -0400    

Click here for diff

The new test code added in ead9e51e8236 is racy -- it hinges on  
shared-memory state, which changes before the WARNING message is logged.  
Put it the other way around.  
  
Backpatch to 13.  
  
Author: Álvaro Herrera <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/test/recovery/t/019_replslot_limit.pl

Doc: document the current-transaction-modes GUCs.

commit   : 35e9d26a1bfe0d4fb726edc5c03b8e4fcc0a9869    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 17 Jul 2021 11:52:54 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 17 Jul 2021 11:52:54 -0400    

Click here for diff

We had documentation of default_transaction_isolation et al,  
but for some reason not of transaction_isolation et al.  
AFAICS this is just an ancient oversight, so repair.  
  
Per bug #17077 from Yanliang Lei.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/config.sgml
M doc/src/sgml/ref/set_transaction.sgml

Fix pg_dump for disabled triggers on partitioned tables

commit   : cc340af3345370c7faf0ceba7dece97b40312cb2    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 16 Jul 2021 17:29:22 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 16 Jul 2021 17:29:22 -0400    

Click here for diff

pg_dump failed to preserve the 'enabled' flag (which can be not only  
disabled, but also REPLICA or ALWAYS) for partitions which had it  
changed from their respective parents.  Attempt to handle that by  
including a definition for such triggers in the dump, but replace the  
standard CREATE TRIGGER line with an ALTER TRIGGER line.  
  
Backpatch to 11, where these triggers can exist.  In branches 11 and 12,  
pick up a few test lines from commit b9b408c48724 to verify that  
pg_upgrade is okay with these arrangements.  
  
Co-authored-by: Justin Pryzby <[email protected]>  
Co-authored-by: Álvaro Herrera <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
M src/bin/pg_dump/t/002_pg_dump.pl
M src/test/regress/expected/sanity_check.out
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql

Preserve firing-on state when cloning row triggers to partitions

commit   : c31516ae5b43d8a1a99e8e43abc84d791ce15ef1    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 16 Jul 2021 13:01:43 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 16 Jul 2021 13:01:43 -0400    

Click here for diff

When triggers are cloned from partitioned tables to their partitions,  
the 'tgenabled' flag (origin/replica/always/disable) was not propagated.  
Make it so that the flag on the trigger on partition is initially set to  
the same value as on the partitioned table.  
  
Add a test case to verify the behavior.  
  
Backpatch to 11, where this appeared in commit 86f575948c77.  
  
Author: Álvaro Herrera <[email protected]>  
Reported-by: Justin Pryzby <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/include/commands/trigger.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql

Advance old-segment horizon properly after slot invalidation

commit   : 866237a6fa01a128325df41ad39b41ea3363c9a9    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 16 Jul 2021 12:07:30 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 16 Jul 2021 12:07:30 -0400    

Click here for diff

When some slots are invalidated due to the max_slot_wal_keep_size limit,  
the old segment horizon should move forward to stay within the limit.  
However, in commit c6550776394e we forgot to call KeepLogSeg again to  
recompute the horizon after invalidating replication slots.  In cases  
where other slots remained, the limits would be recomputed eventually  
for other reasons, but if all slots were invalidated, the limits would  
not move at all afterwards.  Repair.  
  
Backpatch to 13 where the feature was introduced.  
  
Author: Kyotaro Horiguchi <[email protected]>  
Reported-by: Marcin Krupowicz <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/transam/xlog.c
M src/backend/replication/slot.c
M src/include/replication/slot.h
M src/test/recovery/t/019_replslot_limit.pl

Ensure HAVE_DECL_XXX macros in MSVC builds match those in Unix.

commit   : 91a1651057b15809058acb174fead131ef8efab8    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 15 Jul 2021 11:00:43 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 15 Jul 2021 11:00:43 -0400    

Click here for diff

Autoconf's AC_CHECK_DECLS() always defines HAVE_DECL_whatever  
as 1 or 0, but some of the entries in msvc/Solution.pm showed  
such symbols as "undef" instead of 0.  Fix that for consistency.  
There's no live bug in current usages AFAICS, but it's not hard  
to imagine one creeping in if more-complex #if tests get added.  
  
Back-patch to v13, which is as far back as Solution.pm contains  
this data.  The inconsistency still exists in the manually-filled  
pg_config_ext.h.win32 files of older branches; but as long as the  
problem is only latent, it doesn't seem worth the trouble to  
clean things up there.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/tools/msvc/Solution.pm

Fix unexpected error messages for various flavors of ALTER TABLE

commit   : 5226243459f11e0029c803674aabfb3bc25192f7    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 14 Jul 2021 17:15:18 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 14 Jul 2021 17:15:18 +0900    

Click here for diff

Some commands of ALTER TABLE could fail with the following error:  
ERROR:  "tab" is of the wrong type  
  
This error is unexpected, as all the code paths leading to  
ATWrongRelkindError() should use a supported set of relkinds to generate  
correct error messages.  This commit closes the gap with such mistakes,  
by adding all the missing relkind combinations.  Tests are added to  
check all the problems found.  Note that some combinations are not used,  
but these are left around as it could have an impact on applications  
relying on this code.  
  
2ed532e has done a much larger refactoring on HEAD to make such error  
messages easier to manage in the long-term, so nothing is needed there.  
  
Author: Kyotaro Horiguchi  
Reviewed-by: Peter Eisentraut, Ahsan Hadi, Michael Paquier  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 11  

M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/expected/foreign_data.out
M src/test/regress/sql/alter_table.sql
M src/test/regress/sql/foreign_data.sql

Robustify tuplesort's free_sort_tuple function

commit   : 2fde8e49a24706773d0573d3b8888a76353094fc    
  
author   : David Rowley <[email protected]>    
date     : Tue, 13 Jul 2021 13:28:19 +1200    
  
committer: David Rowley <[email protected]>    
date     : Tue, 13 Jul 2021 13:28:19 +1200    

Click here for diff

41469253e went to the trouble of removing a theoretical bug from  
free_sort_tuple by checking if the tuple was NULL before freeing it. Let's  
make this a little more robust by also setting the tuple to NULL so that  
should we be called again we won't end up doing a pfree on the already  
pfree'd tuple. Per advice from Tom Lane.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6, same as 41469253e  

M src/backend/utils/sort/tuplesort.c

Fix theoretical bug in tuplesort

commit   : 204f646a22f294a359ec463c128d01e1bc9117c5    
  
author   : David Rowley <[email protected]>    
date     : Tue, 13 Jul 2021 12:42:43 +1200    
  
committer: David Rowley <[email protected]>    
date     : Tue, 13 Jul 2021 12:42:43 +1200    

Click here for diff

This fixes a theoretical bug in tuplesort.c which, if a bounded sort was  
used in combination with a byval Datum sort (tuplesort_begin_datum), when  
switching the sort to a bounded heap in make_bounded_heap(), we'd call  
free_sort_tuple().  The problem was that when sorting Datums of a byval  
type, the tuple is NULL and free_sort_tuple() would free the memory for it  
regardless of that.  This would result in a crash.  
  
Here we fix that simply by adding a check to see if the tuple is NULL  
before trying to disassociate and free any memory belonging to it.  
  
The reason this bug is only theoretical is that nowhere in the current  
code base do we do tuplesort_set_bound() when performing a Datum sort.  
However, let's backpatch a fix for this as if any extension uses the code  
in this way then it's likely to cause problems.  
  
Author: Ronan Dunklau  
Discussion: https://postgr.es/m/CAApHDvpdoqNC5FjDb3KUTSMs5dg6f+XxH4Bg_dVcLi8UYAG3EQ@mail.gmail.com  
Backpatch-through: 9.6, oldest supported version  

M src/backend/utils/sort/tuplesort.c

doc: Fix typo in function prototype

commit   : 0455f35fd4ecff99df49ee7137a53ba2b6b3c24b    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 12 Jul 2021 22:07:35 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 12 Jul 2021 22:07:35 +0200    

Click here for diff

M doc/src/sgml/indexam.sgml

Remove dead assignment to local variable.

commit   : 2c27cc26592a3f4f6ad1300d976073a256c57c9b    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Mon, 12 Jul 2021 11:13:33 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Mon, 12 Jul 2021 11:13:33 +0300    

Click here for diff

This should have been removed in commit 7e30c186da, which split the loop  
into two. Only the first loop uses the 'from' variable; updating it in  
the second loop is bogus. It was never read after the first loop, so this  
was harmless and surely optimized away by the compiler, but let's be tidy.  
  
Backpatch to all supported versions.  
  
Author: Ranier Vilela  
Discussion: https://www.postgresql.org/message-id/CAEudQAoWq%2BAL3BnELHu7gms2GN07k-np6yLbukGaxJ1vY-zeiQ%40mail.gmail.com  

M src/backend/access/nbtree/nbtxlog.c

Lock the extension during ALTER EXTENSION ADD/DROP.

commit   : 1c612bc98ec363e2464cce66526b75a0d08d657e    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 11 Jul 2021 12:54:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 11 Jul 2021 12:54:24 -0400    

Click here for diff

Although we were careful to lock the object being added or dropped,  
we failed to get any sort of lock on the extension itself.  This  
allowed the ALTER to proceed in parallel with a DROP EXTENSION,  
which is problematic for a couple of reasons.  If both commands  
succeeded we'd be left with a dangling link in pg_depend, which  
would cause problems later.  Also, if the ALTER failed for some  
reason, it might try to print the extension's name, and that could  
result in a crash or (in older branches) a silly error message  
complaining about extension "(null)".  
  
Per bug #17098 from Alexander Lakhin.  Back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/extension.c

Fix assign_record_type_typmod().

commit   : edd9a2bf741be95aa771b9fcacb2ce7378fb9121    
  
author   : Jeff Davis <[email protected]>    
date     : Fri, 9 Jul 2021 14:15:48 -0700    
  
committer: Jeff Davis <[email protected]>    
date     : Fri, 9 Jul 2021 14:15:48 -0700    

Click here for diff

If an error occurred in the wrong place, it was possible to leave an  
unintialized entry in the hash table, leading to a crash. Fixed.  
  
Also, be more careful about the order of operations so that an  
allocation error doesn't leak memory in CacheMemoryContext or  
unnecessarily advance NextRecordTypmod.  
  
Backpatch through version 11. Earlier versions (prior to 35ea75632a5)  
do not exhibit the problem, because an uninitialized hash entry  
contains a valid empty list.  
  
Author: Sait Talha Nisanci <[email protected]>  
Reviewed-by: Andres Freund  
Discussion: https://postgr.es/m/HE1PR8303MB009069D476225B9A9E194B8891779@HE1PR8303MB0090.EURPRD83.prod.outlook.com  
Backpatch-through: 11  

M src/backend/utils/cache/typcache.c

Fix busted test for ldap_initialize.

commit   : 9fca23c1d6de2664aa447e4ef72b9bf60d3bdf35    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 10 Jul 2021 13:19:31 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 10 Jul 2021 13:19:31 -0400    

Click here for diff

Sigh ... I was expecting AC_CHECK_LIB to do something it didn't,  
namely update LIBS.  This led to not finding ldap_initialize.  
Fix by moving the probe for ldap_initialize.  In some sense this  
is more correct anyway, since (at least for now) we care about  
whether ldap_initialize exists in libldap not libldap_r.  
  
Per buildfarm member elver and local testing.  
  
Discussion: https://postgr.es/m/[email protected]  

M configure
M configure.in

Fix numeric_mul() overflow due to too many digits after decimal point.

commit   : f23a9b8a49af1ef89dd65399867ca21318ae6b8e    
  
author   : Dean Rasheed <[email protected]>    
date     : Sat, 10 Jul 2021 12:46:13 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Sat, 10 Jul 2021 12:46:13 +0100    

Click here for diff

This fixes an overflow error when using the numeric * operator if the  
result has more than 16383 digits after the decimal point by rounding  
the result. Overflow errors should only occur if the result has too  
many digits *before* the decimal point.  
  
Discussion: https://postgr.es/m/CAEZATCUmeFWCrq2dNzZpRj5+6LfN85jYiDoqm+ucSXhb9U2TbA@mail.gmail.com  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql

Un-break AIX build, take 2.

commit   : 32d0bdbfcf2f1f024f24797eda14a250c13fa62e    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 16:59:07 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 16:59:07 -0400    

Click here for diff

I incorrectly diagnosed the reason why hoverfly is unhappy.  
Looking closer, it appears that it fails to link libldap  
unless libssl is also present; so the problem was my  
idea of clearing LIBS before making the check.  Revert  
to essentially the original coding, except that instead  
of failing when libldap_r isn't there, use libldap.  
  
Per buildfarm member hoverfly.  
  
Discussion: https://postgr.es/m/[email protected]  

M configure
M configure.in

Un-break AIX build.

commit   : cbcf5ffb180ad18fcebe8b21ab1ec1e90d3a3b14    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 14:15:41 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 14:15:41 -0400    

Click here for diff

In commit d0a02bdb8, I'd supposed that uniformly probing for  
ldap_bind would make the intent clearer.  However, that seems  
not to work on AIX, for obscure reasons (maybe it's a macro  
there?).  Revert to the former behavior of probing  
ldap_simple_bind for thread-safe cases and ldap_bind otherwise.  
  
Per buildfarm member hoverfly.  
  
Discussion: https://postgr.es/m/[email protected]  

M configure
M configure.in

Avoid creating a RESULT RTE that's marked LATERAL.

commit   : 9807b9aedc6e8b8a497db161b3d2d9f83648ea5b    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 13:38:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 13:38:24 -0400    

Click here for diff

Commit 7266d0997 added code to pull up simple constant function  
results, converting the RTE_FUNCTION RTE to a dummy RTE_RESULT  
RTE since it no longer need be scanned.  But I forgot to clear  
the LATERAL flag if the RTE has it set.  If the function reduced  
to a constant, it surely contains no lateral references so this  
simplification is logically OK.  It's needed because various other  
places will Assert that RESULT RTEs aren't LATERAL.  
  
Per bug #17097 from Yaoguang Chen.  Back-patch to v13 where the  
faulty code came in.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql

Update configure's probe for libldap to work with OpenLDAP 2.5.

commit   : 55cccdfdf1ee63fd8e818c2aa885f4d357773adc    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 12:38:55 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 12:38:55 -0400    

Click here for diff

The separate libldap_r is gone and libldap itself is now always  
thread-safe.  Unfortunately there seems no easy way to tell by  
inspection whether libldap is thread-safe, so we have to take  
it on faith that libldap is thread-safe if there's no libldap_r.  
That should be okay, as it appears that libldap_r was a standard  
part of the installation going back at least 20 years.  
  
Report and patch by Adrian Ho.  Back-patch to all supported  
branches, since people might try to build any of them with  
a newer OpenLDAP.  
  
Discussion: https://postgr.es/m/[email protected]  

M configure
M configure.in
M src/include/pg_config.h.in
M src/tools/msvc/Solution.pm

Reject cases where a query in WITH rewrites to just NOTIFY.

commit   : 6edccac166fa55065e88dcc82fc5ed6751485c42    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 11:02:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 9 Jul 2021 11:02:26 -0400    

Click here for diff

Since the executor can't cope with a utility statement appearing  
as a node of a plan tree, we can't support cases where a rewrite  
rule inserts a NOTIFY into an INSERT/UPDATE/DELETE command appearing  
in a WITH clause of a larger query.  (One can imagine ways around  
that, but it'd be a new feature not a bug fix, and so far there's  
been no demand for it.)  RewriteQuery checked for this, but it  
missed the case where the DML command rewrites to *only* a NOTIFY.  
That'd lead to crashes later on in planning.  Add the missed check,  
and improve the level of testing of this area.  
  
Per bug #17094 from Yaoguang Chen.  It's been busted since WITH  
was introduced, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/rewrite/rewriteHandler.c
M src/test/regress/expected/with.out
M src/test/regress/sql/with.sql

Remove more obsolete comments about semaphores.

commit   : 7dd2379df7099a13b04f3977dd78b498cdc437e5    
  
author   : Thomas Munro <[email protected]>    
date     : Fri, 9 Jul 2021 17:51:48 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Fri, 9 Jul 2021 17:51:48 +1200    

Click here for diff

Commit 6753333f stopped using semaphores as the sleep/wake mechanism for  
heavyweight locks, but some obsolete references to that scheme remained  
in comments.  As with similar commit 25b93a29, back-patch all the way.  
  
Reviewed-by: Daniel Gustafsson <[email protected]>  
Discussion: https://postgr.es/m/CA%2BhUKGLafjB1uzXcy%3D%3D2L3cy7rjHkqOVn7qRYGBjk%3D%3DtMJE7Yg%40mail.gmail.com  

M src/backend/storage/lmgr/lwlock.c

Add missing Int64GetDatum macro in dbsize.c

commit   : 87103002c89a80411a96068da2d643d0bc2804e9    
  
author   : David Rowley <[email protected]>    
date     : Fri, 9 Jul 2021 15:12:31 +1200    
  
committer: David Rowley <[email protected]>    
date     : Fri, 9 Jul 2021 15:12:31 +1200    

Click here for diff

I accidentally missed adding this when adjusting 55fe60938 for back  
patching.  This adjustment was made for 9.6 to 13. 14 and master are not  
affected.  
  
Discussion: https://postgr.es/m/CAApHDvp=twCsGAGQG=A=cqOaj4mpknPBW-EZB-sd+5ZS5gCTtA@mail.gmail.com  

M src/backend/utils/adt/dbsize.c

Fix incorrect return value in pg_size_pretty(bigint)

commit   : 6f88b68ff4adfe17a80c0befdf9bb8b0a8218def    
  
author   : David Rowley <[email protected]>    
date     : Fri, 9 Jul 2021 14:04:49 +1200    
  
committer: David Rowley <[email protected]>    
date     : Fri, 9 Jul 2021 14:04:49 +1200    

Click here for diff

Due to how pg_size_pretty(bigint) was implemented, it's possible that when  
given a negative number of bytes that the returning value would not match  
the equivalent positive return value when given the equivalent positive  
number of bytes.  This was due to two separate issues.  
  
1. The function used bit shifting to convert the number of bytes into  
larger units.  The rounding performed by bit shifting is not the same as  
dividing.  For example -3 >> 1 = -2, but -3 / 2 = -1.  These two  
operations are only equivalent with positive numbers.  
  
2. The half_rounded() macro rounded towards positive infinity.  This meant  
that negative numbers rounded towards zero and positive numbers rounded  
away from zero.  
  
Here we fix #1 by dividing the values instead of bit shifting.  We fix #2  
by adjusting the half_rounded macro always to round away from zero.  
  
Additionally, adjust the pg_size_pretty(numeric) function to be more  
explicit that it's using division rather than bit shifting.  A casual  
observer might have believed bit shifting was used due to a static  
function being named numeric_shift_right.  However, that function was  
calculating the divisor from the number of bits and performed division.  
Here we make that more clear.  This change is just cosmetic and does not  
affect the return value of the numeric version of the function.  
  
Here we also add a set of regression tests both versions of  
pg_size_pretty() which test the values directly before and after the  
function switches to the next unit.  
  
This bug was introduced in 8a1fab36a. Prior to that negative values were  
always displayed in bytes.  
  
Author: Dean Rasheed, David Rowley  
Discussion: https://postgr.es/m/CAEZATCXnNW4HsmZnxhfezR5FuiGgp+mkY4AzcL5eRGO4fuadWg@mail.gmail.com  
Backpatch-through: 9.6, where the bug was introduced.  

M src/backend/utils/adt/dbsize.c
M src/test/regress/expected/dbsize.out
M src/test/regress/sql/dbsize.sql

doc: Fix description about pg_stat_statements.track_planning.

commit   : 306c5e05e20fe37fcfe7dcde3ce4677b05a25285    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 7 Jul 2021 21:54:47 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 7 Jul 2021 21:54:47 +0900    

Click here for diff

This commit fixes wrong wording like "a fewer kinds"  
in the description about track_planning option.  
  
Back-patch to v13 where pg_stat_statements.track_planning was added.  
  
Author: Justin Pryzby  
Reviewed-by: Julien Rouhaud, Fujii Masao  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/pgstatstatements.sgml

Avoid doing catalog lookups in postgres_fdw's conversion_error_callback.

commit   : bee18616a6508d054da65902bfded41117113cb3    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 6 Jul 2021 12:36:13 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 6 Jul 2021 12:36:13 -0400    

Click here for diff

As in 50371df26, this is a bad idea since the callback can't really  
know what error is being thrown and thus whether or not it is safe  
to attempt catalog accesses.  Rather than pushing said accesses into  
the mainline code where they'd usually be a waste of cycles, we can  
look at the query's rangetable instead.  
  
This change does mean that we'll be printing query aliases (if any  
were used) rather than the table or column's true name.  But that  
doesn't seem like a bad thing: it's certainly a more useful definition  
in self-join cases, for instance.  In any case, it seems unlikely that  
any applications would be depending on this detail, so it seems safe  
to change.  
  
Patch by me.  Original complaint by Andres Freund; Bharath Rupireddy  
noted the connection to conversion_error_callback.  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/postgres_fdw.c
M contrib/postgres_fdw/sql/postgres_fdw.sql

Doc: add info about timestamps with fractional-minute UTC offsets.

commit   : fa44348105983fb8a20a1eb145ca036e77807ba0    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 6 Jul 2021 10:34:51 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 6 Jul 2021 10:34:51 -0400    

Click here for diff

Our code has supported fractional-minute UTC offsets for ages, but  
there was no mention of the possibility in the main docs, and only  
a very indirect reference in Appendix B.  Improve that.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/datatype.sgml

Reduce overhead of cache-clobber testing in LookupOpclassInfo().

commit   : 2f487116e9988615d05a6c380c3ff8fa8a4286f8    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 5 Jul 2021 16:51:57 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 5 Jul 2021 16:51:57 -0400    

Click here for diff

Commit 03ffc4d6d added logic to bypass all caching behavior in  
LookupOpclassInfo when CLOBBER_CACHE_ALWAYS is enabled.  It doesn't  
look like I stopped to think much about what that would cost, but  
recent investigation shows that the cost is enormous: it roughly  
doubles the time needed for cache-clobber test runs.  
  
There does seem to be value in this behavior when trying to test  
the opclass-cache loading logic itself, but for other purposes the  
cost is excessive.  Hence, let's back off to doing this only when  
debug_invalidate_system_caches_always is at least 3; or in older  
branches, when CLOBBER_CACHE_RECURSIVELY is defined.  
  
While here, clean up some other minor issues in LookupOpclassInfo.  
Re-order the code so we aren't left with broken cache entries (leading  
to later core dumps) in the unlikely case that we suffer OOM while  
trying to allocate space for a new entry.  (That seems to be my  
oversight in 03ffc4d6d.)  Also, in >= v13, stop allocating one array  
entry too many.  That's evidently left over from sloppy reversion in  
851b14b0c.  
  
Back-patch to all supported branches, mainly to reduce the runtime  
of cache-clobbering buildfarm animals.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/cache/relcache.c

Doc: Hash Indexes.

commit   : 27621cc55526fc5acb58bc6c023e79fba3a92672    
  
author   : Amit Kapila <[email protected]>    
date     : Mon, 5 Jul 2021 09:52:05 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Mon, 5 Jul 2021 09:52:05 +0530    

Click here for diff

A new chapter for Hash Indexes, designed to help users understand how  
they work and when to use them.  
  
Backpatch-through 10 where we have made hash indexes durable.  
  
Author: Simon Riggs  
Reviewed-By: Justin Pryzby, Amit Kapila  
Discussion: https://postgr.es/m/CANbhV-HRjNPYgHo--P1ewBrFJ-GpZPb9_25P7=Wgu7s7hy_sLQ@mail.gmail.com  

M doc/src/sgml/filelist.sgml
A doc/src/sgml/hash.sgml
M doc/src/sgml/postgres.sgml

doc: Mention requirement to --enable-tap-tests on section for TAP tests

commit   : 39a21ce06baa6734d52566c300aa55c2934f94de    
  
author   : Michael Paquier <[email protected]>    
date     : Sun, 4 Jul 2021 20:59:14 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sun, 4 Jul 2021 20:59:14 +0900    

Click here for diff

Author: Greg Sabino Mullane  
Discussion: https://postgr.es/m/CAKAnmmJYH2FBn_+Vwd2FD5SaKn8hjhAXOCHpZc6n4wXaUaW_SA@mail.gmail.com  
Backpatch-through: 9.6  

M doc/src/sgml/regress.sgml

Doc: mention that VACUUM can't utilize over 1GB of RAM

commit   : 409d9390e8e944eab2ea4b98984cc5781d70e7d1    
  
author   : David Rowley <[email protected]>    
date     : Sun, 4 Jul 2021 22:29:57 +1200    
  
committer: David Rowley <[email protected]>    
date     : Sun, 4 Jul 2021 22:29:57 +1200    

Click here for diff

Document that setting maintenance_work_mem to values over 1GB has no  
effect on VACUUM.  
  
Reported-by: Martín Marqués  
Author: Laurenz Albe  
Discussion: https://postgr.es/m/CABeG9LsZ2ozUMcqtqWu_-GiFKB17ih3p8wBHXcpfnHqhCnsc7A%40mail.gmail.com  
Backpatch-through: 9.6, oldest supported release  

M doc/src/sgml/config.sgml

doc: adjust "cities" example to be consistent with other SQL

commit   : 650e6359018a54287d99fcd9640fb24fa7209467    
  
author   : Bruce Momjian <[email protected]>    
date     : Fri, 2 Jul 2021 20:42:46 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Fri, 2 Jul 2021 20:42:46 -0400    

Click here for diff

Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.6  

M doc/src/sgml/advanced.sgml

Don't try to print data type names in slot_store_error_callback().

commit   : 7fc97752d566735cdab17a4cc4ac3305413ee54e    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 2 Jul 2021 16:04:54 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 2 Jul 2021 16:04:54 -0400    

Click here for diff

The existing code tried to do syscache lookups in an already-failed  
transaction, which is problematic to say the least.  After some  
consideration of alternatives, the best fix seems to be to just drop  
type names from the error message altogether.  The table and column  
names seem like sufficient localization.  If the user is unsure what  
types are involved, she can check the local and remote table  
definitions.  
  
Having done that, we can also discard the LogicalRepTypMap hash  
table, which had no other use.  Arguably, LOGICAL_REP_MSG_TYPE  
replication messages are now obsolete as well; but we should  
probably keep them in case some other use emerges.  (The complexity  
of removing something from the replication protocol would likely  
outweigh any savings anyhow.)  
  
Masahiko Sawada and Bharath Rupireddy, per complaint from Andres  
Freund.  Back-patch to v10 where this code originated.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/include/replication/logicalrelation.h

add missing tag from commit b8c4261e5e

commit   : 8d2be1402833a8cf0876282165bc8eae224a9c2e    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 1 Jul 2021 15:38:06 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 1 Jul 2021 15:38:06 -0400    

Click here for diff

M GNUmakefile.in
M doc/src/sgml/installation.sgml

Add new make targets world-bin and install-world-bin

commit   : bd0be7f7a42fa84fd9fb770117c9f7c1238a05b1    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 1 Jul 2021 14:21:09 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 1 Jul 2021 14:21:09 -0400    

Click here for diff

These are the same as world and install-world respectively, but without  
building or installing the documentation. There are many reasons for  
wanting to be able to do this, including speed, lack of documentation  
building tools, and wanting to build other formats of the documentation.  
Plans for simplifying the buildfarm client code include using these  
targets.  
  
Backpatch to all live branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M GNUmakefile.in
M doc/src/sgml/installation.sgml

Fix prove_installcheck to use correct paths when used with PGXS

commit   : a8b564b0c9c39e8bdd6cde4245c3340f72c28899    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 1 Jul 2021 08:29:10 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 1 Jul 2021 08:29:10 -0400    

Click here for diff

The prove_installcheck recipe in src/Makefile.global.in was emitting  
bogus paths for a couple of elements when used with PGXS. Here we create  
a separate recipe for the PGXS case that does it correctly. We also take  
the opportunity to make the make the file more readable by breaking up  
the prove_installcheck and prove_check recipes across several lines, and  
to remove the setting for REGRESS_SHLIB to src/test/recovery/Makefile,  
which is the only set of tests that actually need it.  
  
Backpatch to all live branches  
  
Discussion: https://postgr.es/m/[email protected]  

M src/Makefile.global.in
M src/test/recovery/Makefile

Fix incorrect PITR message for transaction ROLLBACK PREPARED

commit   : 41edb2db1b9161fe5c100d3806e08c63a9f89196    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 30 Jun 2021 11:49:16 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 30 Jun 2021 11:49:16 +0900    

Click here for diff

Reaching PITR on such a transaction would cause the generation of a LOG  
message mentioning a transaction committed, not aborted.  
  
Oversight in 4f1b890.  
  
Author: Simon Riggs  
Discussion: https://postgr.es/m/CANbhV-GJ6KijeCgdOrxqMCQ+C8QiK657EMhCy4csjrPcEUFv_Q@mail.gmail.com  
Backpatch-through: 9.6  

M src/backend/access/transam/xlog.c

Don't use abort(3) in libpq's fe-print.c.

commit   : 1603deca34ef403fc40d471ba6512d9f170ba7b5    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 28 Jun 2021 14:17:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 28 Jun 2021 14:17:42 -0400    

Click here for diff

Causing a core dump on out-of-memory seems pretty unfriendly,  
and surely is far outside the expected behavior of a general-purpose  
library.  Just print an error message (as we did already) and return.  
These functions unfortunately don't have an error return convention,  
but code using them is probably just looking for a quick-n-dirty  
print method and wouldn't bother to check anyway.  
  
Although these functions are semi-deprecated, it still seems  
appropriate to back-patch this.  In passing, also back-patch  
b90e6cef1, just to reduce cosmetic differences between the  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/interfaces/libpq/fe-print.c

Don't depend on -fwrapv semantics in pgbench's random() function.

commit   : be567deb3ac952e41edc81b8c08a0dc6115c5f62    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 28 Jun 2021 12:40:37 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 28 Jun 2021 12:40:37 -0400    

Click here for diff

Instead use the common/int.h functions to check for integer overflow  
in a more C-standard-compliant fashion.  This is motivated by recent  
failures on buildfarm member moonjelly, where it appears that  
development-tip gcc is optimizing without regard to the -fwrapv  
switch.  Presumably that's a gcc bug that will be fixed soon, but  
we might as well install cleaner coding here rather than wait.  
  
(This does not address the question of whether we'll ever be able  
to get rid of using -fwrapv.  Testing shows that this spot is the  
only place where doing so creates visible regression test failures,  
but unfortunately that proves very little.)  
  
Back-patch to v12.  The common/int.h functions exist in v11, but  
that branch doesn't use them in any client-side code.  I judge  
that this case isn't interesting enough in the real world to take  
even a small risk of issues from being the first such use.  
  
Tom Lane and Fabien Coelho  
  
Discussion: https://postgr.es/m/[email protected]  

M src/bin/pgbench/pgbench.c

Fix race condition in TransactionGroupUpdateXidStatus().

commit   : 741deb2603c758c67f199c9129d2e92bd80cec2e    
  
author   : Amit Kapila <[email protected]>    
date     : Mon, 28 Jun 2021 08:42:48 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Mon, 28 Jun 2021 08:42:48 +0530    

Click here for diff

When we cannot immediately acquire XactSLRULock in exclusive mode at  
commit time, we add ourselves to a list of processes that need their XIDs  
status update. We do this if the clog page where we need to update the  
current transaction status is the same as the group leader's clog page,  
otherwise, we allow the caller to clear it by itself. Now, when we can't  
add ourselves to any group, we were not clearing the current proc if it  
has already become a member of some group which was leading to an  
assertion failure when the same proc was assigned to another backend after  
the current backend exits.  
  
Reported-by: Alexander Lakhin  
Bug: 17072  
Author: Amit Kapila  
Tested-By: Alexander Lakhin  
Backpatch-through: 11, where it was introduced  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/transam/clog.c

Add test for CREATE INDEX CONCURRENTLY with not-so-immutable predicate

commit   : fd7bc10ab4c6a88846d1b8a35d16cfe0b8f5a487    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 28 Jun 2021 11:17:12 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 28 Jun 2021 11:17:12 +0900    

Click here for diff

83158f7 has improved index_set_state_flags() so as it is possible to use  
transactional updates when updating pg_index state flags, but there was  
not really a test case which stressed directly the possibility it fixed.  
This commit adds such a test, using a predicate that looks valid in  
appearance but calls a stable function.  
  
Author: Andrey Lepikhov  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql

Make index_set_state_flags() transactional

commit   : acb60edf0254e2b0f0be753c31c877daf4012582    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 28 Jun 2021 10:39:09 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 28 Jun 2021 10:39:09 +0900    

Click here for diff

3c84046 is the original commit that introduced index_set_state_flags(),  
where the presence of SnapshotNow made necessary the use of an in-place  
update.  SnapshotNow has been removed in 813fb03, so there is no actual  
reasons to not make this operation transactional.  
  
As reported by Andrey, it is possible to trigger the assertion of this  
routine expecting no transactional updates when switching the pg_index  
state flags, using a predicate mark as immutable but calling stable or  
volatile functions.  83158f7 has been around for a couple of months on  
HEAD now with no issues found related to it, so it looks safe enough for  
a backpatch.  
  
Reported-by: Andrey Lepikhov  
Author: Michael Paquier  
Reviewed-by: Anastasia Lubennikova  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/backend/catalog/index.c

Remove memory leaks in isolationtester.

commit   : 2d094486541acd8baee3af7579d4606f78256165    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 27 Jun 2021 12:45:04 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 27 Jun 2021 12:45:04 -0400    

Click here for diff

specscanner.l leaked a kilobyte of memory per token of the spec file.  
Apparently somebody thought that the introductory code block would be  
executed once; but it's once per yylex() call.  
  
A couple of functions in isolationtester.c leaked small amounts of  
memory due to not bothering to free one-time allocations.  Might  
as well improve these so that valgrind gives this program a clean  
bill of health.  Also get rid of an ugly static variable.  
  
Coverity complained about one of the one-time leaks, which led me  
to try valgrind'ing isolationtester, which led to discovery of the  
larger leak.  

M src/test/isolation/isolationtester.c
M src/test/isolation/specscanner.l

Remove non-existing variable reference in MSVC's Solution.pm

commit   : 88274a7a314eb521b1ee924f2872872fce289fdf    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 26 Jun 2021 13:52:54 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 26 Jun 2021 13:52:54 +0900    

Click here for diff

The version string is grabbed from PACKAGE_VERSION in pg_config.h in the  
MSVC build since 8f4fb4c6, but an error message referenced a variable  
that existed before that.  This had no consequences except if one messes  
up enough with the version number of the build.  
  
Author: Anton Voloshin  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 13  

M src/tools/msvc/Solution.pm

Remove some useless logs from the TAP tests of pgbench

commit   : 0455b7ccce3cba6a85a85aa6524c48d58eb1c93e    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 26 Jun 2021 12:40:03 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 26 Jun 2021 12:40:03 +0900    

Click here for diff

002_pgbench_no_server was printing some array pointers instead of the  
actual contents of those arrays for the expected outputs of stdout and  
stderr for a tested command.  This does not add any new information that  
can help with debugging as the test names allow to track failure  
locations, if any.  
  
This commit simply removes those logs as the rest of the printed  
information is redundant with command_checks_all().  
  
Per discussion with Andrew Dunstan and Álvaro Herrera.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 11  

M src/bin/pgbench/t/002_pgbench_no_server.pl

Remove unnecessary failure cases in RemoveRoleFromObjectPolicy().

commit   : ba815f00a0ce4c9b7696be6d841733b1b481a1f8    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 25 Jun 2021 13:59:38 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 25 Jun 2021 13:59:38 -0400    

Click here for diff

It's not really necessary for this function to open or lock the  
relation associated with the pg_policy entry it's modifying.  The  
error checks it's making on the rel are if anything counterproductive  
(e.g., if we don't want to allow installation of policies on system  
catalogs, here is not the place to prevent that).  In particular, it  
seems just wrong to insist on an ownership check.  That has the net  
effect of forcing people to use superuser for DROP OWNED BY, which  
surely is not an effect we want.  Also there is no point in rebuilding  
the dependencies of the policy expressions, which aren't being  
changed.  Lastly, locking the table also seems counterproductive; it's  
not helping to prevent race conditions, since we failed to re-read the  
pg_policy row after acquiring the lock.  That means that concurrent  
DDL would likely result in "tuple concurrently updated/deleted"  
errors; which is the same behavior this code will produce, with less  
overhead.  
  
Per discussion of bug #17062.  Back-patch to all supported versions,  
as the failure cases this eliminates seem just as undesirable in 9.6  
as in HEAD.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/policy.c

Make walsenders show their replication commands in pg_stat_activity.

commit   : 4a20de9d9ed3e45e41cd0860573387e95132314d    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 25 Jun 2021 10:46:10 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 25 Jun 2021 10:46:10 -0400    

Click here for diff

A walsender process that has executed a SQL command left the text of  
that command in pg_stat_activity.query indefinitely, which is quite  
confusing if it's in RUNNING state but not doing that query.  An easy  
and useful fix is to treat replication commands as if they were SQL  
queries, and show them in pg_stat_activity according to the same rules  
as for regular queries.  While we're at it, it seems also sensible to  
set debug_query_string, allowing error logging and debugging to see  
the replication command.  
  
While here, clean up assorted silliness in exec_replication_command:  
* Clean up SQLCmd code path, and fix its only-accidentally-not-buggy  
  memory management.  
* Remove useless duplicate call of SnapBuildClearExportedSnapshot().  
* replication_scanner_finish() was never called.  
  
Back-patch of commit f560209c6 into v10-v13.  I'd originally felt  
that this didn't merit back-patching, but subsequent confusion  
while debugging walsender problems suggests that it'll be useful.  
Also, the original commit has now aged long enough to provide some  
comfort that it won't cause problems.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/walsender.c

commit   : af2e67b47afe56627c6ced8cd5741d345fc64da9    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 25 Jun 2021 20:15:31 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 25 Jun 2021 20:15:31 +0900    

Click here for diff

This fixes a couple of problems within the so-said code of this commit  
subject:  
- Replace the use of open() with slurp_file(), fixing an issue reported  
by buildfarm member fairywren whose perl installation keep around CRLF  
characters, causing the matching patterns for the logs to fail.  
- Remove the eval block, which is not really necessary.  
  
This set of issues has come into light after fixing a different issue  
with c13585fe, and this is wrong since this code has been introduced.  
  
Reported-by: Andrew Dunstan, and buildfarm member fairywren  
Author: Michael Paquier  
Reviewed-by: Andrew Dunstan  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 11  

M src/bin/pgbench/t/001_pgbench_with_server.pl

doc: Change reloption data type spelling for consistency

commit   : 372a2806ebc28f79de929a7c763b441e35a9c362    
  
author   : Peter Eisentraut <[email protected]>    
date     : Fri, 25 Jun 2021 08:11:10 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Fri, 25 Jun 2021 08:11:10 +0200    

Click here for diff

Use "floating point" rather than "float4", like everywhere else in  
this context.  
  
Author: [email protected]  
Discussion: https://www.postgresql.org/message-id/flat/TYAPR01MB28965989AF84B57FC351B97BC40F9@TYAPR01MB2896.jpnprd01.prod.outlook.com  

M doc/src/sgml/ref/create_table.sgml

Prepare for forthcoming LLVM 13 API change.

commit   : d9c05a9ec49399a45d30acacac748625922cbe40    
  
author   : Thomas Munro <[email protected]>    
date     : Fri, 25 Jun 2021 09:55:26 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Fri, 25 Jun 2021 09:55:26 +1200    

Click here for diff

LLVM 13 (due out in September) has changed the semantics of  
LLVMOrcAbsoluteSymbols(), so we need to bump some reference counts to  
avoid a double-free that causes crashes and bad query results.  
  
A proactive change seems necessary to avoid having a window of time  
where our respective latest releases would interact badly.  It's  
possible that the situation could change before then, though.  
  
Thanks to Fabien Coelho for monitoring bleeding edge LLVM and Andres  
Freund for tracking down the change.  
  
Back-patch to 11, where the JIT code arrived.  
  
Discussion: https://postgr.es/m/CA%2BhUKGLEy8mgtN7BNp0ooFAjUedDTJj5dME7NxLU-m91b85siA%40mail.gmail.com  

M src/backend/jit/llvm/llvmjit.c

Fix pattern matching logic for logs in TAP tests of pgbench

commit   : 7a9eaf111ac0a3ceba24fe15e5402bb2a17891f5    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 25 Jun 2021 06:52:47 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 25 Jun 2021 06:52:47 +0900    

Click here for diff

The logic checking for the format of per-thread logs used grep() with  
directly "$re", which would cause the test to consider all the logs as  
a match without caring about their format at all.  Using "/$re/" makes  
grep() perform a regex test, which is what we want here.  
  
While on it, improve some of the tests to be more picky with the  
patterns expected and add more comments to describe the tests.  
  
Issue discovered while digging into a separate patch.  
  
Author: Fabien Coelho, Michael Paquier  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 11  

M src/bin/pgbench/t/001_pgbench_with_server.pl

Fix ABI break introduced by commit 4daa140a2f.

commit   : 56e366f6757d24b07cc7c2de2e5151dc372d059e    
  
author   : Amit Kapila <[email protected]>    
date     : Thu, 24 Jun 2021 14:38:45 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Thu, 24 Jun 2021 14:38:45 +0530    

Click here for diff

Move the newly defined enum value REORDER_BUFFER_CHANGE_INTERNAL_SPEC_ABORT  
at the end to avoid ABI break in the back branches. We need to back-patch  
this till v11 because before that it is already at the end.  
  
Reported-by: Tomas Vondra  
Backpatch-through: 11  
Discussion: https://postgr.es/m/CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com  

M src/include/replication/reorderbuffer.h

Another fix to relmapper race condition.

commit   : 6fb377e5f74b09ac0b36034da65b179c567228b8    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 24 Jun 2021 11:19:03 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 24 Jun 2021 11:19:03 +0300    

Click here for diff

In previous commit, I missed that relmap_redo() was also not acquiring the  
RelationMappingLock. Thanks to Thomas Munro for pointing that out.  
  
Backpatch-through: 9.6, like previous commit.  
Discussion: https://www.postgresql.org/message-id/CA%2BhUKGLev%3DPpOSaL3WRZgOvgk217et%2BbxeJcRr4eR-NttP1F6Q%40mail.gmail.com  

M src/backend/utils/cache/relmapper.c

Prevent race condition while reading relmapper file.

commit   : 2a0ab13f8dd1346d03b00a737734aeca036514f9    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 24 Jun 2021 10:45:23 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 24 Jun 2021 10:45:23 +0300    

Click here for diff

Contrary to the comment here, POSIX does not guarantee atomicity of a  
read(), if another process calls write() concurrently. Or at least Linux  
does not. Add locking to load_relmap_file() to avoid the race condition.  
  
Fixes bug #17064. Thanks to Alexander Lakhin for the report and test case.  
  
Backpatch-through: 9.6, all supported versions.  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M src/backend/utils/cache/relmapper.c

Doc: Update caveats in synchronous logical replication.

commit   : 7a4ecefe9d77b7bb09caa5d82de433494e017363    
  
author   : Amit Kapila <[email protected]>    
date     : Thu, 24 Jun 2021 09:31:51 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Thu, 24 Jun 2021 09:31:51 +0530    

Click here for diff

Reported-by: Simon Riggs  
Author: Takamichi Osumi  
Reviewed-by: Amit Kapila  
Backpatch-through: 9.6  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M doc/src/sgml/logicaldecoding.sgml

Allow non-quoted identifiers as isolation test session/step names.

commit   : 5179a1ab773c9305192b0261fbb1f9e223d94a3d    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 18:41:39 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 18:41:39 -0400    

Click here for diff

For no obvious reason, isolationtester has always insisted that  
session and step names be written with double quotes.  This is  
fairly tedious and does little for test readability, especially  
since the names that people actually choose almost always look  
like normal identifiers.  Hence, let's tweak the lexer to allow  
SQL-like identifiers not only double-quoted strings.  
  
(They're SQL-like, not exactly SQL, because I didn't add any  
case-folding logic.  Also there's no provision for U&"..." names,  
not that anyone's likely to care.)  
  
There is one incompatibility introduced by this change: if you write  
"foo""bar" with no space, that used to be taken as two identifiers,  
but now it's just one identifier with an embedded quote mark.  
  
I converted all the src/test/isolation/ specfiles to remove  
unnecessary double quotes, but stopped there because my  
eyes were glazing over already.  
  
Like 741d7f104, back-patch to all supported branches, so that this  
isn't a stumbling block for back-patching isolation test changes.  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/test_decoding/specs/oldest_xmin.spec
M src/test/isolation/README
M src/test/isolation/specparse.y
M src/test/isolation/specs/aborted-keyrevoke.spec
M src/test/isolation/specs/alter-table-1.spec
M src/test/isolation/specs/alter-table-2.spec
M src/test/isolation/specs/alter-table-3.spec
M src/test/isolation/specs/alter-table-4.spec
M src/test/isolation/specs/async-notify.spec
M src/test/isolation/specs/classroom-scheduling.spec
M src/test/isolation/specs/create-trigger.spec
M src/test/isolation/specs/deadlock-hard.spec
M src/test/isolation/specs/deadlock-parallel.spec
M src/test/isolation/specs/deadlock-simple.spec
M src/test/isolation/specs/deadlock-soft-2.spec
M src/test/isolation/specs/deadlock-soft.spec
M src/test/isolation/specs/delete-abort-savept-2.spec
M src/test/isolation/specs/delete-abort-savept.spec
M src/test/isolation/specs/drop-index-concurrently-1.spec
M src/test/isolation/specs/eval-plan-qual-trigger.spec
M src/test/isolation/specs/eval-plan-qual.spec
M src/test/isolation/specs/fk-contention.spec
M src/test/isolation/specs/fk-deadlock.spec
M src/test/isolation/specs/fk-deadlock2.spec
M src/test/isolation/specs/fk-partitioned-1.spec
M src/test/isolation/specs/fk-partitioned-2.spec
M src/test/isolation/specs/freeze-the-dead.spec
M src/test/isolation/specs/index-only-scan.spec
M src/test/isolation/specs/inherit-temp.spec
M src/test/isolation/specs/insert-conflict-do-nothing-2.spec
M src/test/isolation/specs/insert-conflict-do-nothing.spec
M src/test/isolation/specs/insert-conflict-do-update-2.spec
M src/test/isolation/specs/insert-conflict-do-update-3.spec
M src/test/isolation/specs/insert-conflict-do-update.spec
M src/test/isolation/specs/insert-conflict-specconflict.spec
M src/test/isolation/specs/lock-committed-keyupdate.spec
M src/test/isolation/specs/lock-committed-update.spec
M src/test/isolation/specs/lock-update-delete.spec
M src/test/isolation/specs/lock-update-traversal.spec
M src/test/isolation/specs/multiple-cic.spec
M src/test/isolation/specs/multiple-row-versions.spec
M src/test/isolation/specs/multixact-no-deadlock.spec
M src/test/isolation/specs/multixact-no-forget.spec
M src/test/isolation/specs/nowait-2.spec
M src/test/isolation/specs/nowait-3.spec
M src/test/isolation/specs/nowait-4.spec
M src/test/isolation/specs/nowait-5.spec
M src/test/isolation/specs/nowait.spec
M src/test/isolation/specs/partial-index.spec
M src/test/isolation/specs/partition-concurrent-attach.spec
M src/test/isolation/specs/partition-key-update-1.spec
M src/test/isolation/specs/partition-key-update-2.spec
M src/test/isolation/specs/partition-key-update-3.spec
M src/test/isolation/specs/partition-key-update-4.spec
M src/test/isolation/specs/plpgsql-toast.spec
M src/test/isolation/specs/predicate-gin.spec
M src/test/isolation/specs/predicate-gist.spec
M src/test/isolation/specs/predicate-hash.spec
M src/test/isolation/specs/predicate-lock-hot-tuple.spec
M src/test/isolation/specs/prepared-transactions-cic.spec
M src/test/isolation/specs/prepared-transactions.spec
M src/test/isolation/specs/project-manager.spec
M src/test/isolation/specs/propagate-lock-delete.spec
M src/test/isolation/specs/read-only-anomaly-2.spec
M src/test/isolation/specs/read-only-anomaly-3.spec
M src/test/isolation/specs/read-only-anomaly.spec
M src/test/isolation/specs/read-write-unique-2.spec
M src/test/isolation/specs/read-write-unique-3.spec
M src/test/isolation/specs/read-write-unique-4.spec
M src/test/isolation/specs/read-write-unique.spec
M src/test/isolation/specs/receipt-report.spec
M src/test/isolation/specs/referential-integrity.spec
M src/test/isolation/specs/reindex-concurrently.spec
M src/test/isolation/specs/ri-trigger.spec
M src/test/isolation/specs/sequence-ddl.spec
M src/test/isolation/specs/serializable-parallel-2.spec
M src/test/isolation/specs/serializable-parallel.spec
M src/test/isolation/specs/simple-write-skew.spec
M src/test/isolation/specs/skip-locked-2.spec
M src/test/isolation/specs/skip-locked-3.spec
M src/test/isolation/specs/skip-locked-4.spec
M src/test/isolation/specs/skip-locked.spec
M src/test/isolation/specs/temporal-range-integrity.spec
M src/test/isolation/specs/timeouts.spec
M src/test/isolation/specs/total-cash.spec
M src/test/isolation/specs/truncate-conflict.spec
M src/test/isolation/specs/tuplelock-conflict.spec
M src/test/isolation/specs/tuplelock-partition.spec
M src/test/isolation/specs/tuplelock-update.spec
M src/test/isolation/specs/tuplelock-upgrade-no-deadlock.spec
M src/test/isolation/specs/two-ids.spec
M src/test/isolation/specs/update-conflict-out.spec
M src/test/isolation/specs/update-locked-tuple.spec
M src/test/isolation/specs/vacuum-concurrent-drop.spec
M src/test/isolation/specs/vacuum-conflict.spec
M src/test/isolation/specs/vacuum-reltuples.spec
M src/test/isolation/specs/vacuum-skip-locked.spec
M src/test/isolation/specscanner.l

Doc: fix confusion about LEAKPROOF in syntax summaries.

commit   : 9aa06956eaf16d94c591ff7aca660c3e39d55591    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 14:27:13 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 14:27:13 -0400    

Click here for diff

The syntax summaries for CREATE FUNCTION and allied commands  
made it look like LEAKPROOF is an alternative to  
IMMUTABLE/STABLE/VOLATILE, when of course it is an orthogonal  
option.  Improve that.  
  
Per gripe from aazamrafeeque0.  Thanks to David Johnston for  
suggestions.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/ref/alter_function.sgml
M doc/src/sgml/ref/alter_routine.sgml
M doc/src/sgml/ref/create_function.sgml

Don't assume GSSAPI result strings are null-terminated.

commit   : 13f3655683bac3cb8dc32f84f15b8141592ab281    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 14:01:32 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 14:01:32 -0400    

Click here for diff

Our uses of gss_display_status() and gss_display_name() assumed  
that the gss_buffer_desc strings returned by those functions are  
null-terminated.  It appears that they generally are, given the  
lack of field complaints up to now.  However, the available  
documentation does not promise this, and some man pages  
for gss_display_status() show examples that rely on the  
gss_buffer_desc.length field instead of expecting null  
termination.  Also, we now have a report that on some  
implementations, clang's address sanitizer is of the opinion  
that the byte after the specified length is undefined.  
  
Hence, change the code to rely on the length field instead.  
  
This might well be cosmetic rather than fixing any real bug, but  
it's hard to be sure, so back-patch to all supported branches.  
While here, also back-patch the v12 changes that made pg_GSS_error  
deal honestly with multiple messages available from  
gss_display_status.  
  
Per report from Sudheer H R.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/interfaces/libpq/fe-gssapi-common.c

Improve display of query results in isolation tests.

commit   : b961bdfe1664f2d699ebcfbca15b1ef288b9790c    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 11:12:31 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 23 Jun 2021 11:12:31 -0400    

Click here for diff

Previously, isolationtester displayed SQL query results using some  
ad-hoc code that clearly hadn't had much effort expended on it.  
Field values longer than 14 characters weren't separated from  
the next field, and usually caused misalignment of the columns  
too.  Also there was no visual separation of a query's result  
from subsequent isolationtester output.  This made test result  
files confusing and hard to read.  
  
To improve matters, let's use libpq's PQprint() function.  Although  
that's long since unused by psql, it's still plenty good enough  
for the purpose here.  
  
Like 741d7f104, back-patch to all supported branches, so that this  
isn't a stumbling block for back-patching isolation test changes.  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/test_decoding/expected/concurrent_ddl_dml.out
M contrib/test_decoding/expected/delayed_startup.out
M contrib/test_decoding/expected/mxact.out
M contrib/test_decoding/expected/oldest_xmin.out
M contrib/test_decoding/expected/ondisk_startup.out
M contrib/test_decoding/expected/snapshot_transfer.out
M contrib/test_decoding/expected/subxact_without_top.out
M src/test/isolation/expected/aborted-keyrevoke.out
M src/test/isolation/expected/alter-table-1.out
M src/test/isolation/expected/alter-table-2.out
M src/test/isolation/expected/alter-table-3.out
M src/test/isolation/expected/alter-table-4.out
M src/test/isolation/expected/async-notify.out
M src/test/isolation/expected/classroom-scheduling.out
M src/test/isolation/expected/create-trigger.out
M src/test/isolation/expected/deadlock-parallel.out
M src/test/isolation/expected/delete-abort-savept-2.out
M src/test/isolation/expected/delete-abort-savept.out
M src/test/isolation/expected/drop-index-concurrently-1.out
M src/test/isolation/expected/drop-index-concurrently-1_2.out
M src/test/isolation/expected/eval-plan-qual-trigger.out
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/isolation/expected/freeze-the-dead.out
M src/test/isolation/expected/inherit-temp.out
M src/test/isolation/expected/insert-conflict-do-nothing-2.out
M src/test/isolation/expected/insert-conflict-do-nothing.out
M src/test/isolation/expected/insert-conflict-do-update-2.out
M src/test/isolation/expected/insert-conflict-do-update-3.out
M src/test/isolation/expected/insert-conflict-do-update.out
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/expected/lock-committed-keyupdate.out
M src/test/isolation/expected/lock-committed-update.out
M src/test/isolation/expected/lock-update-delete.out
M src/test/isolation/expected/lock-update-delete_1.out
M src/test/isolation/expected/lock-update-traversal.out
M src/test/isolation/expected/multiple-cic.out
M src/test/isolation/expected/multiple-row-versions.out
M src/test/isolation/expected/multixact-no-deadlock.out
M src/test/isolation/expected/multixact-no-forget.out
M src/test/isolation/expected/multixact-no-forget_1.out
M src/test/isolation/expected/nowait-2.out
M src/test/isolation/expected/nowait-3.out
M src/test/isolation/expected/nowait-4.out
M src/test/isolation/expected/nowait-4_1.out
M src/test/isolation/expected/nowait-5.out
M src/test/isolation/expected/nowait.out
M src/test/isolation/expected/partial-index.out
M src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/expected/partition-key-update-2.out
M src/test/isolation/expected/partition-key-update-3.out
M src/test/isolation/expected/partition-key-update-4.out
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/expected/predicate-gin.out
M src/test/isolation/expected/predicate-gist.out
M src/test/isolation/expected/predicate-hash.out
M src/test/isolation/expected/predicate-lock-hot-tuple.out
M src/test/isolation/expected/prepared-transactions-cic.out
M src/test/isolation/expected/prepared-transactions.out
M src/test/isolation/expected/project-manager.out
M src/test/isolation/expected/read-only-anomaly-2.out
M src/test/isolation/expected/read-only-anomaly-3.out
M src/test/isolation/expected/read-only-anomaly.out
M src/test/isolation/expected/read-write-unique-2.out
M src/test/isolation/expected/read-write-unique-3.out
M src/test/isolation/expected/read-write-unique-4.out
M src/test/isolation/expected/read-write-unique.out
M src/test/isolation/expected/receipt-report.out
M src/test/isolation/expected/referential-integrity.out
M src/test/isolation/expected/reindex-concurrently.out
M src/test/isolation/expected/ri-trigger.out
M src/test/isolation/expected/sequence-ddl.out
M src/test/isolation/expected/serializable-parallel-2.out
M src/test/isolation/expected/serializable-parallel.out
M src/test/isolation/expected/skip-locked-2.out
M src/test/isolation/expected/skip-locked-3.out
M src/test/isolation/expected/skip-locked-4.out
M src/test/isolation/expected/skip-locked-4_1.out
M src/test/isolation/expected/skip-locked.out
M src/test/isolation/expected/temporal-range-integrity.out
M src/test/isolation/expected/timeouts.out
M src/test/isolation/expected/total-cash.out
M src/test/isolation/expected/truncate-conflict.out
M src/test/isolation/expected/tuplelock-conflict.out
M src/test/isolation/expected/tuplelock-partition.out
M src/test/isolation/expected/tuplelock-update.out
M src/test/isolation/expected/tuplelock-upgrade-no-deadlock.out
M src/test/isolation/expected/two-ids.out
M src/test/isolation/expected/update-conflict-out.out
M src/test/isolation/expected/vacuum-reltuples.out
M src/test/isolation/isolationtester.c
M src/test/modules/brin/expected/summarization-and-inprogress-insertion.out
M src/test/modules/snapshot_too_old/expected/sto_using_cursor.out
M src/test/modules/snapshot_too_old/expected/sto_using_hash_index.out
M src/test/modules/snapshot_too_old/expected/sto_using_select.out

Use annotations to reduce instability of isolation-test results.

commit   : e2cde85ef2c38ab9cc1ff7190ded7433f637b014    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 22 Jun 2021 21:43:12 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 22 Jun 2021 21:43:12 -0400    

Click here for diff

We've long contended with isolation test results that aren't entirely  
stable.  Some test scripts insert long delays to try to force stable  
results, which is not terribly desirable; but other erratic failure  
modes remain, causing unrepeatable buildfarm failures.  I've spent a  
fair amount of time trying to solve this by improving the server-side  
support code, without much success: that way is fundamentally unable  
to cope with diffs that stem from chance ordering of arrival of  
messages from different server processes.  
  
We can improve matters on the client side, however, by annotating  
the test scripts themselves to show the desired reporting order  
of events that might occur in different orders.  This patch adds  
three types of annotations to deal with (a) test steps that might or  
might not complete their waits before the isolationtester can see them  
waiting; (b) test steps in different sessions that can legitimately  
complete in either order; and (c) NOTIFY messages that might arrive  
before or after the completion of a step in another session.  We might  
need more annotation types later, but this seems to be enough to deal  
with the instabilities we've seen in the buildfarm.  It also lets us  
get rid of all the long delays that were previously used, cutting more  
than a minute off the runtime of the isolation tests.  
  
Back-patch to all supported branches, because the buildfarm  
instabilities affect all the branches, and because it seems desirable  
to keep isolationtester's capabilities the same across all branches  
to simplify possible future back-patching of tests.  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/test_decoding/expected/concurrent_ddl_dml.out
M src/test/isolation/README
M src/test/isolation/expected/alter-table-3.out
M src/test/isolation/expected/alter-table-4.out
M src/test/isolation/expected/deadlock-hard.out
M src/test/isolation/expected/deadlock-simple.out
M src/test/isolation/expected/eval-plan-qual-trigger.out
M src/test/isolation/expected/eval-plan-qual.out
M src/test/isolation/expected/fk-deadlock2_1.out
M src/test/isolation/expected/fk-deadlock2_2.out
M src/test/isolation/expected/fk-deadlock_1.out
M src/test/isolation/expected/fk-partitioned-1.out
M src/test/isolation/expected/fk-partitioned-2.out
M src/test/isolation/expected/insert-conflict-do-nothing-2.out
M src/test/isolation/expected/insert-conflict-specconflict.out
M src/test/isolation/expected/lock-committed-keyupdate.out
M src/test/isolation/expected/lock-update-delete_1.out
M src/test/isolation/expected/multiple-cic.out
D src/test/isolation/expected/multiple-cic_1.out
M src/test/isolation/expected/multixact-no-forget_1.out
M src/test/isolation/expected/nowait-4.out
M src/test/isolation/expected/nowait-4_1.out
M src/test/isolation/expected/nowait-5.out
M src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/expected/partition-key-update-1.out
M src/test/isolation/expected/partition-key-update-3.out
M src/test/isolation/expected/propagate-lock-delete.out
M src/test/isolation/expected/read-write-unique-2.out
M src/test/isolation/expected/read-write-unique-3.out
M src/test/isolation/expected/read-write-unique-4.out
M src/test/isolation/expected/read-write-unique.out
M src/test/isolation/expected/sequence-ddl.out
M src/test/isolation/expected/skip-locked-4_1.out
M src/test/isolation/expected/timeouts.out
M src/test/isolation/expected/tuplelock-update.out
M src/test/isolation/isolationtester.c
M src/test/isolation/isolationtester.h
M src/test/isolation/specparse.y
M src/test/isolation/specs/deadlock-hard.spec
M src/test/isolation/specs/deadlock-soft-2.spec
M src/test/isolation/specs/insert-conflict-specconflict.spec
M src/test/isolation/specs/multiple-cic.spec
M src/test/isolation/specs/timeouts.spec
M src/test/isolation/specs/tuplelock-update.spec
M src/test/isolation/specscanner.l

Restore the portal-level snapshot for simple expressions, too.

commit   : 6f1321d5ae8e0db1500def46f12eadf94521f5ee    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 22 Jun 2021 17:48:39 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 22 Jun 2021 17:48:39 -0400    

Click here for diff

Commits 84f5c2908 et al missed the need to cover plpgsql's "simple  
expression" code path.  If the first thing we execute after a  
COMMIT/ROLLBACK is one of those, rather than a full-fledged SPI command,  
we must explicitly do EnsurePortalSnapshotExists() to make sure we have  
an outer snapshot.  Note that it wouldn't be good enough to just push a  
snapshot for the duration of the expression execution: what comes back  
might be toasted, so we'd better have a snapshot protecting it.  
  
The test case demonstrating this fact cheats a bit by marking a SQL  
function immutable even though it fetches from a table.  That's  
nothing that users haven't been seen to do, though.  
  
Per report from Jim Nasby.  Back-patch to v11, like the previous fix.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql

Back-patch "Tolerate version lookup failure for old style Windows locale names."

commit   : 6dcb185bfc38c93c550d7229ab8bc41bfe80f800    
  
author   : Thomas Munro <[email protected]>    
date     : Wed, 4 Nov 2020 14:58:34 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Wed, 4 Nov 2020 14:58:34 +1300    

Click here for diff

If users provide old style pre-standardized Windows locale names in a  
CREATE COLLATION command, the OS is unable to provide version  
information.  Continue without capturing version information, rather  
than exposing an OS error.  
  
This was originally done in commit 9f12a3b9 for 14 only, to support  
future features that might encounter old style names from initdb's  
default.  It wasn't done in 13 because I didn't consider that users  
might actually want to use the old format explicitly (something we  
should consider blocking in a future release with a better error  
message, but that's not a policy we've decided on yet).  
  
Back-patch to 13, based on the field complaint in pgsql-bugs #17058.  
  
Reported-by: Yasushi Yamashita <[email protected]>  
Discussion: https://postgr.es/m/17058-b49f5793c912c5aa%40postgresql.org  

M src/backend/utils/adt/pg_locale.c

Fix misbehavior of DROP OWNED BY with duplicate polroles entries.

commit   : 33af10c598e2bff55be46eb79f7c785440aa13c5    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 18 Jun 2021 18:00:09 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 18 Jun 2021 18:00:09 -0400    

Click here for diff

Ordinarily, a pg_policy.polroles array wouldn't list the same role  
more than once; but CREATE POLICY does not prevent that.  If we  
perform DROP OWNED BY on a role that is listed more than once,  
RemoveRoleFromObjectPolicy either suffered an assertion failure  
or encountered a tuple-updated-by-self error.  Rewrite it to cope  
correctly with duplicate entries, and add a CommandCounterIncrement  
call to prevent the other problem.  
  
Per discussion, there's other cleanup that ought to happen here,  
but this seems like the minimum essential fix.  
  
Per bug #17062 from Alexander Lakhin.  It's been broken all along,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/policy.c
M src/test/regress/expected/rowsecurity.out
M src/test/regress/sql/rowsecurity.sql

Avoid scribbling on input node tree in CREATE/ALTER DOMAIN.

commit   : 102f31a208b85aec542fc93f73a222d98d551694    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 18 Jun 2021 12:09:22 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 18 Jun 2021 12:09:22 -0400    

Click here for diff

This works fine in the "simple Query" code path; but if the  
statement is in the plan cache then it's corrupted for future  
re-execution.  Apply copyObject() to protect the original  
tree from modification, as we've done elsewhere.  
  
This narrow fix is applied only to the back branches.  In HEAD,  
the problem was fixed more generally by commit 7c337b6b5; but  
that changed ProcessUtility's API, so it's infeasible to  
back-patch.  
  
Per bug #17053 from Charles Samborski.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/typecmds.c

Don't set a fast default for anything but a plain table

commit   : 5b6b5e5ee598ced2dc8e1d08a34d499860a4d15b    
  
author   : Andrew Dunstan <[email protected]>    
date     : Fri, 18 Jun 2021 07:44:58 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Fri, 18 Jun 2021 07:44:58 -0400    

Click here for diff

The fast default code added in Release 11 omitted to check that the  
table a fast default was being added to was a plain table. Thus one  
could be added to a foreign table, which predicably blows up. Here we  
perform that check.  
  
In addition, on the back branches, since some of these might have  
escaped into the wild, if we encounter a missing value for  
an attribute of something other than a plain table we ignore it.  
  
Fixes bug #17056  
  
Backpatch to release 11,  
  
Reviewed by: Andres Freund, Álvaro Herrera and Tom Lane  

M src/backend/catalog/heap.c
M src/backend/commands/tablecmds.c
M src/backend/utils/cache/relcache.c
M src/test/regress/expected/fast_default.out
M src/test/regress/sql/fast_default.sql

Fix valgrind issue in pgoutput.c.

commit   : 357cb8f07f95665ea533ff534821c22c35b01288    
  
author   : Amit Kapila <[email protected]>    
date     : Fri, 18 Jun 2021 08:51:18 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Fri, 18 Jun 2021 08:51:18 +0530    

Click here for diff

We use a tuple conversion map for partitions when replicating using an  
ancestor's schema to convert tuples from partition's type to the  
ancestor's. Before this map got initialized, we were processing  
invalidation messages which access this map.  
  
This issue happens only in version 13 as in HEAD we already have a code  
that initializes each relation entry before we can process any  
invalidation message. This issue is introduced by commit d250568121 in  
version 13.  
  
Reported-by: Tom Lane, as per buildfarm meber skink  
Author: Amit Langote  
Reviewed-by: Dilip Kumar, Amit Kapila  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M src/backend/replication/pgoutput/pgoutput.c

Tidy up GetMultiXactIdMembers()'s behavior on error

commit   : 42f782be916807121fcce20811892c0bad95fca3    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 17 Jun 2021 14:50:42 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 17 Jun 2021 14:50:42 +0300    

Click here for diff

One of the error paths left *members uninitialized. That's not a live  
bug, because most callers don't look at *members when the function  
returns -1, but let's be tidy. One caller, in heap_lock_tuple(), does  
"if (members != NULL) pfree(members)", but AFAICS it never passes an  
invalid 'multi' value so it should not reach that error case.  
  
The callers are also a bit inconsistent in their expectations.  
heap_lock_tuple() pfrees the 'members' array if it's not-NULL, others  
pfree() it if "nmembers >= 0", and others if "nmembers > 0". That's  
not a live bug either, because the function should never return 0, but  
add an Assert for that to make it more clear. I left the callers alone  
for now.  
  
I also moved the line where we set *nmembers. It wasn't wrong before,  
but I like to do that right next to the 'return' statement, to make it  
clear that it's always set on return.  
  
Also remove one unreachable return statement after ereport(ERROR), for  
brevity and for consistency with the similar if-block right after it.  
  
Author: Greg Nancarrow with the additional changes by me  
Backpatch-through: 9.6, all supported versions  

M src/backend/access/transam/multixact.c

Fix subtransaction test for Python 3.10

commit   : 3989f8fb9dd84afda7e27de04bf83e31903d46ba    
  
author   : Peter Eisentraut <[email protected]>    
date     : Sat, 5 Jun 2021 07:16:34 +0200    
  
committer: Peter Eisentraut <[email protected]>    
date     : Sat, 5 Jun 2021 07:16:34 +0200    

Click here for diff

Starting with Python 3.10, the stacktrace looks differently:  
  -  PL/Python function "subtransaction_exit_subtransaction_in_with", line 3, in <module>  
  -    s.__exit__(None, None, None)  
  +  PL/Python function "subtransaction_exit_subtransaction_in_with", line 2, in <module>  
  +    with plpy.subtransaction() as s:  
Using try/except specifically makes the error look always the same.  
  
(See https://github.com/python/cpython/pull/25719 for the discussion  
of this change in Python.)  
  
Author: Honza Horak <[email protected]>  
Discussion: https://www.postgresql.org/message-id/flat/853083.1620749597%40sss.pgh.pa.us  
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1959080  

M src/pl/plpython/expected/plpython_subtransaction.out
M src/pl/plpython/sql/plpython_subtransaction.sql

Document a few caveats in synchronous logical replication.

commit   : 9f7bba2629cf3413638936d0376bb50403144332    
  
author   : Amit Kapila <[email protected]>    
date     : Thu, 17 Jun 2021 10:17:13 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Thu, 17 Jun 2021 10:17:13 +0530    

Click here for diff

In a synchronous logical setup, locking [user] catalog tables can cause  
deadlock. This is because logical decoding of transactions can lock  
catalog tables to access them so exclusively locking those in transactions  
can lead to deadlock. To avoid this users must refrain from having  
exclusive locks on catalog tables.  
  
Author: Takamichi Osumi  
Reviewed-by: Vignesh C, Amit Kapila  
Backpatch-through: 9.6  
Discussion: https://www.postgresql.org/message-id/20210222222847.tpnb6eg3yiykzpky%40alap3.anarazel.de  

M doc/src/sgml/logicaldecoding.sgml

Fix plancache refcount leak after error in ExecuteQuery.

commit   : d03a41d1c860ffc97bda40a3673698d4044f5c88    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 16 Jun 2021 19:30:17 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 16 Jun 2021 19:30:17 -0400    

Click here for diff

When stuffing a plan from the plancache into a Portal, one is  
not supposed to risk throwing an error between GetCachedPlan and  
PortalDefineQuery; if that happens, the plan refcount incremented  
by GetCachedPlan will be leaked.  I managed to break this rule  
while refactoring code in 9dbf2b7d7.  There is no visible  
consequence other than some memory leakage, and since nobody is  
very likely to trigger the relevant error conditions many times  
in a row, it's not surprising we haven't noticed.  Nonetheless,  
it's a bug, so rearrange the order of operations to remove the  
hazard.  
  
Noted on the way to looking for a better fix for bug #17053.  
This mistake is pretty old, so back-patch to all supported  
branches.  

M src/backend/commands/prepare.c

Fix outdated comment that talked about seek position of WAL file.

commit   : e89a8e30e0a50e7882ab5a6896a11872bec969e3    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Wed, 16 Jun 2021 12:34:32 +0300    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Wed, 16 Jun 2021 12:34:32 +0300    

Click here for diff

Since commit c24dcd0cfd, we have been using pg_pread() to read the WAL  
file, which doesn't change the seek position (unless we fall back to  
the implementation in src/port/pread.c). Update comment accordingly.  
  
Backpatch-through: 12, where we started to use pg_pread()  

M src/backend/access/transam/xlog.c

Further refinement of stuck_on_old_timeline recovery test

commit   : d906d106f854c51f830af05fe95c71d2898bb6b4    
  
author   : Andrew Dunstan <[email protected]>    
date     : Tue, 15 Jun 2021 15:30:11 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Tue, 15 Jun 2021 15:30:11 -0400    

Click here for diff

TestLib::perl2host can take a file argument as well as a directory  
argument, so that code becomes substantially simpler. Also add comments  
on why we're using forward slashes, and why we're setting  
PERL_BADLANG=0.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/test/recovery/t/025_stuck_on_old_timeline.pl

Fix decoding of speculative aborts.

commit   : 602a32a685e4ca715a996f899d906e5f24308b51    
  
author   : Amit Kapila <[email protected]>    
date     : Tue, 15 Jun 2021 08:41:16 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Tue, 15 Jun 2021 08:41:16 +0530    

Click here for diff

During decoding for speculative inserts, we were relying for cleaning  
toast hash on confirmation records or next change records. But that  
could lead to multiple problems (a) memory leak if there is neither a  
confirmation record nor any other record after toast insertion for a  
speculative insert in the transaction, (b) error and assertion failures  
if the next operation is not an insert/update on the same table.  
  
The fix is to start queuing spec abort change and clean up toast hash  
and change record during its processing. Currently, we are queuing the  
spec aborts for both toast and main table even though we perform cleanup  
while processing the main table's spec abort record. Later, if we have a  
way to distinguish between the spec abort record of toast and the main  
table, we can avoid queuing the change for spec aborts of toast tables.  
  
Reported-by: Ashutosh Bapat  
Author: Dilip Kumar  
Reviewed-by: Amit Kapila  
Backpatch-through: 9.6, where it was introduced  
Discussion: https://postgr.es/m/CAExHW5sPKF-Oovx_qZe4p5oM6Dvof7_P+XgsNAViug15Fm99jA@mail.gmail.com  

M src/backend/replication/logical/decode.c
M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h

Update variant expected-result file.

commit   : 2da5ddf8340f285453fbdcc57e7c40388953fafb    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 14 Jun 2021 21:58:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 14 Jun 2021 21:58:26 -0400    

Click here for diff

This should have been updated in d2d8a229b, but it was overlooked.  
According to 31a877f18 which added it, this file is meant to show the  
results you get under default_transaction_isolation = serializable.  
We've largely lost track of that goal in other isolation tests, but  
as long as we've got this one, it should be right.  
  
Noted while fooling about with the isolationtester.  

M src/test/isolation/expected/drop-index-concurrently-1_2.out

Remove orphaned expected-result file.

commit   : 0483ae7c513bf2604443fa681811b4a5113c6799    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 14 Jun 2021 21:28:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 14 Jun 2021 21:28:21 -0400    

Click here for diff

This should have been removed in 43e084197, which removed the  
corresponding spec file.  Noted while fooling about with the  
isolationtester.  

D src/test/isolation/expected/insert-conflict-toast_1.out

Work around portability issue with newer versions of mktime().

commit   : bc7885b7f5fcaf5e389aa440bd530782460406c2    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 13 Jun 2021 14:32:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 13 Jun 2021 14:32:42 -0400    

Click here for diff

Recent glibc versions have made mktime() fail if tm_isdst is  
inconsistent with the prevailing timezone; in particular it fails for  
tm_isdst = 1 when the zone is UTC.  (This seems wildly inconsistent  
with the POSIX-mandated treatment of "incorrect" values for the other  
fields of struct tm, so if you ask me it's a bug, but I bet they'll  
say it's intentional.)  This has been observed to cause cosmetic  
problems when pg_restore'ing an archive created in a different  
timezone.  
  
To fix, do mktime() using the field values from the archive, and if  
that fails try again with tm_isdst = -1.  This will give a result  
that's off by the UTC-offset difference from the original zone, but  
that was true before, too.  It's not terribly critical since we don't  
do anything with the result except possibly print it.  (Someday we  
should flush this entire bit of logic and record a standard-format  
timestamp in the archive instead.  That's not okay for a back-patched  
bug fix, though.)  
  
Also, guard our only other use of mktime() by having initdb's  
build_time_t() set tm_isdst = -1 not 0.  This case could only have  
an issue in zones that are DST year-round; but I think some do exist,  
or could in future.  
  
Per report from Wells Oliver.  Back-patch to all supported  
versions, since any of them might need to run with a newer glibc.  
  
Discussion: https://postgr.es/m/CAOC+FBWDhDHO7G-i1_n_hjRzCnUeFO+H-Czi1y10mFhRWpBrew@mail.gmail.com  

M src/bin/initdb/findtimezone.c
M src/bin/pg_dump/pg_backup_archiver.c

Further tweaks to stuck_on_old_timeline recovery test

commit   : 47d5781cb137a2b027e2bf230a010a8407aeda00    
  
author   : Andrew Dunstan <[email protected]>    
date     : Sun, 13 Jun 2021 07:10:41 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Sun, 13 Jun 2021 07:10:41 -0400    

Click here for diff

Translate path slashes on target directory path. This was confusing old  
branches, but is applied to all branches for the sake of uniformity.  
Perl is perfectly able to understand paths with forward slashes.  
  
Along the way, restore the previous archive_wait query, for the sake of  
uniformity with other tests, per gripe from Tom Lane.  

M src/test/recovery/t/025_stuck_on_old_timeline.pl

Ignore more environment variables in pg_regress.c

commit   : 545aa6a59357f7fb8e341a6a5b4c85e322e78697    
  
author   : Michael Paquier <[email protected]>    
date     : Sun, 13 Jun 2021 20:07:45 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sun, 13 Jun 2021 20:07:45 +0900    

Click here for diff

This is similar to the work done in 8279f68 for TestLib.pm, where  
environment variables set may cause unwanted failures if using a  
temporary installation with pg_regress.  The list of variables reset is  
adjusted in each stable branch depending on what is supported.  
  
Comments are added to remember that the lists in TestLib.pm and  
pg_regress.c had better be kept in sync.  
  
Reviewed-by: Álvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/test/perl/TestLib.pm
M src/test/regress/pg_regress.c

Restore robustness of TAP tests that wait for postmaster restart.

commit   : 94e21021cdf25e50e28be3e4f3147ed56796433a    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 12 Jun 2021 15:12:10 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 12 Jun 2021 15:12:10 -0400    

Click here for diff

Several TAP tests use poll_query_until() to wait for the postmaster  
to restart.  They were checking to see if a trivial query  
(e.g. "SELECT 1") succeeds.  However, that's problematic in the wake  
of commit 11e9caff8, because now that we feed said query to psql  
via stdin, we risk IPC::Run whining about a SIGPIPE failure if psql  
quits before reading the query.  Hence, we can't use a nonempty  
query in cases where we need to wait for connection failures to  
stop happening.  
  
Per the precedent of commits c757a3da0 and 6d41dd045, we can pass  
"undef" as the query in such cases to ensure that IPC::Run has  
nothing to write.  However, then we have to say that the expected  
output is empty, and this exposes a deficiency in poll_query_until:  
if psql fails altogether and returns empty stdout, poll_query_until  
will treat that as a success!  That's because, contrary to its  
documentation, it makes no actual check for psql failure, looking  
neither at the exit status nor at stderr.  
  
To fix that, adjust poll_query_until to insist on empty stderr as  
well as a stdout match.  (I experimented with checking exit status  
instead, but it seems that psql often does exit(1) in cases that we  
need to consider successes.  That might be something to fix someday,  
but it would be a non-back-patchable behavior change.)  
  
Back-patch to v10.  The test cases needing this exist only as far  
back as v11, but it seems wise to keep poll_query_until's behavior  
the same in v10, in case we back-patch another such test case in  
future.  (9.6 does not currently need this change, because in that  
branch poll_query_until can't be told to accept empty stdout as  
a success case.)  
  
Per assorted buildfarm failures, mostly on hoverfly.  
  
Discussion: https://postgr.es/m/CAA4eK1+zM6L4QSA1XMvXY_qqWwdUmqkOS1+hWvL8QcYEBGA1Uw@mail.gmail.com  

M src/test/perl/PostgresNode.pm
M src/test/recovery/t/013_crash_restart.pl

Ensure pg_filenode_relation(0, 0) returns NULL.

commit   : f479ea94bd24e61199fab3596a4a3e7241c6c28c    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 12 Jun 2021 13:29:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 12 Jun 2021 13:29:24 -0400    

Click here for diff

Previously, a zero value for the relfilenode resulted in  
a confusing error message about "unexpected duplicate".  
This function returns NULL for other invalid relfilenode  
values, so zero should be treated likewise.  
  
It's been like this all along, so back-patch to all supported  
branches.  
  
Justin Pryzby  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/adt/dbsize.c

Don't use Asserts to check for violations of replication protocol.

commit   : 8b5055812cadd00a86a438fc0d8bafeba3c4d874    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 12 Jun 2021 12:59:15 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 12 Jun 2021 12:59:15 -0400    

Click here for diff

Using an Assert to check the validity of incoming messages is an  
extremely poor decision.  In a debug build, it should not be that easy  
for a broken or malicious remote client to crash the logrep worker.  
The consequences could be even worse in non-debug builds, which will  
fail to make such checks at all, leading to who-knows-what misbehavior.  
Hence, promote every Assert that could possibly be triggered by wrong  
or out-of-order replication messages to a full test-and-ereport.  
  
To avoid bloating the set of messages the translation team has to cope  
with, establish a policy that replication protocol violation error  
reports don't need to be translated.  Hence, all the new messages here  
use errmsg_internal().  A couple of old messages are changed likewise  
for consistency.  
  
Along the way, fix some non-idiomatic or outright wrong uses of  
hash_search().  
  
Most of these mistakes are new with the "streaming replication"  
patch (commit 464824323), but a couple go back a long way.  
Back-patch as appropriate.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/logical/worker.c

Fix new recovery test for use under msys

commit   : 45322bd9b9a33f0f3098d34b56f1ca250f14208a    
  
author   : Andrew Dunstan <[email protected]>    
date     : Sat, 12 Jun 2021 08:37:16 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Sat, 12 Jun 2021 08:37:16 -0400    

Click here for diff

Commit caba8f0d43 wasn't quite right for msys, as demonstrated by  
several buildfarm animals, including jacana and fairywren. We need to  
use the msys perl in the archive command, but call it in such a way that  
Windows will understand the path. Furthermore, inside the copy script we  
need to convert a Windows path to an msys path.  

M src/test/recovery/t/025_stuck_on_old_timeline.pl
M src/test/recovery/t/cp_history_files

Improve log pattern detection in recently-added TAP tests

commit   : fad9c8e93a65f087033312cd655511a28d07ab51    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 12 Jun 2021 15:30:00 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 12 Jun 2021 15:30:00 +0900    

Click here for diff

ab55d74 has introduced some tests with rows found as missing in logical  
replication subscriptions for partitioned tables, relying on a logic  
with a lookup of the logs generated, scanning the whole file.  This  
commit makes the logic more precise, by scanning the logs only from the  
position before the key queries are run to the position where we check  
for the logs.  This will reduce the risk of issues with log patterns  
overlapping with each other if those tests get more complicated in the  
future.  
  
Per discussion with Tom Lane.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 13  

M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/013_partition.pl

Remove PGSSLCRLDIR from the list of variables ignored in TAP tests

commit   : f865d2affc456ede17e46949007682e91d19daa4    
  
author   : Michael Paquier <[email protected]>    
date     : Sat, 12 Jun 2021 10:39:21 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Sat, 12 Jun 2021 10:39:21 +0900    

Click here for diff

This variable was present in the list added by 9d660670, but it is not  
supported by this branch.  Issue noticed while diving into a similar  
change for pg_regress.c.  
  
Backpatch-through: 9.6  

M src/test/perl/TestLib.pm

Report sort phase progress in parallel btree build

commit   : 065ce069ae4fe5f59251435ebfe13e14a2d5874c    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 11 Jun 2021 19:07:32 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 11 Jun 2021 19:07:32 -0400    

Click here for diff

We were already reporting it, but only after the parallel workers were  
finished, which is visibly much later than what happens in a serial  
build.  
  
With this change we report it when the leader starts its own sort phase  
when participating in the build (the normal case).  Now this might  
happen a little later than when the workers start their sorting phases,  
but a) communicating the actual phase start from workers is likely to be  
a hassle, and b) the sort phase start is pretty fuzzy anyway, since  
sorting per se is actually initiated by tuplesort.c internally earlier  
than tuplesort_performsort() is called.  
  
Backpatch to pg12, where the progress reporting code for CREATE INDEX  
went in.  
  
Reported-by: Tomas Vondra <[email protected]>  
Author: Matthias van de Meent <[email protected]>  
Reviewed-by: Greg Nancarrow <[email protected]>  
Reviewed-by: Álvaro Herrera <[email protected]>  
Discussion: https://postgr.es/m/1128176d-1eee-55d4-37ca-e63644422adb  

M src/backend/access/nbtree/nbtsort.c

Fix multiple crasher bugs in partitioned-table replication logic.

commit   : b270713fd44b1378cebc3177e092add0155b55b6    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 11 Jun 2021 16:12:36 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 11 Jun 2021 16:12:36 -0400    

Click here for diff

apply_handle_tuple_routing(), having detected and reported that  
the tuple it needed to update didn't exist, tried to update that  
tuple anyway, leading to a null-pointer dereference.  
  
logicalrep_partition_open() failed to ensure that the  
LogicalRepPartMapEntry it built for a partition was fully  
independent of that for the partition root, leading to  
trouble if the root entry was later freed or rebuilt.  
  
Meanwhile, on the publisher's side, pgoutput_change() sometimes  
attempted to apply execute_attr_map_tuple() to a NULL tuple.  
  
The first of these was reported by Sergey Bernikov in bug #17055;  
I found the other two while developing some test cases for this  
sadly under-tested code.  
  
Diagnosis and patch for the first issue by Amit Langote; patches  
for the others by me; new test cases by me.  Back-patch to v13  
where this logic came in.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/logical/relation.c
M src/backend/replication/logical/worker.c
M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/001_rep_changes.pl
M src/test/subscription/t/013_partition.pl

Fix race condition in invalidating obsolete replication slots

commit   : 218b101008b533156d7e5832fe143d1e04a01301    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 11 Jun 2021 12:16:14 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 11 Jun 2021 12:16:14 -0400    

Click here for diff

The code added to mark replication slots invalid in commit c6550776394e  
had the race condition that a slot can be dropped or advanced  
concurrently with checkpointer trying to invalidate it.  Rewrite the  
code to close those races.  
  
The changes to ReplicationSlotAcquire's API added with c6550776394e are  
not necessary anymore.  To avoid an ABI break in released branches, this  
commit leaves that unchanged; it'll be changed in a master-only commit  
separately.  
  
Backpatch to 13, where this code first appeared.  
  
Reported-by: Andres Freund <[email protected]>  
Author: Andres Freund <[email protected]>  
Author: Álvaro Herrera <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/slot.c

Rearrange logrep worker's snapshot handling some more.

commit   : 6e43f1c2df3da18b9d7087edddaf72dec84cfaf4    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 10 Jun 2021 12:27:27 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 10 Jun 2021 12:27:27 -0400    

Click here for diff

It turns out that worker.c's code path for TRUNCATE was also  
careless about establishing a snapshot while executing user-defined  
code, allowing the checks added by commit 84f5c2908 to fail when  
a trigger is fired in that context.  
  
We could just wrap Push/PopActiveSnapshot around the truncate call,  
but it seems better to establish a policy of holding a snapshot  
throughout execution of a replication step.  To help with that and  
possible future requirements, replace the previous ensure_transaction  
calls with pairs of begin/end_replication_step calls.  
  
Per report from Mark Dilger.  Back-patch to v11, like the previous  
changes.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/logical/worker.c

Adjust new test case to set wal_keep_size.

commit   : 3465328aa19e363f3248839bea4521793940ad34    
  
author   : Robert Haas <[email protected]>    
date     : Thu, 10 Jun 2021 09:08:30 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Thu, 10 Jun 2021 09:08:30 -0400    

Click here for diff

Per buildfarm member conchuela and Kyotaro Horiguchi, it's possible  
for the WAL segment that the cascading standby needs to be removed  
too quickly. Hopefully this will prevent that.  
  
Kyotaro Horiguchi  
  
Discussion: http://postgr.es/m/[email protected]  

M src/test/recovery/t/025_stuck_on_old_timeline.pl

Fix corner case failure of new standby to follow new primary.

commit   : 0826564292156762d32c183c6708c94564fcad1c    
  
author   : Robert Haas <[email protected]>    
date     : Wed, 9 Jun 2021 16:17:13 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Wed, 9 Jun 2021 16:17:13 -0400    

Click here for diff

This only happens if (1) the new standby has no WAL available locally,  
(2) the new standby is starting from the old timeline, (3) the promotion  
happened in the WAL segment from which the new standby is starting,  
(4) the timeline history file for the new timeline is available from  
the archive but the WAL files for are not (i.e. this is a race),  
(5) the WAL files for the new timeline are available via streaming,  
and (6) recovery_target_timeline='latest'.  
  
Commit ee994272ca50f70b53074f0febaec97e28f83c4e introduced this  
logic and was an improvement over the previous code, but it mishandled  
this case. If recovery_target_timeline='latest' and restore_command is  
set, validateRecoveryParameters() can change recoveryTargetTLI to be  
different from receiveTLI. If streaming is then tried afterward,  
expectedTLEs gets initialized with the history of the wrong timeline.  
It's supposed to be a list of entries explaining how to get to the  
target timeline, but in this case it ends up with a list of entries  
explaining how to get to the new standby's original timeline, which  
isn't right.  
  
Dilip Kumar and Robert Haas, reviewed by Kyotaro Horiguchi.  
  
Discussion: http://postgr.es/m/CAFiTN-sE-jr=LB8jQuxeqikd-Ux+jHiXyh4YDiZMPedgQKup0g@mail.gmail.com  

M src/backend/access/transam/xlog.c
A src/test/recovery/t/025_stuck_on_old_timeline.pl
A src/test/recovery/t/cp_history_files

Allow PostgresNode.pm's backup method to accept backup_options.

commit   : 99a0a2ada8c584e7d7796190a5966600f44c037f    
  
author   : Robert Haas <[email protected]>    
date     : Wed, 9 Jun 2021 12:28:39 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Wed, 9 Jun 2021 12:28:39 -0400    

Click here for diff

Partial back-port of commit 081876d75ea15c3bd2ee5ba64a794fd8ea46d794.  
A test case for a pending bug fix needs this capability, but the code  
on 9.6 is significantly different, so I'm only back-patching this  
change as far as v10. We'll have to work around the problem another  
way in v9.6.  
  
Discussion: http://postgr.es/m/CAFiTN-tcivNvL0Rg6rD7_CErNfE75H7+gh9WbMxjbgsattja1Q@mail.gmail.com  

M src/test/perl/PostgresNode.pm

Fix inconsistencies in psql --help=commands

commit   : 6bfcc7d0d19a32fe44a8671aa5ecca9e5b39221a    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 9 Jun 2021 16:25:49 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 9 Jun 2021 16:25:49 +0900    

Click here for diff

The set of subcommands supported by \dAp, \do and \dy was described  
incorrectly in psql's --help.  The documentation was already consistent  
with the code.  
  
Reported-by: inoas, from IRC  
Author: Matthijs van der Vleuten  
Reviewed-by: Neil Chen  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/bin/psql/help.c

Force NO SCROLL for plpgsql's implicit cursors.

commit   : c5b28184132d30ce7f77b24b0e7f302a8f37f407    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 8 Jun 2021 18:40:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 8 Jun 2021 18:40:06 -0400    

Click here for diff

Further thought about bug #17050 suggests that it's a good idea  
to use CURSOR_OPT_NO_SCROLL for the implicit cursor opened by  
a plpgsql FOR-over-query loop.  This ensures that, if somebody  
commits inside the loop, PersistHoldablePortal won't try to  
rewind and re-read the cursor.  While we'd have selected NO_SCROLL  
anyway if FOR UPDATE/SHARE appears in the query, there are other  
hazards with volatile functions; and in any case, it's silly to  
expend effort storing rows that we know for certain won't be needed.  
  
(While here, improve the comment in exec_run_select, which was a bit  
confused about the rationale for when we can use parallel mode.  
Cursor operations aren't a hazard for nameless portals.)  
  
This wasn't an issue until v11, which introduced the possibility  
of persisting such cursors.  Hence, back-patch to v11.  
  
Per bug #17050 from Алексей Булгаков.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/pl/plpgsql/src/pl_exec.c

Avoid misbehavior when persisting a non-stable cursor.

commit   : c1fd756fd23f60fcac120c9cd36de2921144e3bd    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 8 Jun 2021 17:50:15 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 8 Jun 2021 17:50:15 -0400    

Click here for diff

PersistHoldablePortal has long assumed that it should store the  
entire output of the query-to-be-persisted, which requires rewinding  
and re-reading the output.  This is problematic if the query is not  
stable: we might get different row contents, or even a different  
number of rows, which'd confuse the cursor state mightily.  
  
In the case where the cursor is NO SCROLL, this is very easy to  
solve: just store the remaining query output, without any rewinding,  
and tweak the portal's cursor state to match.  Aside from removing  
the semantic problem, this could be significantly more efficient  
than storing the whole output.  
  
If the cursor is scrollable, there's not much we can do, but it  
was already the case that scrolling a volatile query's result was  
pretty unsafe.  We can just document more clearly that getting  
correct results from that is not guaranteed.  
  
There are already prohibitions in place on using SCROLL with  
FOR UPDATE/SHARE, which is one way for a SELECT query to have  
non-stable results.  We could imagine prohibiting SCROLL when  
the query contains volatile functions, but that would be  
expensive to enforce.  Moreover, it could break applications  
that work just fine, if they have functions that are in fact  
stable but the user neglected to mark them so.  So settle for  
documenting the hazard.  
  
While this problem has existed in some guise for a long time,  
it got a lot worse in v11, which introduced the possibility  
of persisting plpgsql cursors (perhaps implicit ones) even  
when they violate the rules for what can be marked WITH HOLD.  
Hence, I've chosen to back-patch to v11 but not further.  
  
Per bug #17050 from Алексей Булгаков.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/declare.sgml
M src/backend/commands/portalcmds.c
M src/pl/plpgsql/src/expected/plpgsql_transaction.out
M src/pl/plpgsql/src/sql/plpgsql_transaction.sql

Remove unnecessary declaration in win32_port.h

commit   : 949e32ee5d5990ff94579abc34d7b2c39b134965    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 8 Jun 2021 13:40:03 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 8 Jun 2021 13:40:03 +0900    

Click here for diff

Mis-merge introduced by e2f21ff, where pgwin32_setenv() was listed but  
not defined in win32env.c.  This had no consequences as this routine  
does not exist in this branch.  
  
Only REL_12_STABLE and REL_13_STABLE got that wrong.  
  
Backpatch-through: 12  

M src/include/port/win32_port.h

Stabilize contrib/seg regression test.

commit   : eed17f39fa3e6c15e62ba8c4227118f506288a30    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 7 Jun 2021 14:52:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 7 Jun 2021 14:52:42 -0400    

Click here for diff

If autovacuum comes along just after we fill table test_seg with  
some data, it will update the stats to the point where we prefer  
a plain indexscan over a bitmap scan, breaking the expected  
output (as well as the point of the test case).  To fix, just  
force a bitmap scan to be chosen here.  
  
This has evidently been wrong since commit de1d042f5.  It's not  
clear why we just recently saw any buildfarm failures due to it;  
but prairiedog has failed twice on this test in the past week.  
Hence, backpatch to v11 where this test case came in.  

M contrib/seg/expected/seg.out
M contrib/seg/sql/seg.sql

Fix incautious handling of possibly-miscoded strings in client code.

commit   : 5b64368742b5cdf73395bd5ddd66b5edcbc20823    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 7 Jun 2021 14:15:25 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 7 Jun 2021 14:15:25 -0400    

Click here for diff

An incorrectly-encoded multibyte character near the end of a string  
could cause various processing loops to run past the string's  
terminating NUL, with results ranging from no detectable issue to  
a program crash, depending on what happens to be in the following  
memory.  
  
This isn't an issue in the server, because we take care to verify  
the encoding of strings before doing any interesting processing  
on them.  However, that lack of care leaked into client-side code  
which shouldn't assume that anyone has validated the encoding of  
its input.  
  
Although this is certainly a bug worth fixing, the PG security team  
elected not to regard it as a security issue, primarily because  
any untrusted text should be sanitized by PQescapeLiteral or  
the like before being incorporated into a SQL or psql command.  
(If an app fails to do so, the same technique can be used to  
cause SQL injection, with probably much more dire consequences  
than a mere client-program crash.)  Those functions were already  
made proof against this class of problem, cf CVE-2006-2313.  
  
To fix, invent PQmblenBounded() which is like PQmblen() except it  
won't return more than the number of bytes remaining in the string.  
In HEAD we can make this a new libpq function, as PQmblen() is.  
It seems imprudent to change libpq's API in stable branches though,  
so in the back branches define PQmblenBounded as a macro in the files  
that need it.  (Note that just changing PQmblen's behavior would not  
be a good idea; notably, it would completely break the escaping  
functions' defense against this exact problem.  So we just want a  
version for those callers that don't have any better way of handling  
this issue.)  
  
Per private report from houjingyi.  Back-patch to all supported branches.  

M src/bin/psql/common.c
M src/bin/psql/psqlscanslash.l
M src/bin/psql/stringutils.c
M src/bin/psql/tab-complete.c
M src/bin/scripts/common.c
M src/common/jsonapi.c
M src/common/wchar.c
M src/fe_utils/print.c
M src/include/mb/pg_wchar.h
M src/interfaces/libpq/fe-print.c
M src/interfaces/libpq/fe-protocol3.c

Doc: fix bogus intarray index example.

commit   : b4c027b5f52f029ac85d5743a5df85a955dd1ffe    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 3 Jun 2021 21:07:12 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 3 Jun 2021 21:07:12 -0400    

Click here for diff

The siglen parameter is provided by gist__intbig_ops not  
gist__int_ops.  
  
Simon Norris  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/intarray.sgml

commit   : 6131cb144f8483fdc2223c1e06ce80e7c4ddf056    
  
author   : Michael Paquier <[email protected]>    
date     : Fri, 4 Jun 2021 09:33:22 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Fri, 4 Jun 2021 09:33:22 +0900    

Click here for diff

The link was pointing to the minimum protocol version.  Incorrect as of  
ff8ca5f.  
  
Author: Daniel Gustafsson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 13  

M doc/src/sgml/libpq.sgml

In PostgresNode.pm, don't pass SQL to psql on the command line

commit   : 4ed9dacb2ff35e2fa8a0dc9b5706a8c9c5017c2e    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 3 Jun 2021 16:08:33 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 3 Jun 2021 16:08:33 -0400    

Click here for diff

The Msys shell mangles certain patterns in its command line, so avoid  
handing arbitrary SQL to psql on the command line and instead use  
IPC::Run's redirection facility for stdin. This pattern is already  
mostly whats used, but query_poll_until() was not doing the right thing.  
  
Problem discovered on the buildfarm when a new TAP test failed on msys.  

M src/test/perl/PostgresNode.pm

Reduce risks of conflicts in internal queries of REFRESH MATVIEW CONCURRENTLY

commit   : 75d66d10e0e35529a80d3442081150fcf501bb7d    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 3 Jun 2021 15:28:37 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 3 Jun 2021 15:28:37 +0900    

Click here for diff

The internal SQL queries used by REFRESH MATERIALIZED VIEW CONCURRENTLY  
include some aliases for its diff and temporary relations with  
rather-generic names: diff, newdata, newdata2 and mv.  Depending on the  
queries used for the materialized view, using CONCURRENTLY could lead to  
some internal failures if the query and those internal aliases conflict.  
  
Those names have been chosen in 841c29c8.  This commit switches instead  
to a naming pattern which is less likely going to cause conflicts, based  
on an idea from Thomas Munro, by appending _$ to those aliases.  This is  
not perfect as those new names could still conflict, but at least it has  
the advantage to keep the code readable and simple while reducing the  
likelihood of conflicts to be close to zero.  
  
Reported-by: Mathis Rudolf  
Author: Bharath Rupireddy  
Reviewed-by: Bernd Helmle, Thomas Munro, Michael Paquier  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/backend/commands/matview.c

Ignore more environment variables in TAP tests

commit   : 9d660670541ccc803e1d48576b4a6d02a19321b0    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 3 Jun 2021 11:51:05 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 3 Jun 2021 11:51:05 +0900    

Click here for diff

Various environment variables were not getting reset in the TAP tests,  
which would cause failures depending on the tests or the environment  
variables involved.  For example, PGSSL{MAX,MIN}PROTOCOLVERSION could  
cause failures in the SSL tests.  Even worse, a junk value of  
PGCLIENTENCODING makes a server startup fail.  The list of variables  
reset is adjusted in each stable branch depending on what is supported.  
  
While on it, simplify a bit the code per a suggestion from Andrew  
Dunstan, using a list of variables instead of doing single deletions.  
  
Reviewed-by: Andrew Dunstan, Daniel Gustafsson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/test/perl/TestLib.pm

Fix planner's row-mark code for inheritance from a foreign table.

commit   : 6753a5b7e86cd9bb1eed580b1f9001d5ea1bbdb3    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 2 Jun 2021 14:38:14 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 2 Jun 2021 14:38:14 -0400    

Click here for diff

Commit 428b260f8 broke planning of cases where row marks are needed  
(SELECT FOR UPDATE, etc) and one of the query's tables is a foreign  
table that has regular table(s) as inheritance children.  We got the  
reverse case right, but apparently were thinking that foreign tables  
couldn't be inheritance parents.  Not so; so we need to be able to  
add a CTID junk column while adding a new child, not only a wholerow  
junk column.  
  
Back-patch to v12 where the faulty code came in.  
  
Amit Langote  
  
Discussion: https://postgr.es/m/CA+HiwqEmo3FV1LAQ4TVyS2h1WM=kMkZUmbNuZSCnfHvMcUcPeA@mail.gmail.com  

M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
M src/backend/optimizer/util/inherit.c

Reject SELECT ... GROUP BY GROUPING SETS (()) FOR UPDATE.

commit   : e5b0fffa17f610f03f03952c8c4a247c39e61292    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 1 Jun 2021 11:12:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 1 Jun 2021 11:12:56 -0400    

Click here for diff

This case should be disallowed, just as FOR UPDATE with a plain  
GROUP BY is disallowed; FOR UPDATE only makes sense when each row  
of the query result can be identified with a single table row.  
However, we missed teaching CheckSelectLocking() to check  
groupingSets as well as groupClause, so that it would allow  
degenerate grouping sets.  That resulted in a bad plan and  
a null-pointer dereference in the executor.  
  
Looking around for other instances of the same bug, the only one  
I found was in examine_simple_variable().  That'd just lead to  
silly estimates, but it should be fixed too.  
  
Per private report from Yaoguang Chen.  
Back-patch to all supported branches.  

M src/backend/parser/analyze.c
M src/backend/utils/adt/selfuncs.c
M src/test/regress/expected/errors.out
M src/test/regress/sql/errors.sql

pgoutput: Fix memory leak due to RelationSyncEntry.map.

commit   : d2505681211b4803cae916e9f4c2331943f851dc    
  
author   : Amit Kapila <[email protected]>    
date     : Tue, 1 Jun 2021 14:25:19 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Tue, 1 Jun 2021 14:25:19 +0530    

Click here for diff

Release memory allocated when creating the tuple-conversion map and its  
component TupleDescs when its owning sync entry is invalidated.  
TupleDescs must also be freed when no map is deemed necessary, to begin  
with.  
  
Reported-by: Andres Freund  
Author: Amit Langote  
Reviewed-by: Takamichi Osumi, Amit Kapila  
Backpatch-through: 13, where it was introduced  
Discussion: https://postgr.es/m/MEYP282MB166933B1AB02B4FE56E82453B64D9@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM  

M src/backend/replication/pgoutput/pgoutput.c
M src/test/subscription/t/013_partition.pl

Add fallback implementation for setenv()

commit   : e2f21ff606dae2f7f7a9e323a88c99b464d2f787    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 1 Jun 2021 09:27:25 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 1 Jun 2021 09:27:25 +0900    

Click here for diff

This fixes the code compilation on Windows with MSVC and Kerberos, as  
a missing implementation of setenv() causes a compilation failure of the  
GSSAPI code.  This was only reproducible when building the code with  
Kerberos, something that buildfarm animal hamerkop has fixed recently.  
  
This issue only happens on 12 and 13, as this code has been introduced  
in b0b39f7.  HEAD is already able to compile properly thanks to  
7ca37fb0, and this commit is a minimal cherry-pick of it.  
  
Thanks to Tom Lane for the discussion.  
  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 12  

M configure
M configure.in
M src/include/pg_config.h.in
M src/include/port.h
M src/include/port/win32_port.h
A src/port/setenv.c
M src/tools/msvc/Mkvcbuild.pm
M src/tools/msvc/Solution.pm

Fix mis-planning of repeated application of a projection.

commit   : fe6f63286a00a49d0f82ac73fc252a3ad40d73b4    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 31 May 2021 12:03:00 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 31 May 2021 12:03:00 -0400    

Click here for diff

create_projection_plan contains a hidden assumption (here made  
explicit by an Assert) that a projection-capable Path will yield a  
projection-capable Plan.  Unfortunately, that assumption is violated  
only a few lines away, by create_projection_plan itself.  This means  
that two stacked ProjectionPaths can yield an outcome where we try to  
jam the upper path's tlist into a non-projection-capable child node,  
resulting in an invalid plan.  
  
There isn't any good reason to have stacked ProjectionPaths; indeed the  
whole concept is faulty, since the set of Vars/Aggs/etc needed by the  
upper one wouldn't necessarily be available in the output of the lower  
one, nor could the lower one create such values if they weren't  
available from its input.  Hence, we can fix this by adjusting  
create_projection_path to strip any top-level ProjectionPath from the  
subpath it's given.  (This amounts to saying "oh, we changed our  
minds about what we need to project here".)  
  
The test case added here only fails in v13 and HEAD; before that, we  
don't attempt to shove the Sort into the parallel part of the plan,  
for reasons that aren't entirely clear to me.  However, all the  
directly-related code looks generally the same as far back as v11,  
where the hazard was introduced (by d7c19e62a).  So I've got no faith  
that the same type of bug doesn't exist in v11 and v12, given the  
right test case.  Hence, back-patch the code changes, but not the  
irrelevant test case, into those branches.  
  
Per report from Bas Poot.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql

Raise a timeout to 180s, in test 010_logical_decoding_timelines.pl.

commit   : 23059a0457c0c19c149c964490a5ca95fe0625e0    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 31 May 2021 00:29:58 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 31 May 2021 00:29:58 -0700    

Click here for diff

Per buildfarm member hornet.  Also, update Pod documentation showing the  
lower value.  Back-patch to v10, where the test first appeared.  

M src/test/perl/PostgresNode.pm
M src/test/recovery/t/010_logical_decoding_timelines.pl

Fix race condition when sharing tuple descriptors.

commit   : d41fda6aa3c1016ef5aedb4bf393ead81f6e3a8f    
  
author   : Thomas Munro <[email protected]>    
date     : Sat, 29 May 2021 14:48:15 +1200    
  
committer: Thomas Munro <[email protected]>    
date     : Sat, 29 May 2021 14:48:15 +1200    

Click here for diff

Parallel query processes that called BlessTupleDesc() for identical  
tuple descriptors at the same moment could crash.  There was code to  
handle that rare case, but it dereferenced a bogus DSA pointer.  Repair.  
  
Back-patch to 11, where commit cc5f8136 added support for sharing tuple  
descriptors in parallel queries.  
  
Reported-by: Eric Thinnes <[email protected]>  
Discussion: https://postgr.es/m/99aaa2eb-e194-bf07-c29a-1a76b4f2bcf9%40gmx.de  

M src/backend/utils/cache/typcache.c

fix syntax error

commit   : bb18bc2249239fff10f84c276783962d546c858a    
  
author   : Andrew Dunstan <[email protected]>    
date     : Fri, 28 May 2021 09:35:11 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Fri, 28 May 2021 09:35:11 -0400    

Click here for diff

M src/tools/msvc/Solution.pm

Report configured port in MSVC built pg_config

commit   : c828a7246eed0bf92490660543a072cf3610441a    
  
author   : Andrew Dunstan <[email protected]>    
date     : Fri, 28 May 2021 09:26:30 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Fri, 28 May 2021 09:26:30 -0400    

Click here for diff

This is a long standing omission, discovered when trying to write code  
that relied on it.  
  
Backpatch to all live branches.  

M src/tools/msvc/Solution.pm

Fix MSVC scripts when building with GSSAPI/Kerberos

commit   : ab81d004e40178deecf0c3bcae177d43ce9926ab    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 27 May 2021 20:11:21 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 27 May 2021 20:11:21 +0900    

Click here for diff

The deliverables of upstream Kerberos on Windows are installed with  
paths that do not match our MSVC scripts.  First, the include folder was  
named "inc/" in our scripts, but the upstream MSIs use "include/".  
Second, the build would fail with 64-bit environments as the libraries  
are named differently.  
  
This commit adjusts the MSVC scripts to be compatible with the latest  
installations of upstream, and I have checked that the compilation was  
able to work with the 32-bit and 64-bit installations.  
  
Special thanks to Kondo Yuta for the help in investigating the situation  
in hamerkop, which had an incorrect configuration for the GSS  
compilation.  
  
Reported-by: Brian Ye  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/tools/msvc/Solution.pm

doc: Fix description of some GUCs in docs and postgresql.conf.sample

commit   : 0ab64ab318f23259210ff4d430ff1e7690ca3c08    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 27 May 2021 14:58:09 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 27 May 2021 14:58:09 +0900    

Click here for diff

The following parameters have been imprecise, or incorrect, about their  
description (PGC_POSTMASTER or PGC_SIGHUP):  
- autovacuum_work_mem (docs, as of 9.6~)  
- huge_page_size (docs, as of 14~)  
- max_logical_replication_workers (docs, as of 10~)  
- max_sync_workers_per_subscription (docs, as of 10~)  
- min_dynamic_shared_memory (docs, as of 14~)  
- recovery_init_sync_method (postgresql.conf.sample, as of 14~)  
- remove_temp_files_after_crash (docs, as of 14~)  
- restart_after_crash (docs, as of 9.6~)  
- ssl_min_protocol_version (docs, as of 12~)  
- ssl_max_protocol_version (docs, as of 12~)  
  
This commit adjusts the description of all these parameters to be more  
consistent with the practice used for the others.  
  
Revewed-by: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M doc/src/sgml/config.sgml

Improve docs and error messages for parallel vacuum.

commit   : 9012e5594c6ff803abb6d51e164797e3810845c1    
  
author   : Amit Kapila <[email protected]>    
date     : Tue, 25 May 2021 09:40:16 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Tue, 25 May 2021 09:40:16 +0530    

Click here for diff

The error messages, docs, and one of the options were using  
'parallel degree' to indicate parallelism used by vacuum command. We  
normally use 'parallel workers' at other places so change it for parallel  
vacuum accordingly.  
  
Author: Bharath Rupireddy  
Reviewed-by: Dilip Kumar, Amit Kapila  
Backpatch-through: 13  
Discussion: https://postgr.es/m/CALj2ACWz=PYrrFXVsEKb9J1aiX4raA+UBe02hdRp_zqDkrWUiw@mail.gmail.com  

M doc/src/sgml/ref/vacuumdb.sgml
M src/backend/commands/vacuum.c
M src/bin/scripts/vacuumdb.c
M src/test/regress/expected/vacuum.out

Disallow SSL renegotiation

commit   : a23c0b00f08a983216faf16c5d2e94fe963fa0d1    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 25 May 2021 10:11:13 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 25 May 2021 10:11:13 +0900    

Click here for diff

SSL renegotiation is already disabled as of 48d23c72, however this does  
not prevent the server to comply with a client willing to use  
renegotiation.  In the last couple of years, renegotiation had its set  
of security issues and flaws (like the recent CVE-2021-3449), and it  
could be possible to crash the backend with a client attempting  
renegotiation.  
  
This commit takes one extra step by disabling renegotiation in the  
backend in the same way as SSL compression (f9264d15) or tickets  
(97d3a0b0).  OpenSSL 1.1.0h has added an option named  
SSL_OP_NO_RENEGOTIATION able to achieve that.  In older versions  
there is an option called SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS that  
was undocumented, and could be set within the SSL object created when  
the TLS connection opens, but I have decided not to use it, as it feels  
trickier to rely on, and it is not official.  Note that this option is  
not usable in OpenSSL < 1.1.0h as the internal contents of the *SSL  
object are hidden to applications.  
  
SSL renegotiation concerns protocols up to TLSv1.2.  
  
Per original report from Robert Haas, with a patch based on a suggestion  
by Andres Freund.  
  
Author: Michael Paquier  
Reviewed-by: Daniel Gustafsson  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/backend/libpq/be-secure-openssl.c

Fix access to no-longer-open relcache entry in logical-rep worker.

commit   : 5b4791b4580e95a1230e89c11c08058f18b92225    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 22 May 2021 21:24:48 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 22 May 2021 21:24:48 -0400    

Click here for diff

If we redirected a replicated tuple operation into a partition child  
table, and then tried to fire AFTER triggers for that event, the  
relation cache entry for the child table was already closed.  This has  
no visible ill effects as long as the entry is still there and still  
valid, but an unluckily-timed cache flush could result in a crash or  
other misbehavior.  
  
To fix, postpone the ExecCleanupTupleRouting call (which is what  
closes the child table) until after we've fired triggers.  This  
requires a bit of refactoring so that the cleanup function can  
have access to the necessary state.  
  
In HEAD, I took the opportunity to simplify some of worker.c's  
function APIs based on use of the new ApplyExecutionData struct.  
However, it doesn't seem safe/practical to back-patch that aspect,  
at least not without a lot of analysis of possible interactions  
with a04daa97a.  
  
In passing, add an Assert to afterTriggerInvokeEvents to catch  
such cases.  This seems worthwhile because we've grown a number  
of fairly unstructured ways of calling AfterTriggerEndQuery.  
  
Back-patch to v13, where worker.c grew the ability to deal with  
partitioned target tables.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/trigger.c
M src/backend/replication/logical/worker.c

Disallow whole-row variables in GENERATED expressions.

commit   : 849c7971d1abe39d9105a768918c110414029e48    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 21 May 2021 15:12:08 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 21 May 2021 15:12:08 -0400    

Click here for diff

This was previously allowed, but I think that was just an oversight.  
It's a clear violation of the rule that a generated column cannot  
depend on itself or other generated columns.  Moreover, because the  
code was relying on the assumption that no such cross-references  
exist, it was pretty easy to crash ALTER TABLE and perhaps other  
places.  Even if you managed not to crash, you got quite unstable,  
implementation-dependent results.  
  
Per report from Vitaly Ustinov.  
Back-patch to v12 where GENERATED came in.  
  
Discussion: https://postgr.es/m/CAM_DEiWR2DPT6U4xb-Ehigozzd3n3G37ZB1+867zbsEVtYoJww@mail.gmail.com  

M src/backend/catalog/heap.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql

Fix usage of "tableoid" in GENERATED expressions.

commit   : 77e3204ecbf15ab5dfd295bbc66eeeec4d9ade19    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 21 May 2021 15:02:07 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 21 May 2021 15:02:07 -0400    

Click here for diff

We consider this supported (though I've got my doubts that it's a  
good idea, because tableoid is not immutable).  However, several  
code paths failed to fill the field in soon enough, causing such  
a GENERATED expression to see zero or the wrong value.  This  
occurred when ALTER TABLE adds a new GENERATED column to a table  
with existing rows, and during regular INSERT or UPDATE on a  
foreign table with GENERATED columns.  
  
Noted during investigation of a report from Vitaly Ustinov.  
Back-patch to v12 where GENERATED came in.  
  
Discussion: https://postgr.es/m/CAM_DEiWR2DPT6U4xb-Ehigozzd3n3G37ZB1+867zbsEVtYoJww@mail.gmail.com  

M src/backend/commands/tablecmds.c
M src/backend/executor/nodeModifyTable.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql

Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.

commit   : d18ee6f92d9a22b4fae57f515797b2196bf385c7    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 21 May 2021 14:03:53 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 21 May 2021 14:03:53 -0400    

Click here for diff

COMMIT/ROLLBACK necessarily destroys all snapshots within the session.  
The original implementation of intra-procedure transactions just  
cavalierly did that, ignoring the fact that this left us executing in  
a rather different environment than normal.  In particular, it turns  
out that handling of toasted datums depends rather critically on there  
being an outer ActiveSnapshot: otherwise, when SPI or the core  
executor pop whatever snapshot they used and return, it's unsafe to  
dereference any toasted datums that may appear in the query result.  
It's possible to demonstrate "no known snapshots" and "missing chunk  
number N for toast value" errors as a result of this oversight.  
  
Historically this outer snapshot has been held by the Portal code,  
and that seems like a good plan to preserve.  So add infrastructure  
to pquery.c to allow re-establishing the Portal-owned snapshot if it's  
not there anymore, and add enough bookkeeping support that we can tell  
whether it is or not.  
  
We can't, however, just re-establish the Portal snapshot as part of  
COMMIT/ROLLBACK.  As in normal transaction start, acquiring the first  
snapshot should wait until after SET and LOCK commands.  Hence, teach  
spi.c about doing this at the right time.  (Note that this patch  
doesn't fix the problem for any PLs that try to run intra-procedure  
transactions without using SPI to execute SQL commands.)  
  
This makes SPI's no_snapshots parameter rather a misnomer, so in HEAD,  
rename that to allow_nonatomic.  
  
replication/logical/worker.c also needs some fixes, because it wasn't  
careful to hold a snapshot open around AFTER trigger execution.  
That code doesn't use a Portal, which I suspect someday we're gonna  
have to fix.  But for now, just rearrange the order of operations.  
This includes back-patching the recent addition of finish_estate()  
to centralize the cleanup logic there.  
  
This also back-patches commit 2ecfeda3e into v13, to improve the  
test coverage for worker.c (it was that test that exposed that  
worker.c's snapshot management is wrong).  
  
Per bug #15990 from Andreas Wicht.  Back-patch to v11 where  
intra-procedure COMMIT was added.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/functioncmds.c
M src/backend/executor/spi.c
M src/backend/replication/logical/worker.c
M src/backend/tcop/pquery.c
M src/backend/utils/mmgr/portalmem.c
M src/include/executor/spi_priv.h
M src/include/tcop/pquery.h
M src/include/utils/portal.h
M src/pl/plpgsql/src/pl_exec.c
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/specs/plpgsql-toast.spec
M src/test/subscription/t/013_partition.pl

Fix deadlock for multiple replicating truncates of the same table.

commit   : c83c0257e4c3ec3503f8d72698d70829af979803    
  
author   : Amit Kapila <[email protected]>    
date     : Fri, 21 May 2021 08:03:38 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Fri, 21 May 2021 08:03:38 +0530    

Click here for diff

While applying the truncate change, the logical apply worker acquires  
RowExclusiveLock on the relation being truncated. This allowed truncate on  
the relation at a time by two apply workers which lead to a deadlock. The  
reason was that one of the workers after updating the pg_class tuple tries  
to acquire SHARE lock on the relation and started to wait for the second  
worker which has acquired RowExclusiveLock on the relation. And when the  
second worker tries to update the pg_class tuple, it starts to wait for  
the first worker which leads to a deadlock. Fix it by acquiring  
AccessExclusiveLock on the relation before applying the truncate change as  
we do for normal truncate operation.  
  
Author: Peter Smith, test case by Haiying Tang  
Reviewed-by: Dilip Kumar, Amit Kapila  
Backpatch-through: 11  
Discussion: https://postgr.es/m/CAHut+PsNm43p0jM+idTvWwiGZPcP0hGrHMPK9TOAkc+a4UpUqw@mail.gmail.com  

M src/backend/replication/logical/worker.c
M src/test/subscription/t/010_truncate.pl

Avoid detoasting failure after COMMIT inside a plpgsql FOR loop.

commit   : c64183f234e8e185347edf8820b3459e802b5354    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 20 May 2021 18:32:37 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 20 May 2021 18:32:37 -0400    

Click here for diff

exec_for_query() normally tries to prefetch a few rows at a time  
from the query being iterated over, so as to reduce executor  
entry/exit overhead.  Unfortunately this is unsafe if we have  
COMMIT or ROLLBACK within the loop, because there might be  
TOAST references in the data that we prefetched but haven't  
yet examined.  Immediately after the COMMIT/ROLLBACK, we have  
no snapshots in the session, meaning that VACUUM is at liberty  
to remove recently-deleted TOAST rows.  
  
This was originally reported as a case triggering the "no known  
snapshots" error in init_toast_snapshot(), but even if you miss  
hitting that, you can get "missing toast chunk", as illustrated  
by the added isolation test case.  
  
To fix, just disable prefetching in non-atomic contexts.  Maybe  
there will be performance complaints prompting us to work harder  
later, but it's not clear at the moment that this really costs  
much, and I doubt we'd want to back-patch any complicated fix.  
  
In passing, adjust that error message in init_toast_snapshot()  
to be a little clearer about the likely cause of the problem.  
  
Patch by me, based on earlier investigation by Konstantin Knizhnik.  
  
Per bug #15990 from Andreas Wicht.  Back-patch to v11 where  
intra-procedure COMMIT was added.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/common/toast_internals.c
M src/pl/plpgsql/src/pl_exec.c
M src/test/isolation/expected/plpgsql-toast.out
M src/test/isolation/specs/plpgsql-toast.spec

Clean up cpluspluscheck violation.

commit   : 9dc76f032cd6a1fbde760aa9c264057fec7b1855    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 20 May 2021 13:03:08 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 20 May 2021 13:03:08 -0400    

Click here for diff

"typename" is a C++ keyword, so pg_upgrade.h fails to compile in C++.  
Fortunately, there seems no likely reason for somebody to need to  
do that.  Nonetheless, it's project policy that all .h files should  
pass cpluspluscheck, so rename the argument to fix that.  
  
Oversight in 57c081de0; back-patch as that was.  (The policy requiring  
pg_upgrade.h to pass cpluspluscheck only goes back to v12, but it  
seems best to keep this code looking the same in all branches.)  

M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c

Fix typo and outdated information in README.barrier

commit   : 010429eb1f77a6221f6c89b21b6b0c7820446201    
  
author   : David Rowley <[email protected]>    
date     : Tue, 18 May 2021 09:55:54 +1200    
  
committer: David Rowley <[email protected]>    
date     : Tue, 18 May 2021 09:55:54 +1200    

Click here for diff

README.barrier didn't seem to get the memo when atomics were added. Fix  
that.  
  
Author: Tatsuo Ishii, David Rowley  
Discussion: https://postgr.es/m/20210516.211133.2159010194908437625.t-ishii%40sraoss.co.jp  
Backpatch-through: 9.6, oldest supported release  

M src/backend/storage/lmgr/README.barrier

Be more careful about barriers when releasing BackgroundWorkerSlots.

commit   : c3cc73e144246c253442aa2aa031749d57eb710b    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 15 May 2021 12:21:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 15 May 2021 12:21:06 -0400    

Click here for diff

ForgetBackgroundWorker lacked any memory barrier at all, while  
BackgroundWorkerStateChange had one but unaccountably did  
additional manipulation of the slot after the barrier.  AFAICS,  
the rule must be that the barrier is immediately before setting  
or clearing slot->in_use.  
  
It looks like back in 9.6 when ForgetBackgroundWorker was first  
written, there might have been some case for not needing a  
barrier there, but I'm not very convinced of that --- the fact  
that the load of bgw_notify_pid is in the caller doesn't seem  
to guarantee no memory ordering problem.  So patch 9.6 too.  
  
It's likely that this doesn't fix any observable bug on Intel  
hardware, but machines with weaker memory ordering rules could  
have problems here.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/postmaster/bgworker.c

Harden nbtree deduplication posting split code.

commit   : fa675af59fc828d0b71bd9139042d71456640a28    
  
author   : Peter Geoghegan <[email protected]>    
date     : Fri, 14 May 2021 15:08:00 -0700    
  
committer: Peter Geoghegan <[email protected]>    
date     : Fri, 14 May 2021 15:08:00 -0700    

Click here for diff

Add a defensive "can't happen" error to code that handles nbtree posting  
list splits (promote an existing assertion).  This avoids a segfault in  
the event of an insertion of a newitem that is somehow identical to an  
existing non-pivot tuple in the index.  An nbtree index should never  
have two index tuples with identical TIDs.  
  
This scenario is not particular unlikely in the event of any kind of  
corruption that leaves the index in an inconsistent state relative to  
the heap relation that is indexed.  There are two known reports of  
preventable hard crashes.  Doing nothing seems unacceptable given the  
general expectation that nbtree will cope reasonably well with corrupt  
data.  
  
Discussion: https://postgr.es/m/CAH2-Wz=Jr_d-dOYEEmwz0-ifojVNWho01eAqewfQXgKfoe114w@mail.gmail.com  
Backpatch: 13-, where nbtree deduplication was introduced.  

M src/backend/access/nbtree/nbtdedup.c

Doc: correct erroneous entry in this week's minor release notes.

commit   : 6a4c07156802fa2efee75c273a7d5d9ae81bda28    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 17:36:20 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 17:36:20 -0400    

Click here for diff

The patch to disallow a NULL specification in combination with  
GENERATED ... AS IDENTITY applied to both ALWAYS and BY DEFAULT  
variants of that clause, not only the former.  
  
Noted by Shay Rojansky.  
  
Discussion: https://postgr.es/m/CADT4RqAwD3A=RvGiQU9AiTK-6VeuXcycwPHmJPv_OBCJFYOEww@mail.gmail.com  

M doc/src/sgml/release-13.sgml

Prevent infinite insertion loops in spgdoinsert().

commit   : dc714c120eab6086721e3b0bb7a5a157008ce01b    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 15:07:34 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 15:07:34 -0400    

Click here for diff

Formerly we just relied on operator classes that assert longValuesOK  
to eventually shorten the leaf value enough to fit on an index page.  
That fails since the introduction of INCLUDE-column support (commit  
09c1c6ab4), because the INCLUDE columns might alone take up more  
than a page, meaning no amount of leaf-datum compaction will get  
the job done.  At least with spgtextproc.c, that leads to an infinite  
loop, since spgtextproc.c won't throw an error for not being able  
to shorten the leaf datum anymore.  
  
To fix without breaking cases that would otherwise work, add logic  
to spgdoinsert() to verify that the leaf tuple size is decreasing  
after each "choose" step.  Some opclasses might not decrease the  
size on every single cycle, and in any case, alignment roundoff  
of the tuple size could obscure small gains.  Therefore, allow  
up to 10 cycles without additional savings before throwing an  
error.  (Perhaps this number will need adjustment, but it seems  
quite generous right now.)  
  
As long as we've developed this logic, let's back-patch it.  
The back branches don't have INCLUDE columns to worry about, but  
this seems like a good defense against possible bugs in operator  
classes.  We already know that an infinite loop here is pretty  
unpleasant, so having a defense seems to outweigh the risk of  
breaking things.  (Note that spgtextproc.c is actually the only  
known opclass with longValuesOK support, so that this is all moot  
for known non-core opclasses anyway.)  
  
Per report from Dilip Kumar.  
  
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com  

M doc/src/sgml/spgist.sgml
M src/backend/access/spgist/spgdoinsert.c

Fix query-cancel handling in spgdoinsert().

commit   : c1b72bf04515c118f34b54816b47dcd2b9cff577    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 13:26:55 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 13:26:55 -0400    

Click here for diff

Knowing that a buggy opclass could cause an infinite insertion loop,  
spgdoinsert() intended to allow its loop to be interrupted by query  
cancel.  However, that never actually worked, because in iterations  
after the first, we'd be holding buffer lock(s) which would cause  
InterruptHoldoffCount to be positive, preventing servicing of the  
interrupt.  
  
To fix, check if an interrupt is pending, and if so fall out of  
the insertion loop and service the interrupt after we've released  
the buffers.  If it was indeed a query cancel, that's the end of  
the matter.  If it was a non-canceling interrupt reason, make use  
of the existing provision to retry the whole insertion.  (This isn't  
as wasteful as it might seem, since any upper-level index tuples we  
already created should be usable in the next attempt.)  
  
While there's no known instance of such a bug in existing release  
branches, it still seems like a good idea to back-patch this to  
all supported branches, since the behavior is fairly nasty if a  
loop does happen --- not only is it uncancelable, but it will  
quickly consume memory to the point of an OOM failure.  In any  
case, this code is certainly not working as intended.  
  
Per report from Dilip Kumar.  
  
Discussion: https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com  

M src/backend/access/spgist/spgdoinsert.c

Refactor CHECK_FOR_INTERRUPTS() to add flexibility.

commit   : 63831c16275e5f59fff9bbb8bca9500e2a5797f0    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 12:54:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 14 May 2021 12:54:26 -0400    

Click here for diff

Split up CHECK_FOR_INTERRUPTS() to provide an additional macro  
INTERRUPTS_PENDING_CONDITION(), which just tests whether an  
interrupt is pending without attempting to service it.  This is  
useful in situations where the caller knows that interrupts are  
blocked, and would like to find out if it's worth the trouble  
to unblock them.  
  
Also add INTERRUPTS_CAN_BE_PROCESSED(), which indicates whether  
CHECK_FOR_INTERRUPTS() can be relied on to clear the pending interrupt.  
  
This commit doesn't actually add any uses of the new macros,  
but a follow-on bug fix will do so.  Back-patch to all supported  
branches to provide infrastructure for that fix.  
  
Alvaro Herrera and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/tcop/postgres.c
M src/include/miscadmin.h

Improve documentation example for jsonpath like_regex operator

commit   : ff91d3a22b9a5ab9511f8ed8bffc895a991529f5    
  
author   : Alexander Korotkov <[email protected]>    
date     : Thu, 13 May 2021 16:10:21 +0300    
  
committer: Alexander Korotkov <[email protected]>    
date     : Thu, 13 May 2021 16:10:21 +0300    

Click here for diff

Make sample like_regex match string values of the root object instead of the  
whole document.  The corrected example seems to represent a more relevant  
use case.  
  
Backpatch to 12, when jsonpath was introduced.  
  
Discussion: https://postgr.es/m/13440f8b-4c1f-5875-c8e3-f3f65606af2f%40xs4all.nl  
Author: Erik Rijkers  
Reviewed-by: Michael Paquier, Alexander Korotkov  
Backpatch-through: 12  

M doc/src/sgml/func.sgml

Rename the logical replication global "wrconn"

commit   : e5c071bc0f93c9c47821c876e36459580e08a68c    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 12 May 2021 19:13:54 -0400    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 12 May 2021 19:13:54 -0400    

Click here for diff

The worker.c global wrconn is only meant to be used by logical apply/  
tablesync workers, but there are other variables with the same name. To  
reduce future confusion rename the global from "wrconn" to  
"LogRepWorkerWalRcvConn".  
  
While this is just cosmetic, it seems better to backpatch it all the way  
back to 10 where this code appeared, to avoid future backpatching  
issues.  
  
Author: Peter Smith <[email protected]>  
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com  

M src/backend/replication/logical/launcher.c
M src/backend/replication/logical/tablesync.c
M src/backend/replication/logical/worker.c
M src/include/replication/worker_internal.h

Reduce runtime of privileges.sql test under CLOBBER_CACHE_ALWAYS.

commit   : 834d9284b4fee413c74de3a143a072eb21c05772    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 11 May 2021 20:59:45 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 11 May 2021 20:59:45 -0400    

Click here for diff

Several queries in the privileges regression test cause the planner  
to apply the plpgsql function "leak()" to every element of the  
histogram for atest12.b.  Since commit 0c882e52a increased the size  
of that histogram to 10000 entries, the test invokes that function  
over 100000 times, which takes an absolutely unreasonable amount of  
time in clobber-cache-always mode.  
  
However, there's no real reason why that has to be a plpgsql  
function: for the purposes of this test, all that matters is that  
it not be marked leakproof.  So we can replace the plpgsql  
implementation with a direct call of int4lt, which has the same  
behavior and is orders of magnitude faster.  This is expected to  
cut several hours off the buildfarm cycle time for CCA animals.  
It has some positive impact in normal builds too, though that's  
probably lost in the noise.  
  
Back-patch to v13 where 0c882e52a came in.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql