PostgreSQL 12.5 (upcoming) commit log

Use factorial rather than numeric_fac in create_operator.sql.

commit   : 1af91dc032cfc3f1e47d3bbb976ad6c880668bad    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 18 Sep 2020 18:03:44 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 18 Sep 2020 18:03:44 -0400    

Click here for diff

These two SQL functions are aliases for the same C function, so this  
change has no semantic effect.  However, because we dropped the  
numeric_fac alias in HEAD (commit 76f412ab3), operator definitions  
based on that one don't port forward, causing problems for cross-version  
upgrade tests based on the regression database.  
  
Patch all active back branches to dodge the problem.  
  
Discussion: https://postgr.es/m/449144.1600439950@sss.pgh.pa.us  

M src/test/regress/expected/create_operator.out
M src/test/regress/sql/create_operator.sql

Update parallel BTree scan state when the scan keys can't be satisfied.

commit   : 4bc63462d9d8dd12a5564c3d4ca06c99449d2d07    
  
author   : Amit Kapila <akapila@postgresql.org>    
date     : Thu, 17 Sep 2020 15:16:46 +0530    
  
committer: Amit Kapila <akapila@postgresql.org>    
date     : Thu, 17 Sep 2020 15:16:46 +0530    

Click here for diff

For parallel btree scan to work for array of scan keys, it should reach  
BTPARALLEL_DONE state once for every distinct combination of array keys.  
This is required to ensure that the parallel workers don't try to seize  
blocks at the same time for different scan keys. We missed to update this  
state when we discovered that the scan keys can't be satisfied.  
  
Author: James Hunter  
Reviewed-by: Amit Kapila  
Tested-by: Justin Pryzby  
Backpatch-through: 10, where it was introduced  
Discussion: https://postgr.es/m/4248CABC-25E3-4809-B4D0-128E1BAABC3C@amazon.com  

M src/backend/access/nbtree/nbtsearch.c

Avoid unnecessary recursion to child tables in ALTER TABLE SET NOT NULL.

commit   : 511690ec5dccd7c93573367c2920e8f0919c901b    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 16 Sep 2020 13:38:26 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 16 Sep 2020 13:38:26 -0400    

Click here for diff

If a partitioned table's column is already marked NOT NULL, there is  
no need to examine its partitions, because we can rely on previous  
DDL to have enforced that the child columns are NOT NULL as well.  
(Unfortunately, the same cannot be said for traditional inheritance,  
so for now we have to restrict the optimization to partitioned tables.)  
Hence, we may skip recursing to child tables in this situation.  
  
The reason this case is worth worrying about is that when pg_dump dumps  
a partitioned table having a primary key, it will include the requisite  
NOT NULL markings in the CREATE TABLE commands, and then add the  
primary key as a separate step.  The primary key addition generates a  
SET NOT NULL as a subcommand, just to be sure.  So the situation where  
a SET NOT NULL is redundant does arise in the real world.  
  
Skipping the recursion does more than just save a few cycles: it means  
that a command such as "ALTER TABLE ONLY partition_parent ADD PRIMARY  
KEY" will take locks only on the partition parent table, not on the  
partitions.  It turns out that parallel pg_restore is effectively  
assuming that that's true, and has little choice but to do so because  
the dependencies listed for such a TOC entry don't include the  
partitions.  pg_restore could thus issue this ALTER while data restores  
on the partitions are still in progress.  Taking unnecessary locks on  
the partitions not only hurts concurrency, but can lead to actual  
deadlock failures, as reported by Domagoj Smoljanovic.  
  
(A contributing factor in the deadlock is that TRUNCATE on a child  
partition wants a non-exclusive lock on the parent.  This seems  
likewise unnecessary, but the fix for it is more invasive so we  
won't consider back-patching it.  Fortunately, getting rid of one  
of these two poor behaviors is enough to remove the deadlock.)  
  
Although support for partitioned primary keys came in with v11,  
this patch is dependent on the SET NOT NULL refactoring done by  
commit f4a3fdfbd, so we can only patch back to v12.  
  
Patch by me; thanks to Alvaro Herrera and Amit Langote for review.  
  
Discussion: https://postgr.es/m/VI1PR03MB31670CA1BD9625C3A8C5DD05EB230@VI1PR03MB3167.eurprd03.prod.outlook.com  

M src/backend/commands/tablecmds.c

Fix bogus cache-invalidation logic in logical replication worker.

commit   : 8580631ff77c5bb29d0b9fed8a38ee461872e86a    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 16 Sep 2020 12:07:31 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 16 Sep 2020 12:07:31 -0400    

Click here for diff

The code recorded cache invalidation events by zeroing the "localreloid"  
field of affected cache entries.  However, it's possible for an inval  
event to occur even while we have the entry open and locked.  So an  
ill-timed inval could result in "cache lookup failed for relation 0"  
errors, if the worker's code tried to use the cleared field.  We can  
fix that by creating a separate bool field to record whether the entry  
needs to be revalidated.  (In the back branches, cram the bool into  
what had been padding space, to avoid an ABI break in the somewhat  
unlikely event that any extension is looking at this struct.)  
  
Also, rearrange the logic in logicalrep_rel_open so that it  
does the right thing in cases where table_open would fail.  
We should retry the lookup by name in that case, but we didn't.  
  
The real-world impact of this is probably small.  In the first place,  
the error conditions are very low probability, and in the second place,  
the worker would just exit and get restarted.  We only noticed because  
in a CLOBBER_CACHE_ALWAYS build, the failure can occur repeatedly,  
preventing the worker from making progress.  Nonetheless, it's clearly  
a bug, and it impedes a useful type of testing; so back-patch to v10  
where this code was introduced.  
  
Discussion: https://postgr.es/m/1032727.1600096803@sss.pgh.pa.us  

M src/backend/replication/logical/relation.c
M src/include/replication/logicalrelation.h

Fix interpolation in test name.

commit   : af2d09fa1c28216286649d519f4aca3ebbd5399a    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sun, 13 Sep 2020 23:29:51 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sun, 13 Sep 2020 23:29:51 -0700    

Click here for diff

A pre-commit review had reported the problem, but the fix reached only  
v10 and earlier.  Back-patch to v11.  
  
Discussion: https://postgr.es/m/20200423.140546.1055476118690602079.horikyota.ntt@gmail.com  

M src/test/recovery/t/020_archive_status.pl

Use the properly transformed RangeVar for expandTableLikeClause().

commit   : 1371a1e4161a25a128b666109ac5a9d449da982a    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 13 Sep 2020 12:51:21 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 13 Sep 2020 12:51:21 -0400    

Click here for diff

transformCreateStmt() adjusts the transformed statement's RangeVar  
to specify the target schema explicitly, for the express reason  
of making sure that auxiliary statements derived by parse  
transformation operate on the right table.  But the refactoring  
I did in commit 502898192 got this wrong and passed the untransformed  
RangeVar to expandTableLikeClause().  This could lead to assertion  
failures or weird misbehavior if the wrong table was accessed.  
  
Per report from Alexander Lakhin.  Like the previous patch, back-patch  
to all supported branches.  
  
Discussion: https://postgr.es/m/05051f9d-b32b-cb35-6735-0e9f2ab86b5f@gmail.com  

M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql

commit   : 6a8f6aee0cbea617df90fe3b715d6dd73e0f8c04    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Sun, 6 Sep 2020 16:46:13 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Sun, 6 Sep 2020 16:46:13 +0200    

Click here for diff

The original stylesheets seemed to think this was a good idea, but our  
users find it confusing and unhelpful, so undo that logic.  
  
Reported-by: Fabien COELHO <coelho@cri.ensmp.fr>  
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.22.394.2006210914370.859381%40pseudo  

M doc/src/sgml/stylesheet.xsl

Use _exit(2) for SIGQUIT during ProcessStartupPacket, too.

commit   : 4e10c0c8a6690d9b950a0ab1633de6c02d1a1069    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 10 Sep 2020 12:06:26 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 10 Sep 2020 12:06:26 -0400    

Click here for diff

Bring the signal handling for startup-packet collection into line  
with the policy established in commits bedadc732 and 8e19a8264,  
namely don't risk running atexit callbacks when handling SIGQUIT.  
  
Ideally, we'd not do so for SIGTERM or timeout interrupts either,  
but that change seems a bit too risky for the back branches.  
For now, just improve the comments in this area to describe the risk.  
  
Also relocate where BackendInitialize re-disables these interrupts,  
to minimize the code span where they're active.  This doesn't buy  
a whole lot of safety, but it can't hurt.  
  
In passing, rename startup_die() to remove confusion about whether  
it is for the startup process.  
  
Like the previous commits, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/1850884.1599601164@sss.pgh.pa.us  

M src/backend/postmaster/postmaster.c

doc: Remove buggy ICU collation from documentation

commit   : 697c8e620443ca15b834f46e66d682dc9ce5c191    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Thu, 10 Sep 2020 15:31:09 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Thu, 10 Sep 2020 15:31:09 +0200    

Click here for diff

We have had multiple reports that point to the  
'@colReorder=latn-digit' collation customization being buggy.  We have  
reported this to ICU and are waiting for a fix.  In the meantime,  
remove references to this from the documentation and replace it by  
another reordering example.  Apparently, many users have been picking  
up this example specifically from the documentation.  
  
Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>  
Discussion: https://www.postgresql.org/message-id/flat/153201618542.1404.3611626898935613264%40wrigleys.postgresql.org  

M doc/src/sgml/charset.sgml

Fix title in reference section

commit   : 059872cea147617f688d314664df99d73a9c407e    
  
author   : Magnus Hagander <magnus@hagander.net>    
date     : Thu, 10 Sep 2020 14:15:26 +0200    
  
committer: Magnus Hagander <magnus@hagander.net>    
date     : Thu, 10 Sep 2020 14:15:26 +0200    

Click here for diff

Reported-by: Robert Kahlert  
Author: Daniel Gustafsson  

M doc/src/sgml/biblio.sgml

doc: Fix some grammar and inconsistencies

commit   : 25ff747721a5f4d386127528f50fd81d6cea365a    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Thu, 10 Sep 2020 15:50:46 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Thu, 10 Sep 2020 15:50:46 +0900    

Click here for diff

Some comments are fixed while on it.  
  
Author: Justin Pryzby  
Discussion: https://postgr.es/m/20200818171702.GK17022@telsasoft.com  
Backpatch-through: 9.6  

M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_subscription.sgml
M doc/src/sgml/sources.sgml
M src/backend/storage/lmgr/proc.c

Make archiver's SIGQUIT handler exit via _exit().

commit   : d038c6c6318b1959640a5a4b0f25cd577ebffbdf    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 9 Sep 2020 15:32:34 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 9 Sep 2020 15:32:34 -0400    

Click here for diff

Commit 8e19a8264 changed the SIGQUIT handlers of almost all server  
processes not to run atexit callbacks.  The archiver process was  
skipped, perhaps because it's not connected to shared memory; but  
it's just as true here that running atexit callbacks in a signal  
handler is unsafe.  So let's make it work like the rest.  
  
In HEAD and v13, we can use the common SignalHandlerForCrashExit  
handler.  Before that, just tweak pgarch_exit to use _exit(2)  
explicitly.  
  
Like the previous commit, back-patch to all supported branches.  
  
Kyotaro Horiguchi, back-patching by me  
  
Discussion: https://postgr.es/m/1850884.1599601164@sss.pgh.pa.us  

M src/backend/postmaster/pgarch.c

Check default partitions constraints while descending

commit   : ef1e1250e716056a240ecabc6f2a91c5956ed6c8    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 8 Sep 2020 19:35:15 -0300    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 8 Sep 2020 19:35:15 -0300    

Click here for diff

Partitioning tuple route code assumes that the partition chosen while  
descending the partition hierarchy is always the correct one.  This is  
true except when the partition is the default partition and another  
partition has been added concurrently: the partition constraint changes  
and we don't recheck it.  This can lead to tuples mistakenly being added  
to the default partition that should have been rejected.  
  
Fix by rechecking the default partition constraint while descending the  
hierarchy.  
  
An isolation test based on the reproduction steps described by Hao Wu  
(with tweaks for extra coverage) is included.  
  
Backpatch to 12, where this bug came in with 898e5e3290a7.  
  
Reported by: Hao Wu <hawu@vmware.com>  
Author: Amit Langote <amitlangote09@gmail.com>  
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>  
Discussion: https://postgr.es/m/CA+HiwqFqBmcSSap4sFnCBUEL_VfOMmEKaQ3gwUhyfa4c7J_-nA@mail.gmail.com  
Discussion: https://postgr.es/m/DM5PR0501MB3910E97A9EDFB4C775CF3D75A42F0@DM5PR0501MB3910.namprd05.prod.outlook.com  

M src/backend/executor/execPartition.c
A src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/partition-concurrent-attach.spec

doc: Tweak sentence for pg_checksums when enabling checksums

commit   : 87483c7bb32059f203a3fa36b4ffa9070970fa03    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 7 Sep 2020 14:35:17 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 7 Sep 2020 14:35:17 +0900    

Click here for diff

The previous version of the docs mentioned that files are rewritten,  
implying that a second copy of each file gets created, but each file is  
updated in-place.  
  
Author: Michael Banck  
Reviewed-by: Daniel Gustafsson, Michael Paquier  
Discussion: https://postgr.es/m/858086b6a42fb7d17995b6175856f7e7ec44d0a2.camel@credativ.de  
Backpatch-through: 12  

M doc/src/sgml/ref/pg_checksums.sgml

Fix misleading error message about inconsistent moving-aggregate types.

commit   : f45dd3fed9cf7d336af08b53a1f73bf90c88fb1a    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 6 Sep 2020 12:55:13 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 6 Sep 2020 12:55:13 -0400    

Click here for diff

We reported the wrong types when complaining that an aggregate's  
moving-aggregate implementation is inconsistent with its regular  
implementation.  
  
This was wrong since the feature was introduced, so back-patch  
to all supported branches.  
  
Jeff Janes  
  
Discussion: https://postgr.es/m/CAMkU=1x808LH=LPhZp9mNSP0Xd1xDqEd+XeGcvEe48dfE6xV=A@mail.gmail.com  

M src/backend/catalog/pg_aggregate.c

Remove useless lstat() call in pg_rewind.

commit   : a7bcf391f329561307a3d936443a715ad74a2e9d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 6 Sep 2020 11:50:41 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 6 Sep 2020 11:50:41 -0400    

Click here for diff

This is duplicative of an lstat that was just done by the calling  
function (traverse_datadir), besides which we weren't really doing  
anything with the results.  There's not much point in checking to  
see if someone removed the file since the previous lstat, since the  
FILE_ACTION_REMOVE code would have to deal with missing-file cases  
anyway.  Moreover, the "exists = false" assignment was a dead store;  
nothing was done with that value later.  
  
A syscall saved is a syscall earned, so back-patch to 9.5  
where this code was introduced.  
  
Discussion: https://postgr.es/m/1221796.1599329320@sss.pgh.pa.us  

M src/bin/pg_rewind/filemap.c

Make new authentication test case more robust.

commit   : fc37c6f6105a594467eb8922eb6d47e0d156ab21    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 4 Sep 2020 21:01:59 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 4 Sep 2020 21:01:59 -0400    

Click here for diff

I happened to notice that the new test case I added in b55b4dad9  
falls over if one runs "make check" repeatedly; though not in branches  
after v10.  That's because it was assuming that tmp_check/pgpass  
wouldn't exist already.  However, it's only been since v11 that the  
Makefiles forcibly remove all of tmp_check/ before starting a TAP run.  
This fix to unlink the file is therefore strictly necessary only in  
v10 ... but it seems wisest to do it across the board, rather than  
let the test rely on external logic to get the conditions right.  

M src/test/authentication/t/001_password.pl

Fix over-eager ping'ing in logical replication receiver.

commit   : 9b47ee6e7cba30701b4e11b872455461460650c4    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 4 Sep 2020 20:20:05 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 4 Sep 2020 20:20:05 -0400    

Click here for diff

Commit 3f60f690f only partially fixed the broken-status-tracking  
issue in LogicalRepApplyLoop: we need ping_sent to have the same  
lifetime as last_recv_timestamp.  The effects are much less serious  
than what that commit fixed, though.  AFAICS this would just lead to  
extra ping requests being sent, once per second until the sender  
responds.  Still, it's a bug, so backpatch to v10 as before.  
  
Discussion: https://postgr.es/m/959627.1599248476@sss.pgh.pa.us  

M src/backend/replication/logical/worker.c

Collect attribute data on extension owned tables being dumped

commit   : 616110eac3b5d939954391c5e9330ce9432502e4    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Fri, 4 Sep 2020 13:53:09 -0400    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Fri, 4 Sep 2020 13:53:09 -0400    

Click here for diff

If this data is not collected, pg_dump segfaults if asked for column  
inserts.  
  
Fix by Fabrízio de Royes Mello  
  
Backpatch to release 12 where the bug was introduced.  

M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql

C comment: correct use of 64-"byte" cache line size

commit   : 56f42cf9063d3ddf89962857d30a15d7ac49ce97    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 4 Sep 2020 13:27:52 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 4 Sep 2020 13:27:52 -0400    

Click here for diff

Reported-by: Kelly Min  
  
Discussion: https://postgr.es/m/CAPSbxatOiQO90LYpSC3+svAU9-sHgDfEP4oFhcEUt_X=DqFA9g@mail.gmail.com  
  
Backpatch-through: 9.5  

M src/include/storage/buf_internals.h

Fix rare deadlock failure in create_am regression test.

commit   : aa4eeb38f3aa3e4ebde4abb6edc79dd530ecda79    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 4 Sep 2020 12:40:28 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 4 Sep 2020 12:40:28 -0400    

Click here for diff

The "DROP ACCESS METHOD gist2" test will require locking the index  
to be dropped and then its table; while most ordinary operations  
lock a table first then its index.  While no concurrent test scripts  
should be touching fast_emp4000, autovacuum might chance to be  
processing that table when the DROP runs, resulting in a deadlock  
failure.  This is pretty rare but we see it in the buildfarm from  
time to time.  
  
To fix, acquire a lock on fast_emp4000 before issuing the DROP.  
  
Since the point of the exercise is mostly to prevent buildfarm  
failures, back-patch to 9.6 where this test was introduced.  
  
Discussion: https://postgr.es/m/839004.1599185607@sss.pgh.pa.us  

M src/test/regress/expected/create_am.out
M src/test/regress/sql/create_am.sql

Avoid lockup of a parallel worker when reporting a long error message.

commit   : 82dd373f2c56f7b32a3304260195fdaf6ed7cd9c    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 3 Sep 2020 16:52:09 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 3 Sep 2020 16:52:09 -0400    

Click here for diff

Because sigsetjmp() will restore the initial state with signals blocked,  
the code path in bgworker.c for reporting an error and exiting would  
execute that way.  Usually this is fairly harmless; but if a parallel  
worker had an error message exceeding the shared-memory communication  
buffer size (16K) it would lock up, because it would wait for a  
resume-sending signal from its parallel leader which it would never  
detect.  
  
To fix, just unblock signals at the appropriate point.  
  
This can be shown to fail back to 9.6.  The lack of parallel query  
infrastructure makes it difficult to provide a simple test case for  
9.5; but I'm pretty sure the issue exists in some form there as well,  
so apply the code change there too.  
  
Vignesh C, reviewed by Bharath Rupireddy, Robert Haas, and myself  
  
Discussion: https://postgr.es/m/CALDaNm1d1hHPZUg3xU4XjtWBOLCrA+-2cJcLpw-cePZ=GgDVfA@mail.gmail.com  

M src/backend/postmaster/bgworker.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql

Fix typo in comment

commit   : fcc42756818cef68e61e35e6d71cac6a73e24bb9    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 1 Sep 2020 20:43:23 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 1 Sep 2020 20:43:23 -0400    

Click here for diff

Introduced by 8b08f7d4820f; backpatch to 11.  
  
Discussion: https://postgr.es/m/20200812214918.GA30353@alvherre.pgsql  

M src/bin/pg_dump/pg_dump.h

doc: clarify that max_wal_size is "during" checkpoints

commit   : e3646325f4b6ca78c0a4234bf978cfda53721e4d    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Tue, 1 Sep 2020 17:00:10 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Tue, 1 Sep 2020 17:00:10 -0400    

Click here for diff

Previous wording was "between".  
  
Reported-by: Pavel Luzanov  
  
Discussion: https://postgr.es/m/26906a54-d7cb-2f8e-eed7-e31660024694@postgrespro.ru  
  
Backpatch-through: 9.5  

M doc/src/sgml/config.sgml

Raise error on concurrent drop of partitioned index

commit   : 7067ba1b4b905046819368e787809c5df0ebef04    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 1 Sep 2020 13:40:43 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 1 Sep 2020 13:40:43 -0400    

Click here for diff

We were already raising an error for DROP INDEX CONCURRENTLY on a  
partitioned table, albeit a different and confusing one:  
  ERROR:  DROP INDEX CONCURRENTLY must be first action in transaction  
  
Change that to throw a more comprehensible error:  
  ERROR:  cannot drop partitioned index \"%s\" concurrently  
  
Michael Paquier authored the test case for indexes on temporary  
partitioned tables.  
  
Backpatch to 11, where indexes on partitioned tables were added.  
  
Reported-by: Jan Mussler <jan.mussler@zalando.de>  
Reviewed-by: Michael Paquier <michael@paquier.xyz>  
Discussion: https://postgr.es/m/16594-d2956ca909585067@postgresql.org  

M doc/src/sgml/ref/drop_index.sgml
M src/backend/commands/tablecmds.c
M src/test/regress/expected/indexing.out
M src/test/regress/sql/indexing.sql

Teach libpq to handle arbitrary-length lines in .pgpass files.

commit   : 55aea0c706caa1d69dd550303ebefc310be4672c    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 1 Sep 2020 13:14:44 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 1 Sep 2020 13:14:44 -0400    

Click here for diff

Historically there's been a hard-wired assumption here that no line of  
a .pgpass file could be as long as NAMEDATALEN*5 bytes.  That's a bit  
shaky to start off with, because (a) there's no reason to suppose that  
host names fit in NAMEDATALEN, and (b) this figure fails to allow for  
backslash escape characters.  However, it fails completely if someone  
wants to use a very long password, and we're now hearing reports of  
people wanting to use "security tokens" that can run up to several  
hundred bytes.  Another angle is that the file is specified to allow  
comment lines, but there's no reason to assume that long comment lines  
aren't possible.  
  
Rather than guessing at what might be a more suitable limit, let's  
replace the fixed-size buffer with an expansible PQExpBuffer.  That  
adds one malloc/free cycle to the typical use-case, but that's surely  
pretty cheap relative to the I/O this code has to do.  
  
Also, add TAP test cases to exercise this code, because there was no  
test coverage before.  
  
This reverts most of commit 2eb3bc588, as there's no longer a need for  
a warning message about overlength .pgpass lines.  (I kept the explicit  
check for comment lines, though.)  
  
In HEAD and v13, this also fixes an oversight in 74a308cf5: there's not  
much point in explicit_bzero'ing the line buffer if we only do so in two  
of the three exit paths.  
  
Back-patch to all supported branches, except that the test case only  
goes back to v10 where src/test/authentication/ was added.  
  
Discussion: https://postgr.es/m/4187382.1598909041@sss.pgh.pa.us  

M src/interfaces/libpq/fe-connect.c
M src/test/authentication/t/001_password.pl

doc: add commas after 'i.e.' and 'e.g.'

commit   : 3fb67ac9c65b31ae179a80662b6bed0b00bba248    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 18:33:37 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 18:33:37 -0400    

Click here for diff

This follows the American format,  
https://jakubmarian.com/comma-after-i-e-and-e-g/. There is no intention  
of requiring this format for future text, but making existing text  
consistent every few years makes sense.  
  
Discussion: https://postgr.es/m/20200825183619.GA22369@momjian.us  
  
Backpatch-through: 9.5  

M doc/src/sgml/bki.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/create_database.sgml
M doc/src/sgml/ref/create_event_trigger.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/ref/create_statistics.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/replication-origins.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/sepgsql.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/sslinfo.sgml
M doc/src/sgml/tableam.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/wal.sgml
M doc/src/sgml/xfunc.sgml
M doc/src/sgml/xml2.sgml

C comment: remove mention of use of t_hoff WAL structure member

commit   : b1f52817bcdc3a6d269d4bd9b4d02a635e5d39c3    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 17:51:31 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 17:51:31 -0400    

Click here for diff

Reported-by: Antonin Houska  
  
Discussion: https://postgr.es/m/21643.1595353537@antos  
  
Backpatch-through: 9.5  

M src/include/access/heapam_xlog.h

pg_upgrade doc: mention saving postgresql.conf.auto files

commit   : 9774d9c777ba56b8a46357303aeb8e7d94420011    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 17:36:22 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 17:36:22 -0400    

Click here for diff

Also mention files included by postgresql.conf.  
  
Reported-by: Álvaro Herrera  
  
Discussion: https://postgr.es/m/08AD4526-75AB-457B-B2DD-099663F28040@yesql.se  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/pgupgrade.sgml

docs: in mapping SQL to C data types, timestamp isn't a pointer

commit   : 2bf1bec585972e16daa9104db08531d3b32ada54    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 17:05:53 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 17:05:53 -0400    

Click here for diff

It is an int64.  
  
Reported-by: ajulien@shaktiware.fr  
  
Discussion: https://postgr.es/m/159845038271.24995.15682121015698255155@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/xfunc.sgml

commit   : 02fb7eef91df41fab9c68cc73e6ec9f22c060981    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 16:59:58 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 16:59:58 -0400    

Click here for diff

There is an file-fdw example that reads the server config file, so cross  
link them.  
  
Reported-by: Oleg Samoilov  
  
Discussion: https://postgr.es/m/159800192078.2886.10431506404995508950@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/config.sgml
M doc/src/sgml/file-fdw.sgml

docs: clarify intermediate certificate creation instructions

commit   : 01bcfe705d69e0b9b50c752742a79bb88bdc72ff    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 16:21:03 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 16:21:03 -0400    

Click here for diff

Specifically, explain the v3_ca openssl specification.  
  
Discussion: https://postgr.es/m/20200824175653.GA32411@momjian.us  
  
Backpatch-through: 9.5  

M doc/src/sgml/runtime.sgml

docs: replace "stable storage" with "durable" in descriptions

commit   : 0d447ba962bbaa40f9d47d563086f1debf562b20    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 15:23:19 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 15:23:19 -0400    

Click here for diff

For PG, "durable storage" has a clear meaning, while "stable storage"  
does not, so use the former.  
  
Discussion: https://postgr.es/m/20200817165222.GA31806@momjian.us  
  
Backpatch-through: 9.5  

M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml

doc: improve description of subscripting of arrays

commit   : 2f8ea9116b17036400b71ec95923990e9fa833cf    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 13:49:17 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 13:49:17 -0400    

Click here for diff

It wasn't clear the non-integers are cast to integers for subscripting,  
rather than throwing an error.  
  
Reported-by: sean@materialize.io  
  
Discussion: https://postgr.es/m/159538675800.624.7728794628229799531@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/syntax.sgml

docs: improve 'capitals' inheritance example

commit   : ac736e43aa70dfb3da99e8d12349b801229aa2bc    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 13:43:04 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 13:43:04 -0400    

Click here for diff

Adds constraints and improves wording.  
  
Reported-by: 2552891@gmail.com  
  
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org  
  
Backpatch-through: 9.5  

M doc/src/sgml/advanced.sgml

doc: clarify the useful features of procedures

commit   : 5277f493dde7de00250d33a8ea79a5b93ee89dae    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 13:20:04 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 31 Aug 2020 13:20:04 -0400    

Click here for diff

This was not clearly documented when procedures were added in PG 11.  
  
Reported-by: Robin Abbi  
  
Discussion: https://postgr.es/m/CAGmg_NX327KKVuJmbWZD=pGutYFxzZjX1rU+3ji8UuX=8ONn9Q@mail.gmail.com  
  
Backpatch-through: 11  

M doc/src/sgml/xfunc.sgml

Fix docs bug stating file_fdw requires absolute paths

commit   : 73952310bfcd4ebcffa3db7137d0b705f1bfdae2    
  
author   : Magnus Hagander <magnus@hagander.net>    
date     : Mon, 31 Aug 2020 13:03:54 +0200    
  
committer: Magnus Hagander <magnus@hagander.net>    
date     : Mon, 31 Aug 2020 13:03:54 +0200    

Click here for diff

It has always (since the first commit) worked with relative paths, so  
use the same wording as other parts of the documentation.  
  
Author: Bruce Momjian  
Discussion: https://postgr.es/m/CABUevExx-hm=cit+A9LeKBH39srvk8Y2tEZeEAj5mP8YfzNKUg@mail.gmail.com  

M doc/src/sgml/file-fdw.sgml

Mark factorial operator, and postfix operators in general, as deprecated.

commit   : 9dee85b5b83310598c66efc9322ed6792c77fa78    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 30 Aug 2020 16:03:19 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 30 Aug 2020 16:03:19 -0400    

Click here for diff

Back-patch key parts of 4c5cf5431 and 6ca547cf7 into stable branches.  
I didn't touch pg_description entries here, so it's purely a docs  
change; and I didn't fool with any examples either.  The main point  
is so that anyone who's wondering if factorial() exists in the stable  
branches will be reassured.  
  
Mark Dilger and John Naylor, with some adjustments by me  
  
Discussion: https://postgr.es/m/BE2DF53D-251A-4E26-972F-930E523580E9@enterprisedb.com  

M doc/src/sgml/func.sgml
M doc/src/sgml/ref/create_operator.sgml

Fix code for re-finding scan position in a multicolumn GIN index.

commit   : ff3c16d1e751d8a7dfc753c3871d01ce8cf36089    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 27 Aug 2020 17:36:13 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 27 Aug 2020 17:36:13 -0400    

Click here for diff

collectMatchBitmap() needs to re-find the index tuple it was previously  
looking at, after transiently dropping lock on the index page it's on.  
The tuple should still exist and be at its prior position or somewhere  
to the right of that, since ginvacuum never removes tuples but  
concurrent insertions could add one.  However, there was a thinko in  
that logic, to the effect of expecting any inserted tuples to have the  
same index "attnum" as what we'd been scanning.  Since there's no  
physical separation of tuples with different attnums, it's not terribly  
hard to devise scenarios where this fails, leading to transient "lost  
saved point in index" errors.  (While I've duplicated this with manual  
testing, it seems impossible to make a reproducible test case with our  
available testing technology.)  
  
Fix by just continuing the scan when the attnum doesn't match.  
  
While here, improve the error message used if we do fail, so that it  
matches the wording used in btree for a similar case.  
  
collectMatchBitmap()'s posting-tree code path was previously not  
exercised at all by our regression tests.  While I can't make  
a regression test that exhibits the bug, I can at least improve  
the code coverage here, so do that.  The test case I made for this  
is an extension of one added by 4b754d6c1, so it only works in  
HEAD and v13; didn't seem worth trying hard to back-patch it.  
  
Per bug #16595 from Jesse Kinkead.  This has been broken since  
multicolumn capability was added to GIN (commit 27cb66fdf),  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/16595-633118be8eef9ce2@postgresql.org  

M src/backend/access/gin/ginget.c

docs: client certificates are always sent to the server

commit   : fa3bd8d853edd0f47523f8f0ce3bc74d21cf05f7    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Tue, 25 Aug 2020 09:53:12 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Tue, 25 Aug 2020 09:53:12 -0400    

Click here for diff

They are not "requested" by the server.  
  
Reported-by: Kyotaro Horiguchi  
  
Discussion: https://postgr.es/m/20200825.155320.986648039251743210.horikyota.ntt@gmail.com  
  
Backpatch-through: 9.5  

M doc/src/sgml/libpq.sgml

doc: Fix up title case

commit   : 4534a1e81039866f1a0c4326e0458031a8ef483e    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Tue, 25 Aug 2020 07:29:05 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Tue, 25 Aug 2020 07:29:05 +0200    

Click here for diff

This fixes some instances that were missed in earlier processings and  
that now look a bit strange because they are inconsistent with nearby  
titles.  

M doc/src/sgml/dml.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/plpgsql.sgml

Avoid pushing quals down into sub-queries that have grouping sets.

commit   : 6b701eaaa90e07035f9a965d7ca9831d01586c63    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 22 Aug 2020 14:46:40 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 22 Aug 2020 14:46:40 -0400    

Click here for diff

The trouble with doing this is that an apparently-constant subquery  
output column isn't really constant if it is a grouping column that  
appears in only some of the grouping sets.  A qual using such a  
column would be subject to incorrect const-folding after push-down,  
as seen in bug #16585 from Paul Sivash.  
  
To fix, just disable qual pushdown altogether if the sub-query has  
nonempty groupingSets.  While we could imagine far less restrictive  
solutions, there is not much point in working harder right now,  
because subquery_planner() won't move HAVING clauses to WHERE within  
such a subquery.  If the qual stays in HAVING it's not going to be  
a lot more useful than if we'd kept it at the outer level.  
  
Having said that, this restriction could be removed if we used a  
parsetree representation that distinguished such outputs from actual  
constants, which is something I hope to do in future.  Hence, make  
the patch a minimal addition rather than integrating it more tightly  
(e.g. by renumbering the existing items in subquery_is_pushdown_safe's  
comment).  
  
Back-patch to 9.5 where grouping sets were introduced.  
  
Discussion: https://postgr.es/m/16585-9d8c340d23ade8c1@postgresql.org  

M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql

docs: improve description of how to handle multiple databases

commit   : 2060f0064e856ed0648ef2a77cda085d35837bca    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 21 Aug 2020 20:23:09 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 21 Aug 2020 20:23:09 -0400    

Click here for diff

This is a redesign of the intro to the managing databases chapter.  
  
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org  
  
Author: David G. Johnston  
  
Backpatch-through: 9.5  

M doc/src/sgml/manage-ag.sgml

docs: add COMMENT examples for new features, rename rtree

commit   : 90f6435f672c7de92220a92084fcbef99f95bb4e    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 21 Aug 2020 18:29:37 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 21 Aug 2020 18:29:37 -0400    

Click here for diff

Reported-by: Jürgen Purtz  
  
Discussion: https://postgr.es/m/15ec5428-d46a-1725-f38d-44986a977abb@purtz.de  
  
Author: Jürgen Purtz  
  
Backpatch-through: 11  

M doc/src/sgml/ref/comment.sgml

Fix handling of CREATE TABLE LIKE with inheritance.

commit   : d9253df12e4c06a54e515136b115538c56eace56    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 21 Aug 2020 15:00:43 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 21 Aug 2020 15:00:43 -0400    

Click here for diff

If a CREATE TABLE command uses both LIKE and traditional inheritance,  
Vars in CHECK constraints and expression indexes that are absorbed  
from a LIKE parent table tended to get mis-numbered, resulting in  
wrong answers and/or bizarre error messages (though probably not any  
actual crashes, thanks to validation occurring in the executor).  
  
In v12 and up, the same could happen to Vars in GENERATED expressions,  
even in cases with no LIKE clause but multiple traditional-inheritance  
parents.  
  
The cause of the problem for LIKE is that parse_utilcmd.c supposed  
it could renumber such Vars correctly during transformCreateStmt(),  
which it cannot since we have not yet accounted for columns added via  
inheritance.  Fix that by postponing processing of LIKE INCLUDING  
CONSTRAINTS, DEFAULTS, GENERATED, INDEXES till after we've performed  
DefineRelation().  
  
The error with GENERATED and multiple inheritance is a simple oversight  
in MergeAttributes(); it knows it has to renumber Vars in inherited  
CHECK constraints, but forgot to apply the same processing to inherited  
GENERATED expressions (a/k/a defaults).  
  
Per bug #16272 from Tom Gottfried.  The non-GENERATED variants of the  
issue are ancient, presumably dating right back to the addition of  
CREATE TABLE LIKE; hence back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/16272-6e32da020e9a9381@postgresql.org  

M src/backend/commands/tablecmds.c
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/include/nodes/parsenodes.h
M src/include/parser/parse_utilcmd.h
M src/test/modules/test_ddl_deparse/expected/create_table.out
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql

Fix a few typos in JIT comments and README

commit   : 3248633579e9ed1c718afa70bb93d499fe85d40c    
  
author   : David Rowley <drowley@postgresql.org>    
date     : Fri, 21 Aug 2020 09:35:24 +1200    
  
committer: David Rowley <drowley@postgresql.org>    
date     : Fri, 21 Aug 2020 09:35:24 +1200    

Click here for diff

Reviewed-by: Abhijit Menon-Sen  
Reviewed-by: Andres Freund  
Discussion: https://postgr.es/m/CAApHDvobgmCs6CohqhKTUf7D8vffoZXQTCBTERo9gbOeZmvLTw%40mail.gmail.com  
Backpatch-through: 11, where JIT was added  

M src/backend/jit/README
M src/include/jit/llvmjit_emit.h

Revise REINDEX CONCURRENTLY recovery instructions

commit   : da9ec328dda9625725ae8210ad6e6599ab876206    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Thu, 20 Aug 2020 13:49:04 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Thu, 20 Aug 2020 13:49:04 -0400    

Click here for diff

When the leftover invalid index is "ccold", there's no need to re-run  
the command.  Reword the instructions to make that explicit.  
  
Backpatch to 12, where REINDEX CONCURRENTLY appeared.  
  
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>  
Reviewed-by: Michael Paquier <michael@paquier.xyz>  
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>  
Discussion: https://postgr.es/m/20200819211312.GA15497@alvherre.pgsql  

M doc/src/sgml/ref/reindex.sgml

Suppress unnecessary RelabelType nodes in yet more cases.

commit   : 814a57065ec97e44de58db61a16957ddd09938e8    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 19 Aug 2020 14:07:49 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 19 Aug 2020 14:07:49 -0400    

Click here for diff

Commit a477bfc1d fixed eval_const_expressions() to ensure that it  
didn't generate unnecessary RelabelType nodes, but I failed to notice  
that some other places in the planner had the same issue.  Really  
noplace in the planner should be using plain makeRelabelType(), for  
fear of generating expressions that should be equal() to semantically  
equivalent trees, but aren't.  
  
An example is that because canonicalize_ec_expression() failed  
to be careful about this, we could end up with an equivalence class  
containing both a plain Const, and a Const-with-RelabelType  
representing exactly the same value.  So far as I can tell this led to  
no visible misbehavior, but we did waste a bunch of cycles generating  
and evaluating "Const = Const-with-RelabelType" to prove such entries  
are redundant.  
  
Hence, move the support function added by a477bfc1d to where it can  
be more generally useful, and use it in the places where planner code  
previously used makeRelabelType.  
  
Back-patch to v12, like the previous patch.  While I have no concrete  
evidence of any real misbehavior here, it's certainly possible that  
I overlooked a case where equivalent expressions that aren't equal()  
could cause a user-visible problem.  In any case carrying extra  
RelabelType nodes through planning to execution isn't very desirable.  
  
Discussion: https://postgr.es/m/1311836.1597781384@sss.pgh.pa.us  

M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/nodeFuncs.h

Avoid non-constant format string argument to fprintf().

commit   : aecefffc3f8041c883ab4fb035cf0d5519b5a7ed    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Tue, 18 Aug 2020 13:13:09 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Tue, 18 Aug 2020 13:13:09 +0300    

Click here for diff

As Tom Lane pointed out, it could defeat the compiler's printf() format  
string verification.  
  
Backpatch to v12, like that patch that introduced it.  
  
Discussion: https://www.postgresql.org/message-id/1069283.1597672779%40sss.pgh.pa.us  

M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_rewind/pg_rewind.c

Disable autovacuum for BRIN test table

commit   : 4f47c8e7d438a0ae8ea631cd0c3b92db5593901d    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 17 Aug 2020 16:20:06 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Mon, 17 Aug 2020 16:20:06 -0400    

Click here for diff

This should improve stability in the tests.  
  
Per buildfarm member hyrax (CLOBBER_CACHE_ALWAYS) via Tom Lane.  
  
Discussion: https://postgr.es/m/871534.1597503261@sss.pgh.pa.us  

M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql

Doc: fix description of UNION/CASE/etc type unification.

commit   : 314f65716c37d515979eb2f06430b18bb7a09a0d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 17 Aug 2020 15:40:07 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 17 Aug 2020 15:40:07 -0400    

Click here for diff

The description of what select_common_type() does was not terribly  
accurate.  Improve it.  
  
David Johnston and Tom Lane  
  
Discussion: https://postgr.es/m/1019930.1597613200@sss.pgh.pa.us  

M doc/src/sgml/typeconv.sgml

Fix printing last progress report line in client programs.

commit   : 9f44d71b06b0d457ad3e5d7fdcbba2039f3000ef    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Mon, 17 Aug 2020 09:27:29 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Mon, 17 Aug 2020 09:27:29 +0300    

Click here for diff

A number of client programs have a "--progress" option that when printing  
to a TTY, updates the current line by printing a '\r' and overwriting it.  
After the last line, '\n' needs to be printed to move the cursor to the  
next line. pg_basebackup and pgbench got this right, but pg_rewind and  
pg_checksums were slightly wrong. pg_rewind printed the newline to stdout  
instead of stderr, and pg_checksums printed the newline even when not  
printing to a TTY. Fix them, and also add a 'finished' argument to  
pg_basebackup's progress_report() function, to keep it consistent with  
the other programs.  
  
Backpatch to v12. pg_rewind's newline was broken with the logging changes  
in commit cc8d415117 in v12, and pg_checksums was introduced in v12.  
  
Discussion: https://www.postgresql.org/message-id/82b539e5-ae33-34b0-1aee-22b3379fd3eb@iki.fi  

M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h

doc: Fix description about bgwriter and checkpoint in HA section

commit   : 29bd2cbe36bd3549a6a81df6e5dab172f43ef033    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Mon, 17 Aug 2020 10:24:31 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Mon, 17 Aug 2020 10:24:31 +0900    

Click here for diff

Since 806a2ae, the work of the bgwriter is split the checkpointer, but a  
portion of the documentation did not get the message.  
  
Author: Masahiko Sawada  
Discussion: https://postgr.es/m/CA+fd4k6jXxjAtjMVC=wG3=QGpauZBtcgN3Jhw+oV7zXGKVLKzQ@mail.gmail.com  
Backpatch-through: 9.5  

M doc/src/sgml/high-availability.sgml

Move new LOCKTAG_DATABASE_FROZEN_IDS to end of enum LockTagType.

commit   : 06e50d8f7eee9a0b42c8392b75e0780c0a30fa84    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 15 Aug 2020 16:15:59 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 15 Aug 2020 16:15:59 -0700    

Click here for diff

Several PGXN modules reference LockTagType values; renumbering would  
force a recompile of those modules.  Oversight in back-patch of today's  
commit 566372b3d6435639e4cc4476d79b8505a0297c87.  Back-patch to released  
branches, v12 through 9.5.  
  
Reported by Tom Lane.  
  
Discussion: https://postgr.es/m/921383.1597523945@sss.pgh.pa.us  

M src/backend/utils/adt/lockfuncs.c
M src/include/storage/lock.h

Prevent concurrent SimpleLruTruncate() for any given SLRU.

commit   : 30e68a2abb3890c3292ff0b2422a7ea04d62acdd    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 15 Aug 2020 10:15:53 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 15 Aug 2020 10:15:53 -0700    

Click here for diff

The SimpleLruTruncate() header comment states the new coding rule.  To  
achieve this, add locktype "frozenid" and two LWLocks.  This closes a  
rare opportunity for data loss, which manifested as "apparent  
wraparound" or "could not access status of transaction" errors.  Data  
loss is more likely in pg_multixact, due to released branches' thin  
margin between multiStopLimit and multiWrapLimit.  If a user's physical  
replication primary logged ":  apparent wraparound" messages, the user  
should rebuild standbys of that primary regardless of symptoms.  At less  
risk is a cluster having emitted "not accepting commands" errors or  
"must be vacuumed" warnings at some point.  One can test a cluster for  
this data loss by running VACUUM FREEZE in every database.  Back-patch  
to 9.5 (all supported versions).  
  
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com  

M doc/src/sgml/catalogs.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/commands/vacuum.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/utils/adt/lockfuncs.c
M src/include/storage/lmgr.h
M src/include/storage/lock.h

Be more careful about the shape of hashable subplan clauses.

commit   : 912fb290c5dd087ccc4b04c19692137bb786a9fd    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 Aug 2020 22:14:03 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 Aug 2020 22:14:03 -0400    

Click here for diff

nodeSubplan.c expects that the testexpr for a hashable ANY SubPlan  
has the form of one or more OpExprs whose LHS is an expression of the  
outer query's, while the RHS is an expression over Params representing  
output columns of the subquery.  However, the planner only went as far  
as verifying that the clauses were all binary OpExprs.  This works  
99.99% of the time, because the clauses have the right shape when  
emitted by the parser --- but it's possible for function inlining to  
break that, as reported by PegoraroF10.  To fix, teach the planner  
to check that the LHS and RHS contain the right things, or more  
accurately don't contain the wrong things.  Given that this has been  
broken for years without anyone noticing, it seems sufficient to just  
give up hashing when it happens, rather than go to the trouble of  
commuting the clauses back again (which wouldn't necessarily work  
anyway).  
  
While poking at that, I also noticed that nodeSubplan.c had a baked-in  
assumption that the number of hash clauses is identical to the number  
of subquery output columns.  Again, that's fine as far as parser output  
goes, but it's not hard to break it via function inlining.  There seems  
little reason for that assumption though --- AFAICS, the only thing  
it's buying us is not having to store the number of hash clauses  
explicitly.  Adding code to the planner to reject such cases would take  
more code than getting nodeSubplan.c to cope, so I fixed it that way.  
  
This has been broken for as long as we've had hashable SubPlans,  
so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/1549209182255-0.post@n3.nabble.com  

M src/backend/executor/nodeSubplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql

pg_dump: fix dependencies on FKs to partitioned tables

commit   : 3dadcb2b31946f6c71c764b0cc50a74ec363e4ab    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 14 Aug 2020 17:33:31 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 14 Aug 2020 17:33:31 -0400    

Click here for diff

Parallel-restoring a foreign key that references a partitioned table  
with several levels of partitions can fail:  
  
pg_restore: while PROCESSING TOC:  
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres  
pg_restore: error: could not execute query: ERROR:  there is no unique constraint matching given keys for referenced table "pk"  
Command was: ALTER TABLE fkpart3.fk  
    ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);  
  
This happens in parallel restore mode because some index partitions  
aren't yet attached to the topmost partitioned index that the FK uses,  
and so the index is still invalid.  The current code marks the FK as  
dependent on the first level of index-attach dump objects; the bug is  
fixed by recursively marking the FK on their children.  
  
Backpatch to 12, where FKs to partitioned tables were introduced.  
  
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>  
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>  
Discussion: https://postgr.es/m/3170626.1594842723@sss.pgh.pa.us  
Backpatch: 12-master  

M src/bin/pg_dump/pg_dump.c

Fix postmaster's behavior during smart shutdown.

commit   : 42566a25003d73ec3dca5750cc7a51023f7bd799    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 Aug 2020 13:26:57 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 14 Aug 2020 13:26:57 -0400    

Click here for diff

Up to now, upon receipt of a SIGTERM ("smart shutdown" command), the  
postmaster has immediately killed all "optional" background processes,  
and subsequently refused to launch new ones while it's waiting for  
foreground client processes to exit.  No doubt this seemed like an OK  
policy at some point; but it's a pretty bad one now, because it makes  
for a seriously degraded environment for the remaining clients:  
  
* Parallel queries are killed, and new ones fail to launch. (And our  
parallel-query infrastructure utterly fails to deal with the case  
in a reasonable way --- it just hangs waiting for workers that are  
not going to arrive.  There is more work needed in that area IMO.)  
  
* Autovacuum ceases to function.  We can tolerate that for awhile,  
but if bulk-update queries continue to run in the surviving client  
sessions, there's eventually going to be a mess.  In the worst case  
the system could reach a forced shutdown to prevent XID wraparound.  
  
* The bgwriter and walwriter are also stopped immediately, likely  
resulting in performance degradation.  
  
Hence, let's rearrange things so that the only immediate change in  
behavior is refusing to let in new normal connections.  Once the last  
normal connection is gone, shut everything down as though we'd received  
a "fast" shutdown.  To implement this, remove the PM_WAIT_BACKUP and  
PM_WAIT_READONLY states, instead staying in PM_RUN or PM_HOT_STANDBY  
while normal connections remain.  A subsidiary state variable tracks  
whether or not we're letting in new connections in those states.  
  
This also allows having just one copy of the logic for killing child  
processes in smart and fast shutdown modes.  I moved that logic into  
PostmasterStateMachine() by inventing a new state PM_STOP_BACKENDS.  
  
Back-patch to 9.6 where parallel query was added.  In principle  
this'd be a good idea in 9.5 as well, but the risk/reward ratio  
is not as good there, since lack of autovacuum is not a problem  
during typical uses of smart shutdown.  
  
Per report from Bharath Rupireddy.  
  
Patch by me, reviewed by Thomas Munro  
  
Discussion: https://postgr.es/m/CALj2ACXAZ5vKxT9P7P89D87i3MDO9bfS+_bjMHgnWJs8uwUOOw@mail.gmail.com  

M doc/src/sgml/ref/pg_ctl-ref.sgml
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/libpq-be.h

Fix typo in test comment.

commit   : f81167adbf7414f3ce7baa12e0894501db68f73f    
  
author   : Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Fri, 14 Aug 2020 10:40:50 +0300    
  
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>    
date     : Fri, 14 Aug 2020 10:40:50 +0300    

Click here for diff

M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql

Handle new HOT chains in index-build table scans

commit   : 1122a903e967a388a1a8f20de15dc65ca639d407    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Thu, 13 Aug 2020 17:33:49 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Thu, 13 Aug 2020 17:33:49 -0400    

Click here for diff

When a table is scanned by heapam_index_build_range_scan (née  
IndexBuildHeapScan) and the table lock being held allows concurrent data  
changes, it is possible for new HOT chains to sprout in a page that were  
unknown when the scan of a page happened.  This leads to an error such  
as  
  ERROR:  failed to find parent tuple for heap-only tuple at (X,Y) in table "tbl"  
because the root tuple was not present when we first obtained the list  
of the page's root tuples.  This can be fixed by re-obtaining the list  
of root tuples, if we see that a heap-only tuple appears to point to a  
non-existing root.  
  
This was reported by Anastasia as occurring for BRIN summarization  
(which exists since 9.5), but I think it could theoretically also happen  
with CREATE INDEX CONCURRENTLY (much older) or REINDEX CONCURRENTLY  
(very recent).  It seems a happy coincidence that BRIN forces us to  
backpatch this all the way to 9.5.  
  
Reported-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>  
Diagnosed-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>  
Co-authored-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>  
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>  
Discussion: https://postgr.es/m/602d8487-f0b2-5486-0088-0f372b2549fa@postgrespro.ru  
Backpatch: 9.5 - master  

M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/pruneheap.c

BRIN: Handle concurrent desummarization properly

commit   : 0426c75e7d1d058a3d61bac678e17b675ddaebb9    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 12 Aug 2020 15:33:36 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 12 Aug 2020 15:33:36 -0400    

Click here for diff

If a page range is desummarized at just the right time concurrently with  
an index walk, BRIN would raise an error indicating index corruption.  
This is scary and unhelpful; silently returning that the page range is  
not summarized is sufficient reaction.  
  
This bug was introduced by commit 975ad4e602ff as additional protection  
against a bug whose actual fix was elsewhere.  Backpatch equally.  
  
Reported-By: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>  
Diagnosed-By: Alexander Lakhin <exclusion@gmail.com>  
Discussion: https://postgr.es/m/2588667e-d07d-7e10-74e2-7e1e46194491@postgrespro.ru  
Backpatch: 9.5 - master  

M src/backend/access/brin/brin_revmap.c