PostgreSQL 10.1 commit log

commit   : 958fe549884928cd3bdf009993e9a05df5fd6cee    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 19:46:52 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 19:46:52 -0500    

configure computed PG_VERSION_NUM incorrectly.  (Coulda sworn I tested  
that logic back when, but it had an obvious thinko.)  
  
pg_upgrade had not been taught about the new dispensation with just  
one part in the major version number.  
  
Both things accidentally failed to fail with 10.0, but with 10.1 we  
got the wrong results.  
  
Per buildfarm.  

commit   : 0b35d54c6fc4d98f8e8b38770b53c7358ebf4444    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 17:06:17 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 17:06:17 -0500    

commit   : 50abeafc74a812d2449ec49dc16e99baf8c5023a    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 12:02:30 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 12:02:30 -0500    

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099  

commit   : c30f082d2767c22cefb8875dcb1932e5ed338db6    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 10:29:16 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 10:29:16 -0500    

json{b}_populate_recordset() used the tuple descriptor created from the  
query-level AS clause without worrying about whether it matched the actual  
input record type.  If it didn't, that would usually result in a crash,  
though disclosure of server memory contents seems possible as well, for a  
skilled attacker capable of issuing crafted SQL commands.  Instead, use  
the query-supplied descriptor only when there is no input tuple to look at,  
and otherwise get a tuple descriptor based on the input tuple's own type  
marking.  The core code will detect any type mismatch in the latter case.  
  
Michael Paquier and Tom Lane, per a report from David Rowley.  
Back-patch to 9.3 where this functionality was introduced.  
  
Security: CVE-2017-15098  

commit   : 6b0b983f793b5c9ea68167a408ef58d40b942596    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 6 Nov 2017 07:11:10 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 6 Nov 2017 07:11:10 -0800    

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER  
replaces $PGLOG with a symbolic link, the server will corrupt the  
link-targeted file by appending log messages.  Since these scripts open  
$PGLOG as root, the attack works regardless of target file ownership.  
  
"make install" does not install these scripts anywhere.  Users having  
manually installed them in the past should repeat that process to  
acquire this fix.  Most script users have $PGLOG writable to root only,  
located in $PGDATA.  Just before updating one of these scripts, such  
users should rename $PGLOG to $PGLOG.old.  The script will then recreate  
$PGLOG with proper ownership.  
  
Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.  
  
Security: CVE-2017-12172  

commit   : 3f80895723037c0d1c684dbdd50b7e03453df90f    
  
author   : Dean Rasheed <[email protected]>    
date     : Mon, 6 Nov 2017 09:17:44 +0000    
  
committer: Dean Rasheed <[email protected]>    
date     : Mon, 6 Nov 2017 09:17:44 +0000    

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT  
permission on the columns of the arbiter index, but it failed to check  
for that in the case of an arbiter specified by constraint name.  
  
In addition, for a table with row level security enabled, it failed to  
check updated rows against the table's SELECT policies when the update  
path was taken (regardless of how the arbiter index was specified).  
  
Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.  
  
Security: CVE-2017-15099  

commit   : 937f67800a7d3c30870af7b1115a76f44bc26e78    
  
author   : Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 18:51:08 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 18:51:08 -0800    

Makefile.global assigns this prerequisite to every target named "check",  
but similar targets must mention it explicitly.  Affected targets  
failed, tested $PATH binaries, or tested a stale temporary installation.  
The src/test/modules examples worked properly when called as "make -C  
src/test/modules/$FOO check", but "make -j" allowed the test to start  
before the temporary installation was in place.  Back-patch to 9.5,  
where commit dcae5faccab64776376d354decda0017c648bb53 introduced the  
shared temp-install.  

commit   : 8135c3cd057415bc80fbfcf0d8f1134d8f4efc12    
  
author   : Peter Eisentraut <[email protected]>    
date     : Sun, 5 Nov 2017 17:00:29 -0500    
  
committer: Peter Eisentraut <[email protected]>    
date     : Sun, 5 Nov 2017 17:00:29 -0500    

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 58ffddb2eb9d9b32697223abc420d3e53b884b60  

commit   : 992c3eb1276e6f6abaac8c4ac8dfcfe3421f8a6e    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 5 Nov 2017 13:47:56 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 5 Nov 2017 13:47:56 -0500    

In the v10 branch, also back-patch the effects of 1ff01b390 and c29c57890  
on these files, to reduce future maintenance issues.  (I'd do it further  
back, except that the 9.X branches differ anyway due to xlog-to-wal  
link tag renaming.)  

commit   : 2168f37c4db65f40925f4f8ccbb901544a5edb84    
  
author   : Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 09:25:52 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 09:25:52 -0800    

This restores the ability, essentially lost in commit  
ffaa44cb559db332baeee7d25dedd74a61974203, to use COPY FREEZE under  
REPEATABLE READ isolation.  Back-patch to 9.4, like that commit.  
  
Reviewed by Tom Lane.  
  
Discussion: https://postgr.es/m/CA+TgmoahWDm-7fperBxzU9uZ99LPMUmEpSXLTw9TmrOgzwnORw@mail.gmail.com  

commit   : c3f43a71663cdb9d589f536b544b03675f13e14f    
  
author   : Peter Eisentraut <[email protected]>    
date     : Sat, 4 Nov 2017 14:42:20 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Sat, 4 Nov 2017 14:42:20 -0400    

Reported-by: Tomonari Katsumata <[email protected]>  
Author: Michael Paquier <[email protected]>  

commit   : b17870d754665c4b431d3ce4de6676832a0b10a0    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 3 Nov 2017 20:36:32 +0100    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 3 Nov 2017 20:36:32 +0100    

The previous commit contained a thinko that made a single-range  
summarization request process from there to end of table.  Fix by  
setting the correct end range point.  Per buildfarm.  

commit   : 5159626afbe969bdcb6623e9f75f79ecb13ec79c    
  
author   : Peter Eisentraut <[email protected]>    
date     : Thu, 26 Oct 2017 15:19:56 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Thu, 26 Oct 2017 15:19:56 -0400    

This makes the produced HTML anchors upper case, making it backward  
compatible with the previous (9.6) build system.  
  
Reported-by: Thomas Kellerer <[email protected]>  

commit   : a14b52c61454e3d23f7d0ab0affa56f8abfdf505    
  
author   : Peter Eisentraut <[email protected]>    
date     : Fri, 3 Nov 2017 11:59:20 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Fri, 3 Nov 2017 11:59:20 -0400    

When a publisher table has fewer columns than a subscriber, the update  
of a row on the publisher should result in updating of only the columns  
in common.  The previous coding mistakenly reset the values of  
additional columns on the subscriber to NULL because it failed to skip  
updates of columns not found in the attribute map.  
  
Author: Petr Jelinek <[email protected]>  

commit   : 37a856567c04cf48d38292afc487be5f2665c2a5    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 3 Nov 2017 17:23:13 +0100    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 3 Nov 2017 17:23:13 +0100    

If a process is extending a table concurrently with some BRIN  
summarization process, it is possible for the latter to miss pages added  
by the former because the number of pages is computed ahead of time.  
  
Fix by determining a fresh relation size after inserting the placeholder  
tuple: any process that further extends the table concurrently will  
update the placeholder tuple, while previous pages will be processed by  
the heap scan.  
  
Reported-by: Tomas Vondra  
Reviewed-by: Tom Lane  
Author: Álvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-to: 9.5  

commit   : 0f6bd533341341de7007b6d5ed24742211471c68    
  
author   : Simon Riggs <[email protected]>    
date     : Mon, 2 Oct 2017 10:27:46 +0100    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 2 Oct 2017 10:27:46 +0100    

commit   : c63568b06edeece064771b8f170ef0298d8ab134    
  
author   : Michael Meskes <[email protected]>    
date     : Fri, 3 Nov 2017 11:14:30 +0100    
  
committer: Michael Meskes <[email protected]>    
date     : Fri, 3 Nov 2017 11:14:30 +0100    

commit   : ee46980a73bf5b1b39c3a6be76dc01fa2a5819c7    
  
author   : Michael Meskes <[email protected]>    
date     : Thu, 2 Nov 2017 20:46:34 +0100    
  
committer: Michael Meskes <[email protected]>    
date     : Thu, 2 Nov 2017 20:46:34 +0100    

commit   : 97ba7b8c87e13112fd31d936507e7d060a297aa4    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 2 Nov 2017 12:54:22 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 2 Nov 2017 12:54:22 -0400    

In some cases the BRIN code releases lock on an index page, and later  
re-acquires lock and tries to check that the tuple it was working on is  
still there.  That check was a couple bricks shy of a load.  It didn't  
consider that the page might have turned into a "revmap" page.  (The  
samepage code path doesn't call brin_getinsertbuffer(), so it isn't  
protected by the checks for revmap status there.)  It also didn't check  
whether the tuple offset was now off the end of the linepointer array.  
Since commit 24992c6db the latter case is pretty common, but at least  
in principle it could have occurred before that.  The net result is  
that concurrent updates of a BRIN index could fail with errors like  
"invalid index offnum" or "inconsistent range map".  
  
Per report from Tomas Vondra.  Back-patch to 9.5, since this code is  
substantially the same in all versions containing BRIN.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7a95966bc03cd8af08825de235ea896d682d62bb    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 2 Nov 2017 15:51:05 +0100    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 2 Nov 2017 15:51:05 +0100    

It turns out we misdiagnosed what the real problem was.  Revert the  
previous changes, because they may have worse consequences going  
forward.  A better fix is forthcoming.  
  
The simplistic test case is kept, though disabled.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 769756fb74d6a183442ae3a3b5ce0816226ce64b    
  
author   : Peter Eisentraut <[email protected]>    
date     : Thu, 2 Nov 2017 09:08:03 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Thu, 2 Nov 2017 09:08:03 -0400    

per request of the named person  

commit   : f4e13963ca88365263533d3cf523408d60b9f33f    
  
author   : Noah Misch <[email protected]>    
date     : Wed, 1 Nov 2017 19:16:14 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Wed, 1 Nov 2017 19:16:14 -0700    

Back-patch to v10.  This continues the work of commit  
080351466c5a669bf35a323bdec9e296330a5dbb.  
  
Discussion: https://postgr.es/m/CAKOSWN=ds66zLw2SqkLTM8wbXFgDbc_OdkmT3dJfPT2mE5kipA@mail.gmail.com  

commit   : 1048afc864e84b4f206bb7edd3773b49f1d8cc5b    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 1 Nov 2017 22:07:14 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 1 Nov 2017 22:07:14 -0400    

Reported by Dan Vianello.  
  
Discussion: https://postgr.es/m/e6e12f18f70e46848c058084d42fb651@KSTLMEXGP001.CORP.CHARTERCOM.com  

commit   : dd12b53078c37ea8731e9cb053bbeefe725ea47a    
  
author   : Peter Eisentraut <[email protected]>    
date     : Wed, 1 Nov 2017 10:20:05 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Wed, 1 Nov 2017 10:20:05 -0400    

A candidate path needs to be canonicalized before being checked against  
the mappings, because the mappings are also canonicalized.  This is  
especially relevant on Windows  
  
Reported-by: nb <[email protected]>  
Author: Michael Paquier <[email protected]>  
Reviewed-by: Ashutosh Sharma <[email protected]>  

commit   : e4381c039d5d92736527ee0825a470ddb5045458    
  
author   : Michael Meskes <[email protected]>    
date     : Wed, 1 Nov 2017 13:32:18 +0100    
  
committer: Michael Meskes <[email protected]>    
date     : Wed, 1 Nov 2017 13:32:18 +0100    

Spotted and fixed by 高增琦 <[email protected]>  

commit   : 9cf2b854a59d227eaec3d97780e986ebdd0c6854    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 31 Oct 2017 13:40:23 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 31 Oct 2017 13:40:23 -0400    

Queries running with some non-pg_catalog schema frontmost in their search  
path need to be careful to schema-qualify type names that should be sought  
in pg_catalog.  Vitaly Burovoy reported an oversight of this sort in  
pg_dump's dumpSequence, and grepping detected another one in psql's  
describeOneTableDetails, both introduced by sequence-related changes in  
v10.  In pg_dump, we can fix things by removing the cast altogether, since  
it doesn't really matter what data types are reported for these query  
result columns.  Likewise in psql, the query seemed to be working unduly  
hard to get a result that's guaranteed to be exactly 'bigint'.  
  
I also changed a couple of occurrences of "::char" similarly.  These are  
not bugs, since "char" is a typename keyword and not subject to search_path  
rules, but it seems better to use uniform style.  
  
Vitaly Burovoy and Tom Lane  
  
Discussion: https://postgr.es/m/CAKOSWN=ds66zLw2SqkLTM8wbXFgDbc_OdkmT3dJfPT2mE5kipA@mail.gmail.com  

commit   : 7becb5fa1d8760ee70258ff23ce229ce5451c597    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 30 Oct 2017 16:44:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 30 Oct 2017 16:44:26 -0400    

The change made by commit 906bfcad7 means that if you're writing  
a parenthesized column list in UPDATE ... SET, but that column list  
is only one column, you now need to write ROW(expression) on the  
righthand side, not just a parenthesized expression.  This was an  
intentional change for spec compatibility and potential future  
expansion of the possibilities for the RHS, but I'd neglected to  
document it as a compatibility issue, figuring that hardly anyone  
would bother with parenthesized syntax for a single target column.  
I was wrong, as shown by questions from Justin Pryzby, Adam Brusselback,  
and others.  Move the release note item into the compatibility section  
and point out the behavior change for a single target column.  
  
Discussion: https://postgr.es/m/CAMjNa7cDLzPcs0xnRpkvqmJ6Vb6G3EH8CYGp9ZBjXdpFfTz6dg@mail.gmail.com  

commit   : 52ca7572c3642ccbb46a619c03efe1928811ceae    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 30 Oct 2017 15:52:13 +0100    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 30 Oct 2017 15:52:13 +0100    

In autovacuum's "work item" processing, a few strings were allocated in  
the current transaction's memory context, which goes away during error  
handling; if an error happened during execution of the work item, the  
pfree() calls to clean up afterwards would try to release already-released  
memory, possibly leading to a crash.  In branch master, this was already  
fixed by commit 335f3d04e4c8, so backpatch that to REL_10_STABLE to fix  
the problem there too.  
  
As a secondary problem, verify that the autovacuum worker is connected  
to the right database for each work item; otherwise some items would be  
discarded by workers in other databases.  
  
Reported-by: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0f1fbe7122208f4734dcb285594d06e7ebd07b10    
  
author   : Magnus Hagander <[email protected]>    
date     : Mon, 30 Oct 2017 14:37:44 +0100    
  
committer: Magnus Hagander <[email protected]>    
date     : Mon, 30 Oct 2017 14:37:44 +0100    

commit   : a87c0c7631d2ec738e78b909f5dcea16ea1e832f    
  
author   : Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 22:22:39 +0200    
  
committer: Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 22:22:39 +0200    

This was always intended to work, but due to an oversight in  
max_parallel_hazard_walker, it didn't.  In testing, we missed the  
fact that it was only working for custom plans, where the parameter  
value has been substituted for the parameter itself early enough  
that everything worked.  In a generic plan, the Param node survives  
and must be treated as parallel-safe.  SerializeParamList provides  
for the transmission of parameter values to workers.  
  
Amit Kapila with help from Kuntal Ghosh.  Some changes by me.  
  
Discussion: http://postgr.es/m/CAA4eK1+_BuZrmVCeua5Eqnm4Co9DAXdM5HPAOE2J19ePbR912Q@mail.gmail.com  

commit   : 69125c883df30e6232412d35bf7c89b8c3ad1305    
  
author   : Robert Haas <[email protected]>    
date     : Sun, 29 Oct 2017 12:58:40 +0530    
  
committer: Robert Haas <[email protected]>    
date     : Sun, 29 Oct 2017 12:58:40 +0530    

Without this fix, dropping a role can sometimes result in parallel  
query failures in sessions that have used "SET ROLE" to assume the  
dropped role, even if that setting isn't active any more.  
  
Report by Pavan Deolasee.  Patch by Amit Kapila, reviewed by me.  
  
Discussion: http://postgr.es/m/CABOikdOomRcZsLsLK+Z+qENM1zxyaWnAvFh3MJZzZnnKiF+REg@mail.gmail.com  

commit   : 291a31c42c893574e9676e00121e6c6915a59de5    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 18:16:25 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 18:16:25 -0400    

Commit d5b760ecb wasn't quite right, on second thought: if the  
caller didn't ask for column names then it would happily emit  
more Vars than if the caller did ask for column names.  This  
is surely not a good idea.  Advance the aliasp_item whether or  
not we're preparing a colnames list.  

commit   : ddde3b4f3e888c06f7b866575be242357d9d72be    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 17:10:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 17:10:21 -0400    

expandRTE() supposed that an RTE_SUBQUERY subquery must have exactly  
as many non-junk tlist items as the RTE has column aliases for it.  
This was true at the time the code was written, and is still true so  
far as parse analysis is concerned --- but when the function is used  
during planning, the subquery might have appeared through insertion  
of a view that now has more columns than it did when the outer query  
was parsed.  This results in a core dump if, for instance, we have  
to expand a whole-row Var that references the subquery.  
  
To avoid crashing, we can either stop expanding the RTE when we run  
out of aliases, or invent new aliases for the added columns.  While  
the latter might be more useful, the former is consistent with what  
expandRTE() does for composite-returning functions in the RTE_FUNCTION  
case, so it seems like we'd better do it that way.  
  
Per bug #14876 from Samuel Horwitz.  This has been busted since commit  
ff1ea2173 allowed views to acquire more columns, so back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 8be1022425a395aa4d05b8032aeb9f6b128ea902    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 12:18:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 12:18:56 -0400    

On closer investigation, commits f3ea3e3e8 et al were a few bricks  
shy of a load.  What we need is not so much to lock down the result  
type of a FieldSelect, as to lock down the existence of the column  
it's trying to extract.  Otherwise, we can break it by dropping that  
column.  The dependency on the result type is then held indirectly  
through the column, and doesn't need to be recorded explicitly.  
  
Out of paranoia, I left in the code to record a dependency on the  
result type, but it's used only if we can't identify the pg_class OID  
for the column.  That shouldn't ever happen right now, AFAICS, but  
it seems possible that in future the input node could be marked as  
being of type RECORD rather than some specific composite type.  
  
Likewise for FieldStore.  
  
Like the previous patch, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 69fc2ca4ccdc70c6f9bd7909a771b0983017cb5c    
  
author   : Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 17:29:52 +0200    
  
committer: Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 17:29:52 +0200    

Reduces ABI breakage.  Per Tom Lane.  
  
Discussion: http://postgr.es/m/[email protected]  

commit   : 7b233fce1bb679fbc4adf79b4a06c85aedda8cd1    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 10:46:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 10:46:06 -0400    

The NOTIFY page said this already, but the PREPARE TRANSACTION page  
missed it.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 965a16fa9f8c316c30f4ada3c1c98b5dd4666f25    
  
author   : Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 16:04:01 +0200    
  
committer: Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 16:04:01 +0200    

If we try to run a parallel plan in serial mode because, for example,  
it's going to be scanned via a cursor, but for some reason we're  
already in parallel mode (for example because an outer query is  
running in parallel), we'd incorrectly try to launch workers.  
Fix by adding a flag to the EState, so that we can be certain that  
ExecutePlan() and ExecGather()/ExecGatherMerge() will have the same  
idea about whether we are executing serially or in parallel.  
  
Report and fix by Amit Kapila with help from Kuntal Ghosh.  A few  
tweaks by me.  
  
Discussion: http://postgr.es/m/CAA4eK1+_BuZrmVCeua5Eqnm4Co9DAXdM5HPAOE2J19ePbR912Q@mail.gmail.com  

commit   : 6a81ba1d4d26b46636d652a3a56143c159da899c    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 26 Oct 2017 12:17:40 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 26 Oct 2017 12:17:40 -0400    

Previously, we skipped using search_indexed_tlist_for_sortgroupref()  
if the tlist expression being sought in the child plan node was merely  
a Var.  This is purely an optimization, based on the theory that  
search_indexed_tlist_for_var() is faster, and one copy of a Var should  
be as good as another.  However, the GROUPING SETS patch broke the  
latter assumption: grouping columns containing the "same" Var can  
sometimes have different outputs, as shown in the test case added here.  
So do it the hard way whenever a ressortgroupref marking exists.  
  
(If this seems like a bottleneck, we could imagine building a tlist index  
data structure for ressortgroupref values, as we do for Vars.  But I'll  
let that idea go until there's some evidence it's worthwhile.)  
  
Back-patch to 9.6.  The problem also exists in 9.5 where GROUPING SETS  
came in, but this patch is insufficient to resolve the problem in 9.5:  
there is some obscure dependency on the upper-planner-pathification  
work that happened in 9.6.  Given that this is such a weird corner case,  
and no end users have complained about it, it doesn't seem worth the work  
to develop a fix for 9.5.  
  
Patch by me, per a report from Heikki Linnakangas.  (This does not fix  
Heikki's original complaint, just the follow-on one.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a9d4625f485abd458f8fcc263f0003235430401e    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 26 Oct 2017 10:01:02 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 26 Oct 2017 10:01:02 -0400    

There have been numerous buildfarm failures but the diagnostic is  
currently silent about the reason for failure to open the file. Let's  
see if we can get to the bottom of it.  
  
Backpatch to all live branches.  

commit   : aa828819aa59a84b486837b4cc3d7afc739332f5    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 26 Oct 2017 08:20:00 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 26 Oct 2017 08:20:00 -0400    

commit   : 65ba1b5c25365d53c69622f2bd658a309c7206bc    
  
author   : Michael Meskes <[email protected]>    
date     : Thu, 26 Oct 2017 10:16:04 +0200    
  
committer: Michael Meskes <[email protected]>    
date     : Thu, 26 Oct 2017 10:16:04 +0200    

Patch by Tsunakawa Takayuki <[email protected]>  

commit   : 51e9fffba0fa15d6a1d7d17b06290a991b8e2758    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 25 Oct 2017 19:32:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 25 Oct 2017 19:32:24 -0400    

Some people like to run libpq-using applications in environments where  
there's no home directory.  We've broken that scenario before (cf commits  
5b4067798 and bd58d9d88), and commit ba005f193 broke it again, by making  
it a hard error if we fail to get the home directory name while looking  
for ~/.pgpass.  The previous precedent is that if we can't get the home  
directory name, we should just silently act as though the file we hoped  
to find there doesn't exist.  Rearrange the new code to honor that.  
  
Looking around, the service-file code added by commit 41a4e4595 had the  
same disease.  Apparently, that escaped notice because it only runs when  
a service name has been specified, which I guess the people who use this  
scenario don't do.  Nonetheless, it's wrong too, so fix that case as well.  
  
Add a comment about this policy to pqGetHomeDirectory, in the probably  
vain hope of forestalling the same error in future.  And upgrade the  
rather miserable commenting in parseServiceInfo, too.  
  
In passing, also back off parseServiceInfo's assumption that only ENOENT  
is an ignorable error from stat() when checking a service file.  We would  
need to ignore at least ENOTDIR as well (cf 5b4067798), and seeing that  
the far-better-tested code for ~/.pgpass treats all stat() failures alike,  
I think this code ought to as well.  
  
Per bug #14872 from Dan Watson.  Back-patch the .pgpass change to v10  
where ba005f193 came in.  The service-file bugs are far older, so  
back-patch the other changes to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 06d5ba0d49d2ad756c14cad1d1281e8f8092d0b1    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:34:00 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:34:00 -0400    

json_build_object and json_build_array and the jsonb equivalents did not  
correctly process explicit VARIADIC arguments. They are modified to use  
the new extract_variadic_args() utility function which abstracts away  
the details of the call method.  
  
Michael Paquier, reviewed by Tom Lane and Dmitry Dolgov.  
  
Backpatch to 9.5 for the jsonb fixes and 9.4 for the json fixes, as  
that's where they originated.  

commit   : fb17082d787ec6fbd8fe850f19a58c4a0fae72b9    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:13:11 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:13:11 -0400    

This is epecially useful in the case or "VARIADIC ANY" functions. The  
caller can get the artguments and types regardless of whether or not and  
explicit VARIADIC array argument has been used. The function also  
provides an option to convert arguments on type "unknown" to to "text".  
  
Michael Paquier and me, reviewed by Tom Lane.  
  
Backpatch to 9.4 in order to support the following json bug fix.  

commit   : eccd9d9ff50b9f9124f5f76c62aa7c2c0ced5b87    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 24 Oct 2017 18:42:47 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 24 Oct 2017 18:42:47 -0400    

Although joinaliasvars lists coming out of the parser are quite simple,  
those lists can contain arbitrarily complex expressions after subquery  
pullup.  We do not perform expression preprocessing on them, meaning that  
expressions in those lists will not meet the expectations of later phases  
of the planner (for example, that they do not contain SubLinks).  This had  
been thought pretty harmless, since we don't intentionally touch those  
lists in later phases --- but Andreas Seltenreich found a case in which  
adjust_appendrel_attrs() could recurse into a joinaliasvars list and then  
die on its assertion that it never sees a SubLink.  We considered a couple  
of localized fixes to prevent that specific case from looking at the  
joinaliasvars lists, but really this seems like a generic hazard for all  
expression processing in the planner.  Therefore, probably the best answer  
is to delete the joinaliasvars lists from the parsetree at the end of  
expression preprocessing, so that there are no reachable expressions that  
haven't been through preprocessing.  
  
The case Andreas found seems to be harmless in non-Assert builds, and so  
far there are no field reports suggesting that there are user-visible  
effects in other cases.  I considered back-patching this anyway, but  
it turns out that Andreas' test doesn't fail at all in 9.4-9.6, because  
in those versions adjust_appendrel_attrs contains code (added in commit  
842faa714 and removed again in commit 215b43cdc) to process SubLinks  
rather than complain about them.  Barring discovery of another path by  
which unprocessed joinaliasvars lists can cause trouble, the most  
prudent compromise seems to be to patch this into v10 but not further.  
  
Patch by me, with thanks to Amit Langote for initial investigation  
and review.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0cde562473b87a32c089d3b079b2b8f259056800    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 18:15:36 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 18:15:36 -0400    

DST law changes in Fiji, Namibia, Northern Cyprus, Sudan, Tonga,  
and Turks & Caicos Islands.  Historical corrections for Alaska, Apia,  
Burma, Calcutta, Detroit, Ireland, Namibia, and Pago Pago.  

commit   : dffe7fbc2e88293821e31fa2ce5c0d3f83e4318c    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 17:54:09 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 17:54:09 -0400    

This is a trivial update containing only cosmetic changes.  The point  
is just to get back to being synced with an official release of tzcode,  
rather than some ad-hoc point in their commit history, which is where  
commit 47f849a3c left it.  

commit   : df4aa6e4e874ead86788371ecb4cf7909f2485c8    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 13:57:45 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 13:57:45 -0400    

find_expr_references() neglected to record a dependency on the result type  
of a FieldSelect node, allowing a DROP TYPE to break a view or rule that  
contains such an expression.  I think we'd omitted this case intentionally,  
reasoning that there would always be a related dependency ensuring that the  
DROP would cascade to the view.  But at least with nested field selection  
expressions, that's not true, as shown in bug #14867 from Mansur Galiev.  
Add the dependency, and for good measure a dependency on the node's exposed  
collation.  
  
Likewise add a dependency on the result type of a FieldStore.  I think here  
the reasoning was that it'd only appear within an assignment to a field,  
and the dependency on the field's column would be enough ... but having  
seen this example, I think that's wrong for nested-composites cases.  
  
Looking at nearby code, I notice we're not recording a dependency on the  
exposed collation of CoerceViaIO, which seems inconsistent with our choices  
for related node types.  Maybe that's OK but I'm feeling suspicious of this  
code today, so let's add that; it certainly can't hurt.  
  
This patch does not do anything to protect already-existing views, only  
views created after it's installed.  But seeing that the issue has been  
there a very long time and nobody noticed till now, that's probably good  
enough.  
  
Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 852e3224e76e55f7e2046643d98d7d0786439a9c    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 22 Oct 2017 16:45:16 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 22 Oct 2017 16:45:16 -0400    

It seems that the parray_gin extension has seen fit to introduce a  
"text[] @> text[]" operator, which conflicts with the core  
"anyarray @> anyarray" operator, causing ambiguous-operator failures  
if the input arguments are coercible to text[] without being exactly  
that type.  This strikes me as a bad idea, but it's out there and  
people use it.  As of v10, that breaks psql's query that tries to  
test "pg_statistic_ext.stxkind @> '{d}'", since stxkind is char[].  
The best workaround seems to be to avoid use of that operator.  
We can use a scalar-vs-array test "'d' = any(stxkind)" instead;  
that's arguably more readable anyway.  
  
Per report from Justin Pryzby.  Backpatch to v10 where this  
query was added.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 7a5f8de5525a861af5917c73e78efe15ea3e1bb2    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 20 Oct 2017 17:12:27 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 20 Oct 2017 17:12:27 -0400    

Like the similar logic for arrays and records, it's necessary to examine  
the range's subtype to decide whether the range type can support hashing.  
We can omit checking the subtype for btree-defined operations, though,  
since range subtypes are required to have those operations.  (Possibly  
that simplification for btree cases led us to overlook that it does  
not apply for hash cases.)  
  
This is only an issue if the subtype lacks hash support, which is not  
true of any built-in range type, but it's easy to demonstrate a problem  
with a range type over, eg, money: you can get a "could not identify  
a hash function" failure when the planner is misled into thinking that  
hash join or aggregation would work.  
  
This was born broken, so back-patch to all supported branches.  

commit   : 0ab77a34f8efef0f43fece830bd2004c103e6f91    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 19 Oct 2017 11:16:18 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 19 Oct 2017 11:16:18 -0400    

As noted by M. Justin.  
  
Also, to keep the HEAD and REL_10 versions of release-10.sgml in sync,  
back-patch the effects of c29c57890 on that file.  We have a bigger  
problem there though :-(  
  
Discussion: https://postgr.es/m/CALtA7pmsQyTTD3fC2rmfUWgfivv5sCJJ84PHY0F_5t_SRc07Qg@mail.gmail.com  
Discussion: https://postgr.es/m/[email protected]  

commit   : 9d75dfd0e5f211c56e6bfd79f0af4a1777efe6d3    
  
author   : Magnus Hagander <[email protected]>    
date     : Thu, 19 Oct 2017 13:59:19 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Thu, 19 Oct 2017 13:59:19 +0200    

David Rowley  

commit   : 6eebac2289357067b652485343663c0d994445d2    
  
author   : Magnus Hagander <[email protected]>    
date     : Thu, 19 Oct 2017 13:56:21 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Thu, 19 Oct 2017 13:56:21 +0200    

Spotted by Piotr Stefaniak  

commit   : 11cbab7a12825b828f37e3b1fb885567d74b0398    
  
author   : Alvaro Herrera <[email protected]>    
date     : Wed, 18 Oct 2017 13:21:43 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Wed, 18 Oct 2017 13:21:43 +0200    

Author: David G. Johnston  
Discussion: https:/postgr.es/m/CAKFQuwaCsb-OKOjQXGeN0R7byxiRWvr7OtyKDbJoYgiF2vBG4Q@mail.gmail.com  

commit   : 799037099b4f0a227bed40eee53bf4e995eb5390    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 16 Oct 2017 17:56:43 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 16 Oct 2017 17:56:43 -0400    

setTargetTable threw an error if the proposed target RangeVar's relname  
matched any visible CTE or ENR.  This breaks backwards compatibility in  
the CTE case, since pre-v10 we never looked for a CTE here at all, so that  
CTE names did not mask regular tables.  It does seem like a good idea to  
throw an error for the ENR case, though, thus causing ENRs to mask tables  
for this purpose; ENRs are new in v10 so we're not breaking existing code,  
and we may someday want to allow them to be the targets of DML.  
  
To fix that, replace use of getRTEForSpecialRelationTypes, which was  
overkill anyway, with use of scanNameSpaceForENR.  
  
A second problem was that the check neglected to verify null schemaname,  
so that a CTE or ENR could incorrectly be thought to match a qualified  
RangeVar.  That happened because getRTEForSpecialRelationTypes relied  
on its caller to have checked for null schemaname.  Even though the one  
remaining caller got it right, this is obviously bug-prone, so move  
the check inside getRTEForSpecialRelationTypes.  
  
Also, revert commit 18ce3a4ab's extremely poorly thought out decision to  
add a NULL return case to parserOpenTable --- without either documenting  
that or adjusting any of the callers to check for it.  The current bug  
seems to have arisen in part due to working around that bad idea.  
  
In passing, remove the one-line shim functions transformCTEReference and  
transformENRReference --- they don't seem to be adding any clarity or  
functionality.  
  
Per report from Hugo Mercier (via Julien Rouhaud).  Back-patch to v10  
where the bug was introduced.  
  
Thomas Munro, with minor editing by me  
  
Discussion: https://postgr.es/m/CAOBaU_YdPVH+PTtiKSSLOiiW3mVDYsnNUekK+XPbHXiP=wrFLA@mail.gmail.com  

commit   : 72e9cc9715098e5e2b8c49b5707d876632bea2d3    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 16 Oct 2017 15:24:36 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 16 Oct 2017 15:24:36 -0400    

An aggregate's input expression(s) are not supposed to be evaluated  
at all for a row where its FILTER test fails ... but commit 8ed3f11bb  
overlooked that requirement.  Reshuffle so that aggregates having a  
filter clause evaluate their arguments separately from those without.  
This still gets the benefit of doing only one ExecProject in the  
common case of multiple Aggrefs, none of which have filters.  
  
While at it, arrange for filter clauses to be included in the common  
ExecProject evaluation, thus perhaps buying a little bit even when  
there are filters.  
  
Back-patch to v10 where the bug was introduced.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : cb591fcbfbba1df6fda1839ece53665e85e491e3    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 15 Oct 2017 19:19:19 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 15 Oct 2017 19:19:19 -0400    

While poking around in the aggregate logic, I noticed that commit  
8ed3f11bb broke the logic in nodeAgg.c that purports to detect nested  
aggregates, by moving initialization of regular aggregate argument  
expressions out of the code segment that checks for that.  
  
You could argue that this check is unnecessary, but it's not much code  
so I'm inclined to keep it as a backstop against parser and planner  
bugs.  However, there's certainly zero value in checking only some of  
the subexpressions.  
  
We can make the check complete again, and as a bonus make it a good  
deal more bulletproof against future mistakes of the same ilk, by  
moving it out to the outermost level of ExecInitAgg.  This means we  
need to check only once per Agg node not once per aggregate, which  
also seems like a good thing --- if the check does find something  
wrong, it's not urgent that we report it before the plan node  
initialization finishes.  
  
Since this requires remembering the original length of the aggs list,  
I deleted a long-obsolete stanza that changed numaggs from 0 to 1.  
That's so old it predates our decision that palloc(0) is a valid  
operation, in (digs...) 2004, see commit 24a1e20f1.  
  
In passing improve a few comments.  
  
Back-patch to v10, just in case.  

commit   : a3b1c221893f739950e9232b4b789750f247cee5    
  
author   : Robert Haas <[email protected]>    
date     : Fri, 13 Oct 2017 14:53:28 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Fri, 13 Oct 2017 14:53:28 -0400    

If a Parallel Bitmap Heap scan's chain of leftmost descendents  
includes a BitmapOr whose first child is a BitmapAnd, the prior coding  
would mistakenly create a non-shared TIDBitmap and then try to perform  
shared iteration.  
  
Report by Tomas Vondra.  Patch by Dilip Kumar.  
  
Discussion: http://postgr.es/m/[email protected]  

commit   : d48bf6a94d295c3779c6af4df118d95a6606192f    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 12 Oct 2017 15:20:04 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 12 Oct 2017 15:20:04 -0400    

If we merge the transition calculations for two different aggregates,  
it's reasonable to assume that the transition function should not care  
which of those Aggref structs it gets from AggGetAggref().  It is not  
reasonable to make the same assumption about an aggregate final function,  
however.  Commit 804163bc2 broke this, as it will pass whichever Aggref  
was first associated with the transition state in both cases.  
  
This doesn't create an observable bug so far as the core system is  
concerned, because the only existing uses of AggGetAggref() are in  
ordered-set aggregates that happen to not pay attention to anything  
but the input properties of the Aggref; and besides that, we disabled  
sharing of transition calculations for OSAs yesterday.  Nonetheless,  
if some third-party code were using AggGetAggref() in a normal aggregate,  
they would be entitled to call this a bug.  Hence, back-patch the fix  
to 9.6 where the problem was introduced.  
  
In passing, improve some of the comments about transition state sharing.  
  
Discussion: https://postgr.es/m/CAB4ELO5RZhOamuT9Xsf72ozbenDLLXZKSk07FiSVsuJNZB861A@mail.gmail.com  

commit   : 5c926e68ea61d0b795ad5438e7223e88d3e2c93f    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 12 Oct 2017 11:35:54 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 12 Oct 2017 11:35:54 -0400    

Ioseph Kim  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : a01a3d931e5a14d60eec75f7945d5e66ff9cc575    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 12 Oct 2017 17:23:47 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 12 Oct 2017 17:23:47 +0200    

Vars hidden within a RelabelType would not be detected as compatible  
with some functional dependency.  Repair by properly ignoring the  
RelabelType.  
  
Author: David Rowley  
Reviewed-by: Tomas Vondra  
Discussion: https://postgr.es/m/CAKJS1f-y-UEy=rsBXynBOgiW1fKMr_LVoYSGL9QOc36mLEC-ww@mail.gmail.com  

commit   : 7cde649ab150412344ee50fc90f24d6fe891bcf0    
  
author   : Robert Haas <[email protected]>    
date     : Thu, 12 Oct 2017 10:09:26 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Thu, 12 Oct 2017 10:09:26 -0400    

Before, that would fail to happen unless a BEFORE ROW UPDATE trigger  
was also present.  
  
Noted by me while reviewing a patch from Masahiko Sawada, who also  
wrote this patch.  Reviewed by Petr Jelinek.  
  
Discussion: http://postgr.es/m/CA+TgmobAZvCxduG8y_mQKBK7nz-vhbdLvjM354KEFozpuzMN5A@mail.gmail.com  

commit   : 604723d29a86f38551e43997e6f40daa94f54b27    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 22:18:01 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 22:18:01 -0400    

This ought to work, but the built-in OSAs are not capable of coping,  
because their final-functions destructively modify their transition  
state (specifically, the contained tuplesort object).  That was fine  
when those functions were written, but commit 804163bc2 moved the  
goalposts without telling orderedsetaggs.c.  
  
We should fix the built-in OSAs to support this, but it will take  
a little work, especially if we don't want to sacrifice performance  
in the normal non-shared-state case.  Given that it took a year after  
9.6 release for anyone to notice this bug, we should not prioritize  
sharable-state over nonsharable-state performance.  And a proper fix  
is likely to be more complicated than we'd want to back-patch, too.  
  
Therefore, let's just put in this stop-gap patch to prevent nodeAgg.c  
from choosing to use shared state for OSAs.  We can revert it in HEAD  
when we get a better fix.  
  
Report from Lukas Eder, diagnosis by me, patch by David Rowley.  
Back-patch to 9.6 where the problem was introduced.  
  
Discussion: https://postgr.es/m/CAB4ELO5RZhOamuT9Xsf72ozbenDLLXZKSk07FiSVsuJNZB861A@mail.gmail.com  

commit   : 61ace8fe7fe82dc04c1de493a414597989f05e56    
  
author   : Andres Freund <[email protected]>    
date     : Wed, 11 Oct 2017 12:03:26 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Wed, 11 Oct 2017 12:03:26 -0700    

The previous coding in ProcessInterrupts() could lead to  
idle_in_transaction_session_timeout being ignored, when  
statement_timeout occurred earlier.  
  
The problem was that ProcessInterrupts() would return before  
processing the transaction timeout if QueryCancelPending was set while  
QueryCancelHoldoffCount != 0 - which is the case when reading new  
commands from the client. Ergo when the idle transaction timeout would  
hit.  
  
Fix that by removing the early return. Alternatively the transaction  
timeout code could have been moved up, but that early return seems  
like an issue that could hit other cases too.  
  
Author: Lukas Fittl  
Bug: #14821  
Discussion:  
    https://www.postgresql.org/message-id/20170921010956.17345.61461%40wrigleys.postgresql.org  
    https://www.postgresql.org/message-id/CAP53PkxQnv3OWJpyNPGJYT62uY=n1=2CF_Lpc6gVOFnc0-gazw@mail.gmail.com  
Backpatch: 9.6-, where idle_in_transaction_session_timeout was introduced.  

commit   : fbac00a4a8d3d9ff97ceb7176fce038fea5239b5    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 16:56:23 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 16:56:23 -0400    

The GRANT reference page, which lists the default privileges for new  
objects, failed to mention that USAGE is granted by default for data  
types and domains.  As a lesser sin, it also did not specify anything  
about the initial privileges for sequences, FDWs, foreign servers,  
or large objects.  Fix that, and add a comment to acldefault() in the  
probably vain hope of getting people to maintain this list in future.  
  
Noted by Laurenz Albe, though I editorialized on the wording a bit.  
Back-patch to all supported branches, since they all have this behavior.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 516e29ab6f1f66767a78361dea0392f1530766b6    
  
author   : Robert Haas <[email protected]>    
date     : Wed, 11 Oct 2017 15:55:10 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Wed, 11 Oct 2017 15:55:10 -0400    

Masahiko Sawada  
  
Discussion: http://postgr.es/m/CAD21AoBsfYsMHD6_SL9iN3n_Foaa+oPbL5jG55DxU1ChaujqwQ@mail.gmail.com  

commit   : f4cdf781a1d2188bdaa6a79c5b496b67a1f63541    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 14:28:33 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 14:28:33 -0400    

Up to now async.c has used TransactionIdIsInProgress() to detect whether  
a notify message's source transaction is still running.  However, that  
function has a quick-exit path that reports that XIDs before RecentXmin  
are no longer running.  If a listening backend is doing nothing but  
listening, and not running any queries, there is nothing that will advance  
its value of RecentXmin.  Once 2 billion transactions elapse, the  
RecentXmin check causes active transactions to be reported as not running.  
If they aren't committed yet according to CLOG, async.c decides they  
aborted and discards their messages.  The timing for that is a bit tight  
but it can happen when multiple backends are sending notifies concurrently.  
The net symptom therefore is that a sufficiently-long-surviving  
listen-only backend starts to miss some fraction of NOTIFY traffic,  
but only under heavy load.  
  
The only function that updates RecentXmin is GetSnapshotData().  
A brute-force fix would therefore be to take a snapshot before  
processing incoming notify messages.  But that would add cycles,  
as well as contention for the ProcArrayLock.  We can be smarter:  
having taken the snapshot, let's use that to check for running  
XIDs, and not call TransactionIdIsInProgress() at all.  In this  
way we reduce the number of ProcArrayLock acquisitions from one  
per message to one per notify interrupt; that's the same under  
light load but should be a benefit under heavy load.  Light testing  
says that this change is a wash performance-wise for normal loads.  
  
I looked around for other callers of TransactionIdIsInProgress()  
that might be at similar risk, and didn't find any; all of them  
are inside transactions that presumably have already taken a  
snapshot.  
  
Problem report and diagnosis by Marko Tiikkaja, patch by me.  
Back-patch to all supported branches, since it's been like this  
since 9.0.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : f3b45b6738aa4c6b77508e319de9254a2a7b2de7    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 10 Oct 2017 12:51:09 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 10 Oct 2017 12:51:09 -0400    

I noticed the tmp_check subdirectory wasn't getting cleaned up  
after a check-world run.  Apparently pgxs.mk will only do this  
for you if you've defined REGRESS.  The only other src/test/modules  
Makefile that does not set that is snapshot_too_old, and it  
does it like this.  

commit   : c3723317d08ce6df6c81234f7daebe7fd86a6ecb    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 8 Oct 2017 15:25:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 8 Oct 2017 15:25:26 -0400    

copy_file() reads and writes data 64KB at a time (with default BLCKSZ),  
and historically has issued a pg_flush_data request after each write.  
This turns out to interact really badly with macOS's new APFS file  
system: a large file copy takes over 100X longer than it ought to on  
APFS, as reported by Brent Dearth.  While that's arguably a macOS bug,  
it's not clear whether Apple will do anything about it in the near  
future, and in any case experimentation suggests that issuing flushes  
a bit less often can be helpful on other platforms too.  
  
Hence, rearrange the logic in copy_file() so that flush requests are  
issued once per N writes rather than every time through the loop.  
I set the FLUSH_DISTANCE to 32MB on macOS (any less than that still  
results in a noticeable speed degradation on APFS), but 1MB elsewhere.  
In limited testing on Linux and FreeBSD, this seems slightly faster  
than the previous code, and certainly no worse.  It helps noticeably  
on macOS even with the older HFS filesystem.  
  
A simpler change would have been to just increase the size of the  
copy buffer without changing the loop logic, but that seems likely  
to trash the processor cache without really helping much.  
  
Back-patch to 9.6 where we introduced msync() as an implementation  
option for pg_flush_data().  The problem seems specific to APFS's  
mmap/msync support, so I don't think we need to go further back.  
  
Discussion: https://postgr.es/m/CADkxhTNv-j2jw2g8H57deMeAbfRgYBoLmVuXkC=YCFBXRuCOww@mail.gmail.com  

commit   : 485d49dbc9ed246b26d5033c0774238f70feb1b6    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 19:18:58 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 19:18:58 -0400    

The logical decoding functions do BeginInternalSubTransaction and  
RollbackAndReleaseCurrentSubTransaction to clean up after themselves.  
It turns out that AtEOSubXact_SPI has an unrecognized assumption that  
we always need to cancel the active SPI operation in the SPI context  
that surrounds the subtransaction (if there is one).  That's true  
when the RollbackAndReleaseCurrentSubTransaction call is coming from  
the SPI-using function itself, but not when it's happening inside  
some unrelated function invoked by a SPI query.  In practice the  
affected callers are the various PLs.  
  
To fix, record the current subtransaction ID when we begin a SPI  
operation, and clean up only if that ID is the subtransaction being  
canceled.  
  
Also, remove AtEOSubXact_SPI's assertion that it must have cleaned  
up the surrounding SPI context's active tuptable.  That's proven  
wrong by the same test case.  
  
Also clarify (or, if you prefer, reinterpret) the calling conventions  
for _SPI_begin_call and _SPI_end_call.  The memory context cleanup  
in the latter means that these have always had the flavor of a matched  
resource-management pair, but they weren't documented that way before.  
  
Per report from Ben Chobot.  
  
Back-patch to 9.4 where logical decoding came in.  In principle,  
the SPI changes should go all the way back, since the problem dates  
back to commit 7ec1c5a86.  But given the lack of field complaints  
it seems few people are using internal subtransactions in this way.  
So I don't feel a need to take any risks in 9.2/9.3.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0c25e9652461c08b5caef259a6af27a38707e07a    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 14:28:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 14:28:42 -0400    

Both ExecMakeFunctionResultSet() and evaluation of simple expressions  
need to be done in the per-tuple memory context, not per-query, else  
we leak data until end of query.  This is a consideration that was  
missed while refactoring code in the ProjectSet patch (note that in  
pre-v10, ExecMakeFunctionResult is called in the per-tuple context).  
  
Per bug #14843 from Ben M.  Diagnosed independently by Andres and myself.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 4c20ee5f27a2edbf89273e44eb369fd92d4fe3b2    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 12:20:13 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 12:20:13 -0400    

Sloppy loop coding in set_status_by_pages() resulted in fetching one array  
element more than it should from the subxids[] array.  The odds of this  
resulting in SIGSEGV are pretty small, but we've certainly seen that happen  
with similar mistakes elsewhere.  While at it, we can get rid of an extra  
TransactionIdToPage() calculation per loop.  
  
Per report from David Binderman.  Back-patch to all supported branches,  
since this code is quite old.  
  
Discussion: https://postgr.es/m/HE1PR0802MB2331CBA919CBFFF0C465EB429C710@HE1PR0802MB2331.eurprd08.prod.outlook.com  

commit   : 22576734b805fb0952f9be841ca8f643694ee868    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 6 Oct 2017 17:14:42 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 6 Oct 2017 17:14:42 +0200    

When some tuple versions in an update chain are frozen due to them being  
older than freeze_min_age, the xmax/xmin trail can become broken.  This  
breaks HOT (and probably other things).  A subsequent VACUUM can break  
things in more serious ways, such as leaving orphan heap-only tuples  
whose root HOT redirect items were removed.  This can be seen because  
index creation (or REINDEX) complain like  
  ERROR:  XX000: failed to find parent tuple for heap-only tuple at (0,7) in table "t"  
  
Because of relfrozenxid contraints, we cannot avoid the freezing of the  
early tuples, so we must cope with the results: whenever we see an Xmin  
of FrozenTransactionId, consider it a match for whatever the previous  
Xmax value was.  
  
This problem seems to have appeared in 9.3 with multixact changes,  
though strictly speaking it seems unrelated.  
  
Since 9.4 we have commit 37484ad2a "Change the way we mark tuples as  
frozen", so the fix is simple: just compare the raw Xmin (still stored  
in the tuple header, since freezing merely set an infomask bit) to the  
Xmax.  But in 9.3 we rewrite the Xmin value to FrozenTransactionId, so  
the original value is lost and we have nothing to compare the Xmax with.  
To cope with that case we need to compare the Xmin with FrozenXid,  
assume it's a match, and hope for the best.  Sadly, since you can  
pg_upgrade a 9.3 instance containing half-frozen pages to newer  
releases, we need to keep the old check in newer versions too, which  
seems a bit brittle; I hope we can somehow get rid of that.  
  
I didn't optimize the new function for performance.  The new coding is  
probably a bit slower than before, since there is a function call rather  
than a straight comparison, but I'd rather have it work correctly than  
be fast but wrong.  
  
This is a followup after 20b655224249 fixed a few related problems.  
Apparently, in 9.6 and up there are more ways to get into trouble, but  
in 9.3 - 9.5 I cannot reproduce a problem anymore with this patch, so  
there must be a separate bug.  
  
Reported-by: Peter Geoghegan  
Diagnosed-by: Peter Geoghegan, Michael Paquier, Daniel Wood,  
	Yi Wen Wong, Álvaro  
Discussion: https://postgr.es/m/CAH2-Wznm4rCrhFAiwKPWTpEw2bXDtgROZK7jWWGucXeH3D1fmA@mail.gmail.com  

commit   : ce60c2a2825384ca46e11ccdb418c39c32bf4dc5    
  
author   : Robert Haas <[email protected]>    
date     : Thu, 5 Oct 2017 07:58:02 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Thu, 5 Oct 2017 07:58:02 -0400    

Michael Paquier discovered that this could be triggered via SQL;  
give a nicer message instead.  
  
Patch by Michael Paquier, reviewed by Masahiko Sawada.  
  
Discussion: http://postgr.es/m/CAB7nPqQtPg+LKKtzdKN26judHcvPZ0s1gNigzOT4j8CYuuuBYg@mail.gmail.com  

commit   : 2451de7e9e72943dbc71a3749379cf16946920f0    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 3 Oct 2017 14:00:57 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 3 Oct 2017 14:00:57 -0400    

Commit 597a87ccc introduced a latch pointer variable to replace use  
of a long-lived shared latch in the shared WalRcvData structure.  
This was not well thought out, because there are now hazards of the  
pointer variable changing while it's being inspected by another  
process.  This could obviously lead to a core dump in code like  
  
	if (WalRcv->latch)  
		SetLatch(WalRcv->latch);  
  
and there's a more remote risk of a torn read, if we have any  
platforms where reading/writing a pointer is not atomic.  
  
An actual problem would occur only if the walreceiver process  
exits (gracefully) while the startup process is trying to  
signal it, but that seems well within the realm of possibility.  
  
To fix, treat the pointer variable (not the referenced latch)  
as being protected by the WalRcv->mutex spinlock.  There  
remains a race condition that we could apply SetLatch to a  
process latch that no longer belongs to the walreceiver, but  
I believe that's harmless: at worst it'd cause an extra wakeup  
of the next process to use that PGPROC structure.  
  
Back-patch to v10 where the faulty code was added.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ad40d5f74585d1f14ea4b73c20fb5b2dabfb153f    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 3 Oct 2017 14:58:25 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 3 Oct 2017 14:58:25 +0200    

1. Since commit b1a9bad9e744 we had pstrdup() inside a  
spinlock-protected critical section; reported by Andreas Seltenreich.  
Turn those into strlcpy() to stack-allocated variables instead.  
Backpatch to 9.6.  
  
2. Since commit 9ed551e0a4fd we had a pfree() uselessly inside a  
spinlock-protected critical section.  Tom Lane noticed in code review.  
Move down.  Backpatch to 9.6.  
  
3. Since commit 64233902d22b we had GetCurrentTimestamp() (a kernel  
call) inside a spinlock-protected critical section.  Tom Lane noticed in  
code review.  Move it up.  Backpatch to 9.2.  
  
4. Since commit 1bb2558046cc we did elog(PANIC) while holding spinlock.  
Tom Lane noticed in code review.  Release spinlock before dying.  
Backpatch to 9.2.  
  
Discussion: https://postgr.es/m/[email protected]