PostgreSQL 9.6.6 commit log

commit   : 0a13f1966d4cda2eb49dec0324ab5d482bbdd956    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 17:08:55 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 17:08:55 -0500    

commit   : d69c0710a68068c7a415aaefd2c7d51f3197fe38    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 12:02:30 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 12:02:30 -0500    

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099  

commit   : 38e825632be777a6ea4a39796e121c39728403b3    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 10:29:16 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 6 Nov 2017 10:29:16 -0500    

json{b}_populate_recordset() used the tuple descriptor created from the  
query-level AS clause without worrying about whether it matched the actual  
input record type.  If it didn't, that would usually result in a crash,  
though disclosure of server memory contents seems possible as well, for a  
skilled attacker capable of issuing crafted SQL commands.  Instead, use  
the query-supplied descriptor only when there is no input tuple to look at,  
and otherwise get a tuple descriptor based on the input tuple's own type  
marking.  The core code will detect any type mismatch in the latter case.  
  
Michael Paquier and Tom Lane, per a report from David Rowley.  
Back-patch to 9.3 where this functionality was introduced.  
  
Security: CVE-2017-15098  

commit   : b7d6f75072b3569d7d2acce04669e72086b547cb    
  
author   : Noah Misch <[email protected]>    
date     : Mon, 6 Nov 2017 07:11:10 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Mon, 6 Nov 2017 07:11:10 -0800    

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER  
replaces $PGLOG with a symbolic link, the server will corrupt the  
link-targeted file by appending log messages.  Since these scripts open  
$PGLOG as root, the attack works regardless of target file ownership.  
  
"make install" does not install these scripts anywhere.  Users having  
manually installed them in the past should repeat that process to  
acquire this fix.  Most script users have $PGLOG writable to root only,  
located in $PGDATA.  Just before updating one of these scripts, such  
users should rename $PGLOG to $PGLOG.old.  The script will then recreate  
$PGLOG with proper ownership.  
  
Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.  
  
Security: CVE-2017-12172  

commit   : 1f23d1cd21ed46dba882729bedd9c40b71995989    
  
author   : Dean Rasheed <[email protected]>    
date     : Mon, 6 Nov 2017 09:16:24 +0000    
  
committer: Dean Rasheed <[email protected]>    
date     : Mon, 6 Nov 2017 09:16:24 +0000    

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT  
permission on the columns of the arbiter index, but it failed to check  
for that in the case of an arbiter specified by constraint name.  
  
In addition, for a table with row level security enabled, it failed to  
check updated rows against the table's SELECT policies when the update  
path was taken (regardless of how the arbiter index was specified).  
  
Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.  
  
Security: CVE-2017-15099  

commit   : 971983f42fe6e8b90490a76a649f0c92905d7d47    
  
author   : Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 18:51:08 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 18:51:08 -0800    

Makefile.global assigns this prerequisite to every target named "check",  
but similar targets must mention it explicitly.  Affected targets  
failed, tested $PATH binaries, or tested a stale temporary installation.  
The src/test/modules examples worked properly when called as "make -C  
src/test/modules/$FOO check", but "make -j" allowed the test to start  
before the temporary installation was in place.  Back-patch to 9.5,  
where commit dcae5faccab64776376d354decda0017c648bb53 introduced the  
shared temp-install.  

commit   : 4077723cbf0900776a8424d0a65474f0635afe45    
  
author   : Peter Eisentraut <[email protected]>    
date     : Sun, 5 Nov 2017 17:01:24 -0500    
  
committer: Peter Eisentraut <[email protected]>    
date     : Sun, 5 Nov 2017 17:01:24 -0500    

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 078119e4b862f81bbb7207eff6788efc99ec4d97  

commit   : efa7dfaf7334b9cc8b40e36cd1d507674b8ea9fc    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 5 Nov 2017 13:47:56 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 5 Nov 2017 13:47:56 -0500    

In the v10 branch, also back-patch the effects of 1ff01b390 and c29c57890  
on these files, to reduce future maintenance issues.  (I'd do it further  
back, except that the 9.X branches differ anyway due to xlog-to-wal  
link tag renaming.)  

commit   : 1cac62dac0cfdcacdb2c2b1577dadf101ef7f1f4    
  
author   : Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 09:25:52 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sun, 5 Nov 2017 09:25:52 -0800    

This restores the ability, essentially lost in commit  
ffaa44cb559db332baeee7d25dedd74a61974203, to use COPY FREEZE under  
REPEATABLE READ isolation.  Back-patch to 9.4, like that commit.  
  
Reviewed by Tom Lane.  
  
Discussion: https://postgr.es/m/CA+TgmoahWDm-7fperBxzU9uZ99LPMUmEpSXLTw9TmrOgzwnORw@mail.gmail.com  

commit   : bd8e2b3cf4cfe5f26ed314af662ef988b4cfad4b    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 3 Nov 2017 17:23:13 +0100    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 3 Nov 2017 17:23:13 +0100    

If a process is extending a table concurrently with some BRIN  
summarization process, it is possible for the latter to miss pages added  
by the former because the number of pages is computed ahead of time.  
  
Fix by determining a fresh relation size after inserting the placeholder  
tuple: any process that further extends the table concurrently will  
update the placeholder tuple, while previous pages will be processed by  
the heap scan.  
  
Reported-by: Tomas Vondra  
Reviewed-by: Tom Lane  
Author: Álvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-to: 9.5  

commit   : 6cf68e223643203494292c2787b72a7353264c7d    
  
author   : Michael Meskes <[email protected]>    
date     : Fri, 3 Nov 2017 11:14:30 +0100    
  
committer: Michael Meskes <[email protected]>    
date     : Fri, 3 Nov 2017 11:14:30 +0100    

commit   : 049dab00942165c3df3aeace9e8e86cd2679dce0    
  
author   : Michael Meskes <[email protected]>    
date     : Thu, 2 Nov 2017 20:46:34 +0100    
  
committer: Michael Meskes <[email protected]>    
date     : Thu, 2 Nov 2017 20:46:34 +0100    

commit   : a43cd427e7033771645abd4f9dbd2b1b2ab5ed5c    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 2 Nov 2017 12:54:23 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 2 Nov 2017 12:54:23 -0400    

In some cases the BRIN code releases lock on an index page, and later  
re-acquires lock and tries to check that the tuple it was working on is  
still there.  That check was a couple bricks shy of a load.  It didn't  
consider that the page might have turned into a "revmap" page.  (The  
samepage code path doesn't call brin_getinsertbuffer(), so it isn't  
protected by the checks for revmap status there.)  It also didn't check  
whether the tuple offset was now off the end of the linepointer array.  
Since commit 24992c6db the latter case is pretty common, but at least  
in principle it could have occurred before that.  The net result is  
that concurrent updates of a BRIN index could fail with errors like  
"invalid index offnum" or "inconsistent range map".  
  
Per report from Tomas Vondra.  Back-patch to 9.5, since this code is  
substantially the same in all versions containing BRIN.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 08ba67d596a1443c32d0b5300aaad8042729cd41    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 2 Nov 2017 15:51:05 +0100    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 2 Nov 2017 15:51:05 +0100    

It turns out we misdiagnosed what the real problem was.  Revert the  
previous changes, because they may have worse consequences going  
forward.  A better fix is forthcoming.  
  
The simplistic test case is kept, though disabled.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 1cf96d6c7169d0eb21770eac67b60c12dcaf113c    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 1 Nov 2017 22:07:14 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 1 Nov 2017 22:07:14 -0400    

Reported by Dan Vianello.  
  
Discussion: https://postgr.es/m/e6e12f18f70e46848c058084d42fb651@KSTLMEXGP001.CORP.CHARTERCOM.com  

commit   : 4ba0ffaaee3f1e1c1103802c786c221a56a70497    
  
author   : Peter Eisentraut <[email protected]>    
date     : Wed, 1 Nov 2017 10:20:05 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Wed, 1 Nov 2017 10:20:05 -0400    

A candidate path needs to be canonicalized before being checked against  
the mappings, because the mappings are also canonicalized.  This is  
especially relevant on Windows  
  
Reported-by: nb <[email protected]>  
Author: Michael Paquier <[email protected]>  
Reviewed-by: Ashutosh Sharma <[email protected]>  

commit   : e0ec1cbffe0065a4c2d344241824bfe383a94181    
  
author   : Michael Meskes <[email protected]>    
date     : Wed, 1 Nov 2017 13:32:18 +0100    
  
committer: Michael Meskes <[email protected]>    
date     : Wed, 1 Nov 2017 13:32:18 +0100    

Spotted and fixed by 高增琦 <[email protected]>  

commit   : f74f871b80a13ef72b19e3d829a109f8df0df792    
  
author   : Robert Haas <[email protected]>    
date     : Sun, 29 Oct 2017 12:58:40 +0530    
  
committer: Robert Haas <[email protected]>    
date     : Sun, 29 Oct 2017 12:58:40 +0530    

Without this fix, dropping a role can sometimes result in parallel  
query failures in sessions that have used "SET ROLE" to assume the  
dropped role, even if that setting isn't active any more.  
  
Report by Pavan Deolasee.  Patch by Amit Kapila, reviewed by me.  
  
Discussion: http://postgr.es/m/CABOikdOomRcZsLsLK+Z+qENM1zxyaWnAvFh3MJZzZnnKiF+REg@mail.gmail.com  

commit   : 21daada10ebf444fb1fc06a705fb22b890867083    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 18:16:25 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 18:16:25 -0400    

Commit d5b760ecb wasn't quite right, on second thought: if the  
caller didn't ask for column names then it would happily emit  
more Vars than if the caller did ask for column names.  This  
is surely not a good idea.  Advance the aliasp_item whether or  
not we're preparing a colnames list.  

commit   : 7e5e8b36d47891c1ffa9f6c592f02be850684d68    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 17:10:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 17:10:21 -0400    

expandRTE() supposed that an RTE_SUBQUERY subquery must have exactly  
as many non-junk tlist items as the RTE has column aliases for it.  
This was true at the time the code was written, and is still true so  
far as parse analysis is concerned --- but when the function is used  
during planning, the subquery might have appeared through insertion  
of a view that now has more columns than it did when the outer query  
was parsed.  This results in a core dump if, for instance, we have  
to expand a whole-row Var that references the subquery.  
  
To avoid crashing, we can either stop expanding the RTE when we run  
out of aliases, or invent new aliases for the added columns.  While  
the latter might be more useful, the former is consistent with what  
expandRTE() does for composite-returning functions in the RTE_FUNCTION  
case, so it seems like we'd better do it that way.  
  
Per bug #14876 from Samuel Horwitz.  This has been busted since commit  
ff1ea2173 allowed views to acquire more columns, so back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : cf0331a54c9b173f778ae09fbd30a432da247afc    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 12:18:57 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 12:18:57 -0400    

On closer investigation, commits f3ea3e3e8 et al were a few bricks  
shy of a load.  What we need is not so much to lock down the result  
type of a FieldSelect, as to lock down the existence of the column  
it's trying to extract.  Otherwise, we can break it by dropping that  
column.  The dependency on the result type is then held indirectly  
through the column, and doesn't need to be recorded explicitly.  
  
Out of paranoia, I left in the code to record a dependency on the  
result type, but it's used only if we can't identify the pg_class OID  
for the column.  That shouldn't ever happen right now, AFAICS, but  
it seems possible that in future the input node could be marked as  
being of type RECORD rather than some specific composite type.  
  
Likewise for FieldStore.  
  
Like the previous patch, back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 0ff56a59d2496a8f896362b91d1faa9b14f6a6d9    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 10:46:06 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 27 Oct 2017 10:46:06 -0400    

The NOTIFY page said this already, but the PREPARE TRANSACTION page  
missed it.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 036b6bd50365b9ae6ec0b4cb21d172510ef440a9    
  
author   : Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 16:04:01 +0200    
  
committer: Robert Haas <[email protected]>    
date     : Fri, 27 Oct 2017 16:04:01 +0200    

If we try to run a parallel plan in serial mode because, for example,  
it's going to be scanned via a cursor, but for some reason we're  
already in parallel mode (for example because an outer query is  
running in parallel), we'd incorrectly try to launch workers.  
Fix by adding a flag to the EState, so that we can be certain that  
ExecutePlan() and ExecGather()/ExecGatherMerge() will have the same  
idea about whether we are executing serially or in parallel.  
  
Report and fix by Amit Kapila with help from Kuntal Ghosh.  A few  
tweaks by me.  
  
Discussion: http://postgr.es/m/CAA4eK1+_BuZrmVCeua5Eqnm4Co9DAXdM5HPAOE2J19ePbR912Q@mail.gmail.com  

commit   : 37b4e0fe9964fc23d6a38973eaf67b287ac199ca    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 26 Oct 2017 12:17:40 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 26 Oct 2017 12:17:40 -0400    

Previously, we skipped using search_indexed_tlist_for_sortgroupref()  
if the tlist expression being sought in the child plan node was merely  
a Var.  This is purely an optimization, based on the theory that  
search_indexed_tlist_for_var() is faster, and one copy of a Var should  
be as good as another.  However, the GROUPING SETS patch broke the  
latter assumption: grouping columns containing the "same" Var can  
sometimes have different outputs, as shown in the test case added here.  
So do it the hard way whenever a ressortgroupref marking exists.  
  
(If this seems like a bottleneck, we could imagine building a tlist index  
data structure for ressortgroupref values, as we do for Vars.  But I'll  
let that idea go until there's some evidence it's worthwhile.)  
  
Back-patch to 9.6.  The problem also exists in 9.5 where GROUPING SETS  
came in, but this patch is insufficient to resolve the problem in 9.5:  
there is some obscure dependency on the upper-planner-pathification  
work that happened in 9.6.  Given that this is such a weird corner case,  
and no end users have complained about it, it doesn't seem worth the work  
to develop a fix for 9.5.  
  
Patch by me, per a report from Heikki Linnakangas.  (This does not fix  
Heikki's original complaint, just the follow-on one.)  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ea8480a2493e396715e2fa0709c137b513a680aa    
  
author   : Andrew Dunstan <[email protected]>    
date     : Thu, 26 Oct 2017 10:10:37 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Thu, 26 Oct 2017 10:10:37 -0400    

There have been numerous buildfarm failures but the diagnostic is  
currently silent about the reason for failure to open the file. Let's  
see if we can get to the bottom of it.  
  
Backpatch to all live branches.  

commit   : 41753604b3d06a4ca0ee4ce9f008e9b0709230c7    
  
author   : Michael Meskes <[email protected]>    
date     : Thu, 26 Oct 2017 10:16:04 +0200    
  
committer: Michael Meskes <[email protected]>    
date     : Thu, 26 Oct 2017 10:16:04 +0200    

Patch by Tsunakawa Takayuki <[email protected]>  

commit   : 7dc66a2f6e3a0d07ebea7b66df6d5364baddf7b7    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 25 Oct 2017 19:32:24 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 25 Oct 2017 19:32:24 -0400    

Some people like to run libpq-using applications in environments where  
there's no home directory.  We've broken that scenario before (cf commits  
5b4067798 and bd58d9d88), and commit ba005f193 broke it again, by making  
it a hard error if we fail to get the home directory name while looking  
for ~/.pgpass.  The previous precedent is that if we can't get the home  
directory name, we should just silently act as though the file we hoped  
to find there doesn't exist.  Rearrange the new code to honor that.  
  
Looking around, the service-file code added by commit 41a4e4595 had the  
same disease.  Apparently, that escaped notice because it only runs when  
a service name has been specified, which I guess the people who use this  
scenario don't do.  Nonetheless, it's wrong too, so fix that case as well.  
  
Add a comment about this policy to pqGetHomeDirectory, in the probably  
vain hope of forestalling the same error in future.  And upgrade the  
rather miserable commenting in parseServiceInfo, too.  
  
In passing, also back off parseServiceInfo's assumption that only ENOENT  
is an ignorable error from stat() when checking a service file.  We would  
need to ignore at least ENOTDIR as well (cf 5b4067798), and seeing that  
the far-better-tested code for ~/.pgpass treats all stat() failures alike,  
I think this code ought to as well.  
  
Per bug #14872 from Dan Watson.  Back-patch the .pgpass change to v10  
where ba005f193 came in.  The service-file bugs are far older, so  
back-patch the other changes to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 98efa5ebf0b44d9b5ac7ba0dccff3e4870b4d3c9    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:34:00 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:34:00 -0400    

json_build_object and json_build_array and the jsonb equivalents did not  
correctly process explicit VARIADIC arguments. They are modified to use  
the new extract_variadic_args() utility function which abstracts away  
the details of the call method.  
  
Michael Paquier, reviewed by Tom Lane and Dmitry Dolgov.  
  
Backpatch to 9.5 for the jsonb fixes and 9.4 for the json fixes, as  
that's where they originated.  

commit   : 5c3a1bbb4778c9a4db6982e1f54add3164c81488    
  
author   : Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:13:11 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Wed, 25 Oct 2017 07:13:11 -0400    

This is epecially useful in the case or "VARIADIC ANY" functions. The  
caller can get the artguments and types regardless of whether or not and  
explicit VARIADIC array argument has been used. The function also  
provides an option to convert arguments on type "unknown" to to "text".  
  
Michael Paquier and me, reviewed by Tom Lane.  
  
Backpatch to 9.4 in order to support the following json bug fix.  

commit   : fae550e5264b8038c955c77366683ca550a82a9b    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 18:15:36 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 18:15:36 -0400    

DST law changes in Fiji, Namibia, Northern Cyprus, Sudan, Tonga,  
and Turks & Caicos Islands.  Historical corrections for Alaska, Apia,  
Burma, Calcutta, Detroit, Ireland, Namibia, and Pago Pago.  

commit   : 173b7a4a71dd642351baffcbb9559be25626d8d0    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 17:54:09 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 17:54:09 -0400    

This is a trivial update containing only cosmetic changes.  The point  
is just to get back to being synced with an official release of tzcode,  
rather than some ad-hoc point in their commit history, which is where  
commit 47f849a3c left it.  

commit   : 285b850d518d5ade4633aea7ca419b8315422368    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 13:57:45 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 23 Oct 2017 13:57:45 -0400    

find_expr_references() neglected to record a dependency on the result type  
of a FieldSelect node, allowing a DROP TYPE to break a view or rule that  
contains such an expression.  I think we'd omitted this case intentionally,  
reasoning that there would always be a related dependency ensuring that the  
DROP would cascade to the view.  But at least with nested field selection  
expressions, that's not true, as shown in bug #14867 from Mansur Galiev.  
Add the dependency, and for good measure a dependency on the node's exposed  
collation.  
  
Likewise add a dependency on the result type of a FieldStore.  I think here  
the reasoning was that it'd only appear within an assignment to a field,  
and the dependency on the field's column would be enough ... but having  
seen this example, I think that's wrong for nested-composites cases.  
  
Looking at nearby code, I notice we're not recording a dependency on the  
exposed collation of CoerceViaIO, which seems inconsistent with our choices  
for related node types.  Maybe that's OK but I'm feeling suspicious of this  
code today, so let's add that; it certainly can't hurt.  
  
This patch does not do anything to protect already-existing views, only  
views created after it's installed.  But seeing that the issue has been  
there a very long time and nobody noticed till now, that's probably good  
enough.  
  
Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : b1752c3a7e1083014d2a46382d2fab163135be77    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 20 Oct 2017 17:12:27 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 20 Oct 2017 17:12:27 -0400    

Like the similar logic for arrays and records, it's necessary to examine  
the range's subtype to decide whether the range type can support hashing.  
We can omit checking the subtype for btree-defined operations, though,  
since range subtypes are required to have those operations.  (Possibly  
that simplification for btree cases led us to overlook that it does  
not apply for hash cases.)  
  
This is only an issue if the subtype lacks hash support, which is not  
true of any built-in range type, but it's easy to demonstrate a problem  
with a range type over, eg, money: you can get a "could not identify  
a hash function" failure when the planner is misled into thinking that  
hash join or aggregation would work.  
  
This was born broken, so back-patch to all supported branches.  

commit   : 2ac59887475a20b7168a4e0dff7ad14602db0c2c    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 17 Oct 2017 12:15:08 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 17 Oct 2017 12:15:08 -0400    

This back-patches the v10-cycle commit 1e5a5d03d into 9.3 - 9.6.  
I had noticed at the time that that was fixing a bug, namely that  
next_token() might advance *lineptr past the line-terminating '\0',  
but given the lack of field complaints I too easily convinced myself  
that the problem was only latent.  It's not, because tokenize_file()  
decides whether there's more on the line using "strlen(lineptr)".  
  
The bug is indeed latent on a newline-terminated line, because then  
the newline-stripping bit in tokenize_file() means we'll have two  
or more consecutive '\0's in the buffer, masking the fact that we  
accidentally advanced over the first one.  But the last line in  
the file might not be null-terminated, allowing the loop to see  
and process garbage, as reported by Mark Jones in bug #14859.  
  
The bug doesn't exist in <= 9.2; there next_token() is reading directly  
from a file, and termination of the outer loop relies on an feof() test  
not a buffer pointer check.  Probably commit 7f49a67f9 can be blamed  
for this bug, but I didn't track it down exactly.  
  
Commit 1e5a5d03d does a bit more than the minimum needed to fix the  
bug, but I felt the rest of it was good cleanup, so applying it all.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : aa1e9b3a466102fc43dc96906b9b3ced299ca7ed    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 12 Oct 2017 15:20:04 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 12 Oct 2017 15:20:04 -0400    

If we merge the transition calculations for two different aggregates,  
it's reasonable to assume that the transition function should not care  
which of those Aggref structs it gets from AggGetAggref().  It is not  
reasonable to make the same assumption about an aggregate final function,  
however.  Commit 804163bc2 broke this, as it will pass whichever Aggref  
was first associated with the transition state in both cases.  
  
This doesn't create an observable bug so far as the core system is  
concerned, because the only existing uses of AggGetAggref() are in  
ordered-set aggregates that happen to not pay attention to anything  
but the input properties of the Aggref; and besides that, we disabled  
sharing of transition calculations for OSAs yesterday.  Nonetheless,  
if some third-party code were using AggGetAggref() in a normal aggregate,  
they would be entitled to call this a bug.  Hence, back-patch the fix  
to 9.6 where the problem was introduced.  
  
In passing, improve some of the comments about transition state sharing.  
  
Discussion: https://postgr.es/m/CAB4ELO5RZhOamuT9Xsf72ozbenDLLXZKSk07FiSVsuJNZB861A@mail.gmail.com  

commit   : 96cfc7e19ae1b70597a90487262cfb9f4531d846    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 22:18:01 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 22:18:01 -0400    

This ought to work, but the built-in OSAs are not capable of coping,  
because their final-functions destructively modify their transition  
state (specifically, the contained tuplesort object).  That was fine  
when those functions were written, but commit 804163bc2 moved the  
goalposts without telling orderedsetaggs.c.  
  
We should fix the built-in OSAs to support this, but it will take  
a little work, especially if we don't want to sacrifice performance  
in the normal non-shared-state case.  Given that it took a year after  
9.6 release for anyone to notice this bug, we should not prioritize  
sharable-state over nonsharable-state performance.  And a proper fix  
is likely to be more complicated than we'd want to back-patch, too.  
  
Therefore, let's just put in this stop-gap patch to prevent nodeAgg.c  
from choosing to use shared state for OSAs.  We can revert it in HEAD  
when we get a better fix.  
  
Report from Lukas Eder, diagnosis by me, patch by David Rowley.  
Back-patch to 9.6 where the problem was introduced.  
  
Discussion: https://postgr.es/m/CAB4ELO5RZhOamuT9Xsf72ozbenDLLXZKSk07FiSVsuJNZB861A@mail.gmail.com  

commit   : 0da46d75e31ddfa9180345a14d720814e36922fa    
  
author   : Andres Freund <[email protected]>    
date     : Wed, 11 Oct 2017 12:03:26 -0700    
  
committer: Andres Freund <[email protected]>    
date     : Wed, 11 Oct 2017 12:03:26 -0700    

The previous coding in ProcessInterrupts() could lead to  
idle_in_transaction_session_timeout being ignored, when  
statement_timeout occurred earlier.  
  
The problem was that ProcessInterrupts() would return before  
processing the transaction timeout if QueryCancelPending was set while  
QueryCancelHoldoffCount != 0 - which is the case when reading new  
commands from the client. Ergo when the idle transaction timeout would  
hit.  
  
Fix that by removing the early return. Alternatively the transaction  
timeout code could have been moved up, but that early return seems  
like an issue that could hit other cases too.  
  
Author: Lukas Fittl  
Bug: #14821  
Discussion:  
    https://www.postgresql.org/message-id/20170921010956.17345.61461%40wrigleys.postgresql.org  
    https://www.postgresql.org/message-id/CAP53PkxQnv3OWJpyNPGJYT62uY=n1=2CF_Lpc6gVOFnc0-gazw@mail.gmail.com  
Backpatch: 9.6-, where idle_in_transaction_session_timeout was introduced.  

commit   : 19989d8480d4e802dd85fd6a16cdb8fbb0fcc1b2    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 16:56:23 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 16:56:23 -0400    

The GRANT reference page, which lists the default privileges for new  
objects, failed to mention that USAGE is granted by default for data  
types and domains.  As a lesser sin, it also did not specify anything  
about the initial privileges for sequences, FDWs, foreign servers,  
or large objects.  Fix that, and add a comment to acldefault() in the  
probably vain hope of getting people to maintain this list in future.  
  
Noted by Laurenz Albe, though I editorialized on the wording a bit.  
Back-patch to all supported branches, since they all have this behavior.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 36c687a22f49d5ec3fb3fc2827fa87ba7486149d    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 14:28:33 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 11 Oct 2017 14:28:33 -0400    

Up to now async.c has used TransactionIdIsInProgress() to detect whether  
a notify message's source transaction is still running.  However, that  
function has a quick-exit path that reports that XIDs before RecentXmin  
are no longer running.  If a listening backend is doing nothing but  
listening, and not running any queries, there is nothing that will advance  
its value of RecentXmin.  Once 2 billion transactions elapse, the  
RecentXmin check causes active transactions to be reported as not running.  
If they aren't committed yet according to CLOG, async.c decides they  
aborted and discards their messages.  The timing for that is a bit tight  
but it can happen when multiple backends are sending notifies concurrently.  
The net symptom therefore is that a sufficiently-long-surviving  
listen-only backend starts to miss some fraction of NOTIFY traffic,  
but only under heavy load.  
  
The only function that updates RecentXmin is GetSnapshotData().  
A brute-force fix would therefore be to take a snapshot before  
processing incoming notify messages.  But that would add cycles,  
as well as contention for the ProcArrayLock.  We can be smarter:  
having taken the snapshot, let's use that to check for running  
XIDs, and not call TransactionIdIsInProgress() at all.  In this  
way we reduce the number of ProcArrayLock acquisitions from one  
per message to one per notify interrupt; that's the same under  
light load but should be a benefit under heavy load.  Light testing  
says that this change is a wash performance-wise for normal loads.  
  
I looked around for other callers of TransactionIdIsInProgress()  
that might be at similar risk, and didn't find any; all of them  
are inside transactions that presumably have already taken a  
snapshot.  
  
Problem report and diagnosis by Marko Tiikkaja, patch by me.  
Back-patch to all supported branches, since it's been like this  
since 9.0.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 13a8924ecf00b57b118e307f9b01503f07dd3b28    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 8 Oct 2017 15:25:26 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 8 Oct 2017 15:25:26 -0400    

copy_file() reads and writes data 64KB at a time (with default BLCKSZ),  
and historically has issued a pg_flush_data request after each write.  
This turns out to interact really badly with macOS's new APFS file  
system: a large file copy takes over 100X longer than it ought to on  
APFS, as reported by Brent Dearth.  While that's arguably a macOS bug,  
it's not clear whether Apple will do anything about it in the near  
future, and in any case experimentation suggests that issuing flushes  
a bit less often can be helpful on other platforms too.  
  
Hence, rearrange the logic in copy_file() so that flush requests are  
issued once per N writes rather than every time through the loop.  
I set the FLUSH_DISTANCE to 32MB on macOS (any less than that still  
results in a noticeable speed degradation on APFS), but 1MB elsewhere.  
In limited testing on Linux and FreeBSD, this seems slightly faster  
than the previous code, and certainly no worse.  It helps noticeably  
on macOS even with the older HFS filesystem.  
  
A simpler change would have been to just increase the size of the  
copy buffer without changing the loop logic, but that seems likely  
to trash the processor cache without really helping much.  
  
Back-patch to 9.6 where we introduced msync() as an implementation  
option for pg_flush_data().  The problem seems specific to APFS's  
mmap/msync support, so I don't think we need to go further back.  
  
Discussion: https://postgr.es/m/CADkxhTNv-j2jw2g8H57deMeAbfRgYBoLmVuXkC=YCFBXRuCOww@mail.gmail.com  

commit   : 185279da3f3712abb89fe522bc9067eb7c8ad2c9    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 19:18:58 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 19:18:58 -0400    

The logical decoding functions do BeginInternalSubTransaction and  
RollbackAndReleaseCurrentSubTransaction to clean up after themselves.  
It turns out that AtEOSubXact_SPI has an unrecognized assumption that  
we always need to cancel the active SPI operation in the SPI context  
that surrounds the subtransaction (if there is one).  That's true  
when the RollbackAndReleaseCurrentSubTransaction call is coming from  
the SPI-using function itself, but not when it's happening inside  
some unrelated function invoked by a SPI query.  In practice the  
affected callers are the various PLs.  
  
To fix, record the current subtransaction ID when we begin a SPI  
operation, and clean up only if that ID is the subtransaction being  
canceled.  
  
Also, remove AtEOSubXact_SPI's assertion that it must have cleaned  
up the surrounding SPI context's active tuptable.  That's proven  
wrong by the same test case.  
  
Also clarify (or, if you prefer, reinterpret) the calling conventions  
for _SPI_begin_call and _SPI_end_call.  The memory context cleanup  
in the latter means that these have always had the flavor of a matched  
resource-management pair, but they weren't documented that way before.  
  
Per report from Ben Chobot.  
  
Back-patch to 9.4 where logical decoding came in.  In principle,  
the SPI changes should go all the way back, since the problem dates  
back to commit 7ec1c5a86.  But given the lack of field complaints  
it seems few people are using internal subtransactions in this way.  
So I don't feel a need to take any risks in 9.2/9.3.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 69e931f96ef47e14c62e32b91726ed0d6e7f3d73    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 12:20:13 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 6 Oct 2017 12:20:13 -0400    

Sloppy loop coding in set_status_by_pages() resulted in fetching one array  
element more than it should from the subxids[] array.  The odds of this  
resulting in SIGSEGV are pretty small, but we've certainly seen that happen  
with similar mistakes elsewhere.  While at it, we can get rid of an extra  
TransactionIdToPage() calculation per loop.  
  
Per report from David Binderman.  Back-patch to all supported branches,  
since this code is quite old.  
  
Discussion: https://postgr.es/m/HE1PR0802MB2331CBA919CBFFF0C465EB429C710@HE1PR0802MB2331.eurprd08.prod.outlook.com  

commit   : d441cff142490cda2abff60532dd03b345a97fec    
  
author   : Alvaro Herrera <[email protected]>    
date     : Fri, 6 Oct 2017 17:14:42 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Fri, 6 Oct 2017 17:14:42 +0200    

When some tuple versions in an update chain are frozen due to them being  
older than freeze_min_age, the xmax/xmin trail can become broken.  This  
breaks HOT (and probably other things).  A subsequent VACUUM can break  
things in more serious ways, such as leaving orphan heap-only tuples  
whose root HOT redirect items were removed.  This can be seen because  
index creation (or REINDEX) complain like  
  ERROR:  XX000: failed to find parent tuple for heap-only tuple at (0,7) in table "t"  
  
Because of relfrozenxid contraints, we cannot avoid the freezing of the  
early tuples, so we must cope with the results: whenever we see an Xmin  
of FrozenTransactionId, consider it a match for whatever the previous  
Xmax value was.  
  
This problem seems to have appeared in 9.3 with multixact changes,  
though strictly speaking it seems unrelated.  
  
Since 9.4 we have commit 37484ad2a "Change the way we mark tuples as  
frozen", so the fix is simple: just compare the raw Xmin (still stored  
in the tuple header, since freezing merely set an infomask bit) to the  
Xmax.  But in 9.3 we rewrite the Xmin value to FrozenTransactionId, so  
the original value is lost and we have nothing to compare the Xmax with.  
To cope with that case we need to compare the Xmin with FrozenXid,  
assume it's a match, and hope for the best.  Sadly, since you can  
pg_upgrade a 9.3 instance containing half-frozen pages to newer  
releases, we need to keep the old check in newer versions too, which  
seems a bit brittle; I hope we can somehow get rid of that.  
  
I didn't optimize the new function for performance.  The new coding is  
probably a bit slower than before, since there is a function call rather  
than a straight comparison, but I'd rather have it work correctly than  
be fast but wrong.  
  
This is a followup after 20b655224249 fixed a few related problems.  
Apparently, in 9.6 and up there are more ways to get into trouble, but  
in 9.3 - 9.5 I cannot reproduce a problem anymore with this patch, so  
there must be a separate bug.  
  
Reported-by: Peter Geoghegan  
Diagnosed-by: Peter Geoghegan, Michael Paquier, Daniel Wood,  
	Yi Wen Wong, Álvaro  
Discussion: https://postgr.es/m/CAH2-Wznm4rCrhFAiwKPWTpEw2bXDtgROZK7jWWGucXeH3D1fmA@mail.gmail.com  

commit   : 9d742e19da22aca5f872ccb07b1b8abe78de23dd    
  
author   : Robert Haas <[email protected]>    
date     : Thu, 5 Oct 2017 07:58:02 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Thu, 5 Oct 2017 07:58:02 -0400    

Michael Paquier discovered that this could be triggered via SQL;  
give a nicer message instead.  
  
Patch by Michael Paquier, reviewed by Masahiko Sawada.  
  
Discussion: http://postgr.es/m/CAB7nPqQtPg+LKKtzdKN26judHcvPZ0s1gNigzOT4j8CYuuuBYg@mail.gmail.com  

commit   : 86076395ef81c7482a5f5a62dc05a8d8075470bd    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 3 Oct 2017 14:58:25 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 3 Oct 2017 14:58:25 +0200    

1. Since commit b1a9bad9e744 we had pstrdup() inside a  
spinlock-protected critical section; reported by Andreas Seltenreich.  
Turn those into strlcpy() to stack-allocated variables instead.  
Backpatch to 9.6.  
  
2. Since commit 9ed551e0a4fd we had a pfree() uselessly inside a  
spinlock-protected critical section.  Tom Lane noticed in code review.  
Move down.  Backpatch to 9.6.  
  
3. Since commit 64233902d22b we had GetCurrentTimestamp() (a kernel  
call) inside a spinlock-protected critical section.  Tom Lane noticed in  
code review.  Move it up.  Backpatch to 9.2.  
  
4. Since commit 1bb2558046cc we did elog(PANIC) while holding spinlock.  
Tom Lane noticed in code review.  Release spinlock before dying.  
Backpatch to 9.2.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 858e9f27be2e0344cc97592cf3c23be54cb0db2c    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 1 Oct 2017 12:43:47 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 1 Oct 2017 12:43:47 -0400    

Buildfarm members skink and sungazer have both recently failed this  
test, with symptoms indicating that the default 3-second timeout  
isn't quite enough for those very slow systems.  There's no reason  
to be miserly with this timeout, so boost it to 60 seconds.  
  
Back-patch to all versions containing this test.  That may be overkill,  
because the failure has only been observed in the v10 branch, but  
I don't feel like having to revisit this later.  

commit   : 3a135bdb1b5ea2877bd3cd111668c20a3316d7d3    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 28 Sep 2017 16:44:01 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 28 Sep 2017 16:44:01 +0200    

Vacuum calls page-level HOT prune to remove dead HOT tuples before doing  
liveness checks (HeapTupleSatisfiesVacuum) on the remaining tuples.  But  
concurrent transaction commit/abort may turn DEAD some of the HOT tuples  
that survived the prune, before HeapTupleSatisfiesVacuum tests them.  
This happens to activate the code that decides to freeze the tuple ...  
which resuscitates it, duplicating data.  
  
(This is especially bad if there's any unique constraints, because those  
are now internally violated due to the duplicate entries, though you  
won't know until you try to REINDEX or dump/restore the table.)  
  
One possible fix would be to simply skip doing anything to the tuple,  
and hope that the next HOT prune would remove it.  But there is a  
problem: if the tuple is older than freeze horizon, this would leave an  
unfrozen XID behind, and if no HOT prune happens to clean it up before  
the containing pg_clog segment is truncated away, it'd later cause an  
error when the XID is looked up.  
  
Fix the problem by having the tuple freezing routines cope with the  
situation: don't freeze the tuple (and keep it dead).  In the cases that  
the XID is older than the freeze age, set the HEAP_XMAX_COMMITTED flag  
so that there is no need to look up the XID in pg_clog later on.  
  
An isolation test is included, authored by Michael Paquier, loosely  
based on Daniel Wood's original reproducer.  It only tests one  
particular scenario, though, not all the possible ways for this problem  
to surface; it be good to have a more reliable way to test this more  
fully, but it'd require more work.  
In message https://postgr.es/m/[email protected]  
I outlined another test case (more closely matching Dan Wood's) that  
exposed a few more ways for the problem to occur.  
  
Backpatch all the way back to 9.3, where this problem was introduced by  
multixact juggling.  In branches 9.3 and 9.4, this includes a backpatch  
of commit e5ff9fefcd50 (of 9.5 era), since the original is not  
correctable without matching the coding pattern in 9.5 up.  
  
Reported-by: Daniel Wood  
Diagnosed-by: Daniel Wood  
Reviewed-by: Yi Wen Wong, Michaël Paquier  
Discussion: https://postgr.es/m/[email protected]  

commit   : def03e4bfe30c230d7532f2d7cfe5d7485a658a8    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 27 Sep 2017 17:05:53 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 27 Sep 2017 17:05:53 -0400    

float8_numeric() and float4_numeric() failed to consider the possibility  
that the input is an IEEE infinity.  The results depended on the  
platform-specific behavior of sprintf(): on most platforms you'd get  
something like  
  
ERROR:  invalid input syntax for type numeric: "inf"  
  
but at least on Windows it's possible for the conversion to succeed and  
deliver a finite value (typically 1), due to a nonstandard output format  
from sprintf and lack of syntax error checking in these functions.  
  
Since our numeric type lacks the concept of infinity, a suitable conversion  
is impossible; the best thing to do is throw an explicit error before  
letting sprintf do its thing.  
  
While at it, let's use snprintf not sprintf.  Overrunning the buffer  
should be impossible if sprintf does what it's supposed to, but this  
is cheap insurance against a stack smash if it doesn't.  
  
Problem reported by Taiki Kondo.  Patch by me based on fix suggestion  
from KaiGai Kohei.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : af4bd075911d449ff1627188a4175d71d4e317d2    
  
author   : Dean Rasheed <[email protected]>    
date     : Wed, 27 Sep 2017 17:10:35 +0100    
  
committer: Dean Rasheed <[email protected]>    
date     : Wed, 27 Sep 2017 17:10:35 +0100    

Provide a correct description of how multiple policies are combined,  
clarify when SELECT permissions are required, mention SELECT FOR  
UPDATE/SHARE, and do some other more minor tidying up.  
  
Reviewed by Stephen Frost  
  
Discussion: https://postgr.es/m/CAEZATCVrxyYbOFU8XbGHicz%2BmXPYzw%3DhfNL2XTphDt-53TomQQ%40mail.gmail.com  
  
Back-patch to 9.5.  

commit   : b5ee5328bdd086fa7db1845928ef0bc87b495aaf    
  
author   : Noah Misch <[email protected]>    
date     : Tue, 26 Sep 2017 22:39:44 -0700    
  
committer: Noah Misch <[email protected]>    
date     : Tue, 26 Sep 2017 22:39:44 -0700    

It drops objects outside information_schema that depend on objects  
inside information_schema.  For example, it will drop a user-defined  
view if the view query refers to information_schema.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : f4aac742c8293b1a0535a8133f926d46474300f0    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 26 Sep 2017 15:25:56 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 26 Sep 2017 15:25:56 -0400    

Per suggestions from Peter Eisentraut and David Johnston.  
Back-patch, like the previous commit.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 12ac252f9014a96c0b7159912659fa4d0f9cbc07    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 26 Sep 2017 13:42:53 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 26 Sep 2017 13:42:53 -0400    

posix_fallocate() is not quite a drop-in replacement for fallocate(),  
because it is defined to return the error code as its function result,  
not in "errno".  I (tgl) missed this because RHEL6's version seems  
to set errno as well.  That is not the case on more modern Linuxen,  
though, as per buildfarm results.  
  
Aside from fixing the return-convention confusion, remove the test  
for ENOSYS; we expect that glibc will mask that for posix_fallocate,  
though it does not for fallocate.  Keep the test for EINTR, because  
POSIX specifies that as a possible result, and buildfarm results  
suggest that it can happen in practice.  
  
Back-patch to 9.4, like the previous commit.  
  
Thomas Munro  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 1750612224704043218fac81e29c05d866e866ce    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 25 Sep 2017 16:09:20 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 25 Sep 2017 16:09:20 -0400    

On Linux, shared memory segments created with shm_open() are backed by  
swap files created in tmpfs.  If the swap file needs to be extended,  
but there's no tmpfs space left, you get a very unfriendly SIGBUS trap.  
To avoid this, force allocation of the full request size when we create  
the segment.  This adds a few cycles, but none that we wouldn't expend  
later anyway, assuming the request isn't hugely bigger than the actual  
need.  
  
Make this code #ifdef __linux__, because (a) there's not currently a  
reason to think the same problem exists on other platforms, and (b)  
applying posix_fallocate() to an FD created by shm_open() isn't very  
portable anyway.  
  
Back-patch to 9.4 where the DSM code came in.  
  
Thomas Munro, per a bug report from Amul Sul  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 10aafbdbe4224846095198644a1169807ab9b391    
  
author   : Andrew Dunstan <[email protected]>    
date     : Mon, 25 Sep 2017 08:03:05 -0400    
  
committer: Andrew Dunstan <[email protected]>    
date     : Mon, 25 Sep 2017 08:03:05 -0400    

Haribabu Kommi, reviewed by Takeshi Ideriha and Christian Ullrich  
  
Backpatch to 9.6  

commit   : a1f30ecc5115b0186ad02ec76f75296906813d26    
  
author   : Peter Eisentraut <[email protected]>    
date     : Fri, 22 Sep 2017 16:50:59 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Fri, 22 Sep 2017 16:50:59 -0400    

In two cases, we set a different umask for some piece of code and  
restore it afterwards.  But if the contained code errors out, the umask  
is not restored.  So add TRY/CATCH blocks to fix that.  

commit   : e25f4401dad7829463efff1f4655f2c6d6b076ff    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 22 Sep 2017 00:04:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 22 Sep 2017 00:04:21 -0400    

This patch absorbs a few unreleased fixes in the IANA code.  
It corresponds to commit 2d8b944c1cec0808ac4f7a9ee1a463c28f9cd00a  
in https://github.com/eggert/tz.  Non-cosmetic changes include:  
  
TZDEFRULESTRING is updated to match current US DST practice,  
rather than what it was over ten years ago.  This only matters  
for interpretation of POSIX-style zone names (e.g., "EST5EDT"),  
and only if the timezone database doesn't include either an exact  
match for the zone name or a "posixrules" entry.  The latter  
should not be true in any current Postgres installation, but  
this could possibly matter when using --with-system-tzdata.  
  
Get rid of a nonportable use of "++var" on a bool var.  
This is part of a larger fix that eliminates some vestigial  
support for consecutive leap seconds, and adds checks to  
the "zic" compiler that the data files do not specify that.  
  
Remove a couple of ancient compatibility hacks.  The IANA  
crew think these are obsolete, and I tend to agree.  But  
perhaps our buildfarm will think different.  
  
Back-patch to all supported branches, in line with our policy  
that all branches should be using current IANA code.  Before v10,  
this includes application of current pgindent rules, to avoid  
whitespace problems in future back-patches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : ea31541f564851611fee58fe2019c866ce0e8aa0    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 21 Sep 2017 18:13:11 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 21 Sep 2017 18:13:11 -0400    

Previously, the code didn't think about this case and would just try to  
analyze such a column twice.  That would fail at the point of inserting  
the second version of the pg_statistic row, with obscure error messsages  
like "duplicate key value violates unique constraint" or "tuple already  
updated by self", depending on context and PG version.  We could allow  
the case by ignoring duplicate column specifications, but it seems better  
to reject it explicitly.  
  
The bogus error messages seem like arguably a bug, so back-patch to  
all supported versions.  
  
Nathan Bossart, per a report from Michael Paquier, and whacked  
around a bit by me.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 33dd10ef2f1df34e834c64f78622302ea5d78a93    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 20 Sep 2017 11:10:42 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 20 Sep 2017 11:10:42 -0400    

GRANT, REVOKE, and some allied commands allow the noise word GROUP  
before a role name (cf. grantee production in gram.y).  This option  
does not exist elsewhere, but it had nonetheless snuck into the  
documentation for ALTER ROLE, ALTER USER, and CREATE SCHEMA.  
  
Seems to be a copy-and-pasteo in commit 31eae6028, which did expand the  
syntax choices here, but not in that way.  Back-patch to 9.5 where that  
came in.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 95a231aef9bb872913f67f30a8cffcfefaf8ca51    
  
author   : Bruce Momjian <[email protected]>    
date     : Wed, 20 Sep 2017 09:36:19 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Wed, 20 Sep 2017 09:36:19 -0400    

This step was erroneously removed four days ago by me.  
  
Reported-by: Magnus via IM  
  
Backpatch-through: 9.5  

commit   : ea96568396830e2c5e409dbafbcece30c72b8698    
  
author   : Magnus Hagander <[email protected]>    
date     : Wed, 20 Sep 2017 14:09:05 +0200    
  
committer: Magnus Hagander <[email protected]>    
date     : Wed, 20 Sep 2017 14:09:05 +0200    

Since rsync 3.0.0 (released in 2008), the default way to enumerate  
changes was changed in a way that makes it less likely that the hardlink  
sync mode works. Since the whole point of the documented procedure is  
for the hardlinks to work, change our docs to suggest using the  
backwards compatibility switch.  

commit   : 59b5a3e5c72e4b91876df226eab09dc97c6e190e    
  
author   : Michael Meskes <[email protected]>    
date     : Mon, 11 Sep 2017 21:10:36 +0200    
  
committer: Michael Meskes <[email protected]>    
date     : Mon, 11 Sep 2017 21:10:36 +0200    

commit   : 86e4ebb9af7ed07f11410d568e0fd72427f3b0e3    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 17 Sep 2017 15:28:51 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 17 Sep 2017 15:28:51 -0400    

This lets it do the right thing for, eg, varchar columns.  
Back-patch to 9.5 where this logic appeared.  
  
David Rowley, per report from Kim Rose Carlsen  
  
Discussion: https://postgr.es/m/VI1PR05MB17091F9A9876528055D6A827C76D0@VI1PR05MB1709.eurprd05.prod.outlook.com  

commit   : c0d21bdb86b1eadd25ae56f07729aa53e1ca64c0    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 17 Sep 2017 14:50:01 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 17 Sep 2017 14:50:01 -0400    

AfterTriggerEndQuery correctly notes that the query_stack could get  
repalloc'd during a trigger firing, but it nonetheless passes the address  
of a query_stack entry to afterTriggerInvokeEvents, so that if such a  
repalloc occurs, afterTriggerInvokeEvents is already working with an  
obsolete dangling pointer while it scans the rest of the events.  Oops.  
The only code at risk is its "delete_ok" cleanup code, so we can  
prevent unsafe behavior by passing delete_ok = false instead of true.  
  
However, that could have a significant performance penalty, because the  
point of passing delete_ok = true is to not have to re-scan possibly  
a large number of dead trigger events on the next time through the loop.  
There's more than one way to skin that cat, though.  What we can do is  
delete all the "chunks" in the event list except the last one, since  
we know all events in them must be dead.  Deleting the chunks is work  
we'd have had to do later in AfterTriggerEndQuery anyway, and it ends  
up saving rescanning of just about the same events we'd have gotten  
rid of with delete_ok = true.  
  
In v10 and HEAD, we also have to be careful to mop up any per-table  
after_trig_events pointers that would become dangling.  This is slightly  
annoying, but I don't think that normal use-cases will traverse this code  
path often enough for it to be a performance problem.  
  
It's pretty hard to hit this in practice because of the unlikelihood  
of the query_stack getting resized at just the wrong time.  Nonetheless,  
it's definitely a live bug of ancient standing, so back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : c4c45945e27c9fc90b0dcb1d9af7fd6ea5220f81    
  
author   : Bruce Momjian <[email protected]>    
date     : Sat, 16 Sep 2017 11:58:00 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Sat, 16 Sep 2017 11:58:00 -0400    

Document that rsync is an _optional_ way to upgrade standbys, suggest  
rsync option --dry-run, and mention a way of upgrading one standby from  
another using rsync.  Also clarify some instructions by specifying if  
they operate on the old or new clusters.  
  
Reported-by: Stephen Frost, Magnus Hagander  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

commit   : 353328ad1afd14211e8f19c03aba19f2804f48ae    
  
author   : Robert Haas <[email protected]>    
date     : Thu, 14 Sep 2017 16:25:19 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Thu, 14 Sep 2017 16:25:19 -0400    

Otherwise, log_statement = 'ddl' causes errors if those statement  
types are used.  
  
Michael Paquier, reviewed by Ashutosh Sharma  
  
Discussion: http://postgr.es/m/CAB7nPqStC3HkE76Q1MnHsVd1vF1Td9zXApzYadzDMyLMRkkGrw@mail.gmail.com  

commit   : caae416aac83db7c96695c1927a254e82f7da5dc    
  
author   : Stephen Frost <[email protected]>    
date     : Wed, 13 Sep 2017 20:02:27 -0400    
  
committer: Stephen Frost <[email protected]>    
date     : Wed, 13 Sep 2017 20:02:27 -0400    

The order in which GRANTs are output is important as GRANTs which have  
been GRANT'd by individuals via WITH GRANT OPTION GRANTs have to come  
after the GRANT which included the WITH GRANT OPTION.  This happens  
naturally in the backend during normal operation as we only change  
existing ACLs in-place, only add new ACLs to the end, and when removing  
an ACL we remove any which depend on it also.  
  
Also, adjust the comments in acl.h to make this clear.  
  
Unfortunately, the updates to pg_dump to handle initial privileges  
involved pulling apart ACLs and then combining them back together and  
could end up putting them back together in an invalid order, leading to  
dumps which wouldn't restore.  
  
Fix this by adjusting the queries used by pg_dump to ensure that the  
ACLs are rebuilt in the same order in which they were originally.  
  
Back-patch to 9.6 where the changes for initial privileges were done.  

commit   : 407e660781b8c3866fe97fd5e326a078f549db61    
  
author   : Michael Meskes <[email protected]>    
date     : Sat, 26 Aug 2017 19:07:25 +0200    
  
committer: Michael Meskes <[email protected]>    
date     : Sat, 26 Aug 2017 19:07:25 +0200    

commit   : 839ee1811d053da4cdd0a44aea8fe7b011284be2    
  
author   : Michael Meskes <[email protected]>    
date     : Sat, 26 Aug 2017 12:57:21 +0200    
  
committer: Michael Meskes <[email protected]>    
date     : Sat, 26 Aug 2017 12:57:21 +0200    

Fix threaded test cases on Windows not to crash in setlocale() which can be  
global or local to a thread on Windows.  
  
Author: Christian Ullrich  

commit   : e5c8d43abde8cbb4b7fbbb5a215e497f11bf6fe6    
  
author   : Bruce Momjian <[email protected]>    
date     : Wed, 13 Sep 2017 09:22:18 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Wed, 13 Sep 2017 09:22:18 -0400    

Backpatch-through: 9.5  

commit   : 3b3327ef2762bf6753ccd5bee75a4935a09bd0a1    
  
author   : Bruce Momjian <[email protected]>    
date     : Wed, 13 Sep 2017 09:11:28 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Wed, 13 Sep 2017 09:11:28 -0400    

This makes it clear that pg_upgrade standby upgrade instructions should  
only be used in link mode, adds examples, and explains how rsync works  
with links.  
  
Reported-by: Andreas Joseph Krogh  
  
Discussion: https://postgr.es/m/VisenaEmail.6c.c0e592c5af4ef0a2.15e785dcb61@tc7-visena  
  
Backpatch-through: 9.5  

commit   : 8d787bd66c8f64db85e0102f1dc1a0978d060fa1    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 12 Sep 2017 13:17:52 -0400    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 12 Sep 2017 13:17:52 -0400    

This explains how rsync accomplishes updating standby servers and  
clarifies the instructions.  
  
Reported-by: Andreas Joseph Krogh  
  
Discussion: https://postgr.es/m/VisenaEmail.10.2b4049e43870bd16.15d898d696f@tc7-visena  
  
Backpatch-through: 9.5  

commit   : 64e2b29bdea0861d6b7509653a676ccc9261bfbb    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 11 Sep 2017 22:02:58 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 11 Sep 2017 22:02:58 -0400    

When copying from an active database tree, it's possible for files to be  
deleted after we see them in a readdir() scan but before we can open them.  
(Once we've got a file open, we don't expect any further errors from it  
getting unlinked, though.)  Tweak RecursiveCopy so it can cope with this  
case, so as to avoid irreproducible test failures.  
  
Back-patch to 9.6 where this code was added.  In v10 and HEAD, also  
remove unused "use RecursiveCopy" in one recovery test script.  
  
Michael Paquier and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : bd75335a83bf0564d6d256d1b78499a5ea489d74    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 4 Sep 2017 11:08:52 +0200    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 4 Sep 2017 11:08:52 +0200    

Discussion: https://postgr.es/m/[email protected]  

commit   : a4b0a4d437dc71839999c0d5bed23445abf82d88    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 3 Sep 2017 11:01:08 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 3 Sep 2017 11:01:08 -0400    

In commit 9d6b160d7, I tweaked pg_config.h.win32 to use  
"#define HAVE_LONG_LONG_INT_64 1" rather than defining it as empty,  
for consistency with what happens in an autoconf'd build.  
But Solution.pm injects another definition of that macro into  
ecpg_config.h, leading to justifiable (though harmless) compiler whining.  
Make that one consistent too.  Back-patch, like the previous patch.  
  
Discussion: https://postgr.es/m/CAEepm=1dWsXROuSbRg8PbKLh0S=8Ou-V8sr05DxmJOF5chBxqQ@mail.gmail.com  

commit   : 991a5ba73eb0d59713aec299e1245d506a72150e    
  
author   : Peter Eisentraut <[email protected]>    
date     : Fri, 1 Sep 2017 22:59:27 -0400    
  
committer: Peter Eisentraut <[email protected]>    
date     : Fri, 1 Sep 2017 22:59:27 -0400    

Author: Alexander Lakhin <[email protected]>  

commit   : 3a0f8e7d3f9a54ab34b69d1fc8c9f7806d67521a    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 1 Sep 2017 15:14:18 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 1 Sep 2017 15:14:18 -0400    

Instead of using a cast to force the constant to be the right width,  
assume we can plaster on an L, UL, LL, or ULL suffix as appropriate.  
The old approach to this is very hoary, dating from before we were  
willing to require compilers to have working int64 types.  
  
This fix makes the PG_INT64_MIN, PG_INT64_MAX, and PG_UINT64_MAX  
constants safe to use in preprocessor conditions, where a cast  
doesn't work.  Other symbolic constants that might be defined using  
[U]INT64CONST are likewise safer than before.  
  
Also fix the SIZE_MAX macro to be similarly safe, if we are forced  
to provide a definition for that.  The test added in commit 2e70d6b5e  
happens to do what we want even with the hack "(size_t) -1" definition,  
but we could easily get burnt on other tests in future.  
  
Back-patch to all supported branches, like the previous commits.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : e50d401a83968436c5d794a77b2cc8fa0a31906d    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 1 Sep 2017 13:52:53 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 1 Sep 2017 13:52:53 -0400    

Pre-C99 platforms may lack <stdint.h> and thereby SIZE_MAX.  We have  
a couple of places using the hack "(size_t) -1" as a fallback, but  
it wasn't universally available; which means the code added in commit  
2e70d6b5e fails to compile everywhere.  Move that hack to c.h so that  
we can rely on having SIZE_MAX everywhere.  
  
Per discussion, it'd be a good idea to make the macro's value safe  
for use in #if-tests, but that will take a bit more work.  This is  
just a quick expedient to get the buildfarm green again.  
  
Back-patch to all supported branches, like the previous commit.  
  
Discussion: https://postgr.es/m/[email protected]  

commit   : 9b1d48506a87a0ac61a1501a4253b6e6e524b87a    
  
author   : Robert Haas <[email protected]>    
date     : Thu, 31 Aug 2017 15:56:21 -0400    
  
committer: Robert Haas <[email protected]>    
date     : Thu, 31 Aug 2017 15:56:21 -0400    

Our documentation hasn't really caught up with the fact that  
non-exclusive backups can now be taken using pg_start_backup and  
pg_stop_backup even on standbys.  Update.  
  
David Steele, reviewed by Robert Haas and Michael Paquier  
  
Discussion: http://postgr.es/m/[email protected]  

commit   : 941e47fc43cc3d80bfd4112e013a1d11235ead6a    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 29 Aug 2017 15:38:05 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 29 Aug 2017 15:38:05 -0400    

As long as PQntuples, PQgetvalue, etc, use "int" for row numbers, we're  
pretty much stuck with this limitation.  The documentation formerly stated  
that the result of PQntuples "might overflow on 32-bit operating systems",  
which is just nonsense: that's not where the overflow would happen, and  
if you did reach an overflow it would not be on a 32-bit machine, because  
you'd have OOM'd long since.  
  
Discussion: https://postgr.es/m/CA+FnnTxyLWyjY1goewmJNxC==HQCCF4fKkoCTa9qR36oRAHDPw@mail.gmail.com  

commit   : bc95e5874ac9bc027d1549677ec4f61b9d36a2a3    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 29 Aug 2017 15:18:01 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 29 Aug 2017 15:18:01 -0400    

Adding more than 1 billion rows to a PGresult would overflow its ntups and  
tupArrSize fields, leading to client crashes.  It'd be desirable to use  
wider fields on 64-bit machines, but because all of libpq's external APIs  
use plain "int" for row counters, that's going to be hard to accomplish  
without an ABI break.  Given the lack of complaints so far, and the general  
pain that would be involved in using such huge PGresults, let's settle for  
just preventing the overflow and reporting a useful error message if it  
does happen.  Also, for a couple more lines of code we can increase the  
threshold of trouble from INT_MAX/2 to INT_MAX rows.  
  
To do that, refactor pqAddTuple() to allow returning an error message that  
replaces the default assumption that it failed because of out-of-memory.  
  
Along the way, fix PQsetvalue() so that it reports all failures via  
pqInternalNotice().  It already did so in the case of bad field number,  
but neglected to report anything for other error causes.  
  
Because of the potential for crashes, this seems like a back-patchable  
bug fix, despite the lack of field reports.  
  
Michael Paquier, per a complaint from Igor Korot.  
  
Discussion: https://postgr.es/m/CA+FnnTxyLWyjY1goewmJNxC==HQCCF4fKkoCTa9qR36oRAHDPw@mail.gmail.com  

commit   : dbc7a7d9201da6cd85dc0bba4499c2367a2cf9d7    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 29 Aug 2017 09:34:21 -0400    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 29 Aug 2017 09:34:21 -0400    

The explanation about "0" versus "9" format characters was confusing  
and arguably wrong; the discussion of sign handling wasn't very good  
either.  Notably, while it's accurate to say that "FM" strips leading  
zeroes in date/time values, what it really does with numeric values  
is to strip *trailing* zeroes, and then only if you wrote "9" rather  
than "0".  Per gripes from Erwin Brandstetter.  
  
Discussion: https://postgr.es/m/CAGHENJ7jgRbTn6nf48xNZ=FHgL2WQ4X8mYsUAU57f-vq8PubEw@mail.gmail.com  
Discussion: https://postgr.es/m/CAGHENJ45ymd=GOCu1vwV9u7GmCR80_5tW0fP9C_gJKbruGMHvQ@mail.gmail.com