PostgreSQL 9.4.16 commit log

Stamp 9.4.16.

commit   : b7e1ca7d8e65319cc325bdbc757e723ce5b02937    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 5 Feb 2018 16:07:03 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 5 Feb 2018 16:07:03 -0500    

Click here for diff

M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc

Last-minute updates for release notes.

commit   : 9a12ac37be9683da09c3dc33d0ee43728ca49517    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 5 Feb 2018 14:43:40 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 5 Feb 2018 14:43:40 -0500    

Click here for diff

Security: CVE-2018-1052, CVE-2018-1053  

M doc/src/sgml/release-9.3.sgml
M doc/src/sgml/release-9.4.sgml

Translation updates

commit   : 0a5dcba2abc68996a22387f1186889447a4126ce    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Mon, 5 Feb 2018 12:45:45 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Mon, 5 Feb 2018 12:45:45 -0500    

Click here for diff

Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: 1bde68354ce9389733ebee17e76817f3bda1edf1  

M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/it.po
M src/backend/po/ru.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/it.po
M src/bin/pg_basebackup/po/it.po
M src/bin/pg_config/po/it.po
M src/bin/pg_controldata/po/it.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/it.po
M src/bin/pg_dump/po/it.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_resetxlog/po/fr.po
M src/bin/pg_resetxlog/po/it.po
M src/bin/psql/po/de.po
M src/bin/psql/po/it.po
M src/bin/psql/po/ru.po
M src/bin/scripts/po/it.po
M src/interfaces/ecpg/ecpglib/po/it.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/it.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/libpq/po/it.po
M src/interfaces/libpq/po/ru.po
M src/pl/plperl/po/it.po
M src/pl/plpgsql/src/po/it.po
M src/pl/plpython/po/it.po
M src/pl/tcl/po/it.po

Ensure that all temp files made during pg_upgrade are non-world-readable.

commit   : c3456208d1d4b281417c1e43fd24d211bf35ec95    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 5 Feb 2018 10:58:27 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 5 Feb 2018 10:58:27 -0500    

Click here for diff

pg_upgrade has always attempted to ensure that the transient dump files  
it creates are inaccessible except to the owner.  However, refactoring  
in commit 76a7650c4 broke that for the file containing "pg_dumpall -g"  
output; since then, that file was protected according to the process's  
default umask.  Since that file may contain role passwords (hopefully  
encrypted, but passwords nonetheless), this is a particularly unfortunate  
oversight.  Prudent users of pg_upgrade on multiuser systems would  
probably run it under a umask tight enough that the issue is moot, but  
perhaps some users are depending only on pg_upgrade's umask changes to  
protect their data.  
  
To fix this in a future-proof way, let's just tighten the umask at  
process start.  There are no files pg_upgrade needs to write at a  
weaker security level; and if there were, transiently relaxing the  
umask around where they're created would be a safer approach.  
  
Report and patch by Tom Lane; the idea for the fix is due to Noah Misch.  
Back-patch to all supported branches.  
  
Security: CVE-2018-1053  

M contrib/pg_upgrade/dump.c
M contrib/pg_upgrade/file.c
M contrib/pg_upgrade/pg_upgrade.c
M contrib/pg_upgrade/pg_upgrade.h

Release notes for 10.2, 9.6.7, 9.5.11, 9.4.16, 9.3.21.

commit   : f20a9972c7501f55390ff3dff33caed7340cd217    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 4 Feb 2018 15:13:45 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 4 Feb 2018 15:13:45 -0500    

Click here for diff

M doc/src/sgml/release-9.3.sgml
M doc/src/sgml/release-9.4.sgml

commit   : 27a5eaa71b72866be0a63b4e68868bc87bbee5a5    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Wed, 31 Jan 2018 16:54:33 -0500    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Wed, 31 Jan 2018 16:54:33 -0500    

Click here for diff

Also remove outdated comment about SPI subtransactions.  
  
Reported-by: gregory@arenius.com  
  
Discussion: https://postgr.es/m/151726276676.1240.10501743959198501067@wrigleys.postgresql.org  
  
Backpatch-through: 9.3  

M doc/src/sgml/contrib-spi.sgml
M doc/src/sgml/spi.sgml

pgcrypto’s encrypt() supports AES-128, AES-192, and AES-256

commit   : 7c348e16c279f4e3f606171e5e9be8fee2489e60    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Wed, 31 Jan 2018 16:28:11 -0500    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Wed, 31 Jan 2018 16:28:11 -0500    

Click here for diff

Previously, only 128 was mentioned, but the others are also supported.  
  
Thomas Munro, reviewed by Michael Paquier and extended a bit by me.  
  
Discussion: http://postgr.es/m/CAEepm=1XbBHXYJKofGjnM2Qfz-ZBVqhGU4AqvtgR+Hegy4fdKg@mail.gmail.com  

M contrib/pgcrypto/expected/rijndael.out
M contrib/pgcrypto/sql/rijndael.sql
M doc/src/sgml/pgcrypto.sgml

Exclude common/int128.h from cpluspluscheck

commit   : 9f050c0b41685e61c7373d7af9f97740b61a316d    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Tue, 30 Jan 2018 19:21:19 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Tue, 30 Jan 2018 19:21:19 -0500    

Click here for diff

It uses static assertions, which are not supported under C++ in this  
branch.  
  
This change only goes into the 9.4 branch, because 9.5 and beyond will  
primarily use the USE_NATIVE_INT128 branch, so cpluspluscheck isn't  
bothered.  In PG11 we will have C++ support for static assertions, so  
the issue will go away altogether.  

M src/tools/pginclude/cpluspluscheck

psql documentation fixes

commit   : b37422de89e569a474509cd26ec49e27866d01e8    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Mon, 29 Jan 2018 14:04:32 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Mon, 29 Jan 2018 14:04:32 -0500    

Click here for diff

Update the documentation for \pset to mention  
columns|linestyle.  
  
Author: Дилян Палаузов <dpa-postgres@aegee.org>  

M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/help.c

Add stack-overflow guards in set-operation planning.

commit   : 06efc5cf53b96ef5b9c10995fc7c9ec06c1b4846    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 28 Jan 2018 13:39:07 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sun, 28 Jan 2018 13:39:07 -0500    

Click here for diff

create_plan_recurse lacked any stack depth check.  This is not per  
our normal coding rules, but I'd supposed it was safe because earlier  
planner processing is more complex and presumably should eat more  
stack.  But bug #15033 from Andrew Grossman shows this isn't true,  
at least not for queries having the form of a many-thousand-way  
INTERSECT stack.  
  
Further testing showed that recurse_set_operations is also capable  
of being crashed in this way, since it likewise will recurse to the  
bottom of a parsetree before calling any support functions that  
might themselves contain any stack checks.  However, its stack  
consumption is only perhaps a third of create_plan_recurse's.  
  
It's possible that this particular problem with create_plan_recurse can  
only manifest in 9.6 and later, since before that we didn't build a Path  
tree for set operations.  But having seen this example, I now have no  
faith in the proposition that create_plan_recurse doesn't need a stack  
check, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/20180127050845.28812.58244@wrigleys.postgresql.org  

M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/prep/prepunion.c

Update time zone data files to tzdata release 2018c.

commit   : fa86a32f9b3a8f44c118510cf21db6efea3a1d51    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 27 Jan 2018 16:42:28 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 27 Jan 2018 16:42:28 -0500    

Click here for diff

DST law changes in Brazil, Sao Tome and Principe.  Historical corrections  
for Bolivia, Japan, and South Sudan.  The "US/Pacific-New" zone has been  
removed (it was only a link to America/Los_Angeles anyway).  

M src/timezone/data/tzdata.zi
M src/timezone/known_abbrevs.txt
M src/timezone/tznames/Africa.txt

Teach reparameterize_path() to handle AppendPaths.

commit   : 54e1599c76dfa90d179b61615f6473c437cf94f1    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 23 Jan 2018 16:50:35 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 23 Jan 2018 16:50:35 -0500    

Click here for diff

If we're inside a lateral subquery, there may be no unparameterized paths  
for a particular child relation of an appendrel, in which case we *must*  
be able to create similarly-parameterized paths for each other child  
relation, else the planner will fail with "could not devise a query plan  
for the given query".  This means that there are situations where we'd  
better be able to reparameterize at least one path for each child.  
  
This calls into question the assumption in reparameterize_path() that  
it can just punt if it feels like it.  However, the only case that is  
known broken right now is where the child is itself an appendrel so that  
all its paths are AppendPaths.  (I think possibly I disregarded that in  
the original coding on the theory that nested appendrels would get folded  
together --- but that only happens *after* reparameterize_path(), so it's  
not excused from handling a child AppendPath.)  Given that this code's been  
like this since 9.3 when LATERAL was introduced, it seems likely we'd have  
heard of other cases by now if there were a larger problem.  
  
Per report from Elvis Pranskevichus.  Back-patch to 9.3.  
  
Discussion: https://postgr.es/m/5981018.zdth1YWmNy@hammer.magicstack.net  

M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql

doc: simplify intermediate certificate mention in libpq docs

commit   : a0950b1b96477fd64e210a0816e68d1c0db2053a    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Tue, 23 Jan 2018 10:18:21 -0500    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Tue, 23 Jan 2018 10:18:21 -0500    

Click here for diff

Backpatch-through: 9.3  

M doc/src/sgml/libpq.sgml

Make pg_dump’s ACL, sec label, and comment entries reliably identifiable.

commit   : da83ca7d9d9873e66c91e9872644092bf93d5f59    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 22 Jan 2018 12:06:19 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 22 Jan 2018 12:06:19 -0500    

Click here for diff

_tocEntryRequired() expects that it can identify ACL, SECURITY LABEL,  
and COMMENT TOC entries that are for large objects by seeing whether  
the tag for them starts with "LARGE OBJECT ".  While that works fine  
for actual large objects, which are indeed tagged that way, it's  
subject to false positives unless every such entry's tag starts with an  
appropriate type ID.  And in fact it does not work for ACLs, because  
up to now we customarily tagged those entries with just the bare name  
of the object.  This means that an ACL for an object named  
"LARGE OBJECT something" would be misclassified as data not schema,  
with undesirable results in a schema-only or data-only dump ---  
although pg_upgrade seems unaffected, due to the special case for  
binary-upgrade mode further down in _tocEntryRequired().  
  
We can fix this by changing all the dumpACL calls to use the label  
strings already in use for comments and security labels, which do  
follow the convention of starting with an object type indicator.  
  
Well, mostly they follow it.  dumpDatabase() got it wrong, using  
just the bare database name for those purposes, so that a database  
named "LARGE OBJECT something" would similarly be subject to having  
its comment or security label dropped or included when not wanted.  
Bring that into line too.  (Note that up to now, database ACLs have  
not been processed by pg_dump, so that this issue doesn't affect them.)  
  
_tocEntryRequired() itself is not free of fault: it was overly liberal  
about matching object tags to "LARGE OBJECT " in binary-upgrade mode.  
This looks like it is probably harmless because there would be no data  
component to strip anyway in that mode, but at best it's trouble  
waiting to happen, so tighten that up too.  
  
The possible misclassification of SECURITY LABEL entries for databases is  
in principle a security problem, but the opportunities for actual exploits  
seem too narrow to be interesting.  The other cases seem like just bugs,  
since an object owner can change its ACL or comment for himself, he needn't  
try to trick someone else into doing it by choosing a strange name.  
  
This has been broken since per-large-object TOC entries were introduced  
in 9.0, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/21714.1516553459@sss.pgh.pa.us  

M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_dump.c

doc: update intermediate certificate instructions

commit   : f840557181af76ffb28ddd86be869781d0d14542    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Sat, 20 Jan 2018 21:47:02 -0500    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Sat, 20 Jan 2018 21:47:02 -0500    

Click here for diff

Document how to properly create root and intermediate certificates using  
v3_ca extensions and where to place intermediate certificates so they  
are properly transferred to the remote side with the leaf certificate to  
link to the remote root certificate.  This corrects docs that used to  
say that intermediate certificates must be stored with the root  
certificate.  
  
Also add instructions on how to create root, intermediate, and leaf  
certificates.  
  
Discussion: https://postgr.es/m/20180116002238.GC12724@momjian.us  
  
Reviewed-by: Michael Paquier  
  
Backpatch-through: 9.3  

M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml

Fix StoreCatalogInheritance1 to use 32bit inhseqno

commit   : 1284d18b5de980952db1494021d62e093e7ba577    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 19 Jan 2018 10:15:08 -0300    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 19 Jan 2018 10:15:08 -0300    

Click here for diff

For no apparent reason, this function was using a 16bit-wide inhseqno  
value, rather than the correct 32 bit width which is what is stored in  
the pg_inherits catalog.  This becomes evident if you try to create a  
table with more than 65535 parents, because this error appears:  
  
ERROR:  duplicate key value violates unique constraint «pg_inherits_relid_seqno_index»  
DETAIL:  Key (inhrelid, inhseqno)=(329371, 0) already exists.  
  
Needless to say, having so many parents is an uncommon situations, which  
explains why this error has never been reported despite being having  
been introduced with the Postgres95 1.01 sources in commit d31084e9d111:  
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/commands/creatinh.c;hb=d31084e9d111#l349  
  
Backpatch all the way back.  
  
David Rowley noticed this while reviewing a patch of mine.  
Discussion: https://postgr.es/m/CAKJS1f8Dn7swSEhOWwzZzssW7747YB=2Hi+T7uGud40dur69-g@mail.gmail.com  

M src/backend/commands/tablecmds.c

Cope with indicator arrays that do not have the correct length.

commit   : 2c1c4b060c4381a83dd0500a73b26eb7dea4c693    
  
author   : Michael Meskes <meskes@postgresql.org>    
date     : Sat, 13 Jan 2018 14:56:49 +0100    
  
committer: Michael Meskes <meskes@postgresql.org>    
date     : Sat, 13 Jan 2018 14:56:49 +0100    

Click here for diff

Patch by: "Rader, David" <davidr@openscg.com>  

M src/interfaces/ecpg/preproc/type.c

Avoid unnecessary failure in SELECT concurrent with ALTER NO INHERIT.

commit   : 8b0e5e7e7cf0f205d93677b39b4fcbe60dce3b34    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 12 Jan 2018 15:46:38 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 12 Jan 2018 15:46:38 -0500    

Click here for diff

If a query against an inheritance tree runs concurrently with an ALTER  
TABLE that's disinheriting one of the tree members, it's possible to get  
a "could not find inherited attribute" error because after obtaining lock  
on the removed member, make_inh_translation_list sees that its columns  
have attinhcount=0 and decides they aren't the columns it's looking for.  
  
An ideal fix, perhaps, would avoid including such a just-removed member  
table in the query at all; but there seems no way to accomplish that  
without adding expensive catalog rechecks or creating a likelihood of  
deadlocks.  Instead, let's just drop the check on attinhcount.  In this  
way, a query that's included a just-disinherited child will still  
succeed, which is not a completely unreasonable behavior.  
  
This problem has existed for a long time, so back-patch to all supported  
branches.  Also add an isolation test verifying related behaviors.  
  
Patch by me; the new isolation test is based on Kyotaro Horiguchi's work.  
  
Discussion: https://postgr.es/m/20170626.174612.23936762.horiguchi.kyotaro@lab.ntt.co.jp  

M src/backend/optimizer/prep/prepunion.c
A src/test/isolation/expected/alter-table-4.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/alter-table-4.spec

Fix sample INSTR() functions in the plpgsql documentation.

commit   : 493cdc8ee503eeb2a48736a3c5c2a59d52d26838    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 10 Jan 2018 17:13:29 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 10 Jan 2018 17:13:29 -0500    

Click here for diff

These functions are stated to be Oracle-compatible, but they weren't.  
Yugo Nagata noticed that while our code returns zero for a zero or  
negative fourth parameter (occur_index), Oracle throws an error.  
Further testing by me showed that there was also a discrepancy in the  
interpretation of a negative third parameter (beg_index): Oracle thinks  
that a negative beg_index indicates the last place where the target  
substring can *begin*, whereas our code thinks it is the last place  
where the target can *end*.  
  
Adjust the sample code to behave like Oracle in both these respects.  
Also change it to be a CDATA[] section, simplifying copying-and-pasting  
out of the documentation source file.  And fix minor problems in the  
introductory comment, which wasn't very complete or accurate.  
  
Back-patch to all supported branches.  Although this patch only touches  
documentation, we should probably call it out as a bug fix in the next  
minor release notes, since users who have adopted the functions will  
likely want to update their versions.  
  
Yugo Nagata and Tom Lane  
  
Discussion: https://postgr.es/m/20171229191705.c0b43a8c.nagata@sraoss.co.jp  

M doc/src/sgml/plpgsql.sgml

Change some bogus PageGetLSN calls to BufferGetLSNAtomic

commit   : c6187964047f5892fb91e3af170996a93af33ed9    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 9 Jan 2018 15:54:38 -0300    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 9 Jan 2018 15:54:38 -0300    

Click here for diff

As src/backend/access/transam/README says, PageGetLSN may only be called  
by processes holding either exclusive lock on buffer, or a shared lock  
on buffer plus buffer header lock.  Therefore any place that only holds  
a shared buffer lock must use BufferGetLSNAtomic instead of PageGetLSN,  
which internally obtains buffer header lock prior to reading the LSN.  
  
A few callsites failed to comply with this rule.  This was detected by  
running all tests under a new (not committed) assertion that verifies  
PageGetLSN locking contract.  All but one of the callsites that failed  
the assertion are fixed by this patch.  Remaining callsites were  
inspected manually and determined not to need any change.  
  
The exception (unfixed callsite) is in TestForOldSnapshot, which only  
has a Page argument, making it impossible to access the corresponding  
Buffer from it.  Fixing that seems a much larger patch that will have to  
be done separately; and that's just as well, since it was only  
introduced in 9.6 and other bugs are much older.  
  
Some of these bugs are ancient; backpatch all the way back to 9.3.  
  
Authors: Jacob Champion, Asim Praveen, Ashwin Agrawal  
Reviewed-by: Michaël Paquier  
Discussion: https://postgr.es/m/CABAq_6GXgQDVu3u12mK9O5Xt5abBZWQ0V40LZCE+oUf95XyNFg@mail.gmail.com  

M src/backend/access/gist/gist.c
M src/backend/access/gist/gistvacuum.c

pg_upgrade: simplify code layout in a few places

commit   : 9162560bc3ac36b515e222f35db1efbb9ccfab83    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Fri, 5 Jan 2018 14:11:14 -0500    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Fri, 5 Jan 2018 14:11:14 -0500    

Click here for diff

Backpatch-through: 9.4 (9.3 didn't need improving)  

M contrib/pg_upgrade/exec.c

Fix failure to delete spill files of aborted transactions

commit   : f68c49f86aea271a083f26147a8648ac74ed2e76    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 5 Jan 2018 12:17:10 -0300    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Fri, 5 Jan 2018 12:17:10 -0300    

Click here for diff

Logical decoding's reorderbuffer.c may spill transaction files to disk  
when transactions are large.  These are supposed to be removed when they  
become "too old" by xid; but file removal requires the boundary LSNs of  
the transaction to be known.  The final_lsn is only set when we see the  
commit or abort record for the transaction, but nothing sets the value  
for transactions that crash, so the removal code misbehaves -- in  
assertion-enabled builds, it crashes by a failed assertion.  
  
To fix, modify the final_lsn of transactions that don't have a value  
set, to the LSN of the very latest change in the transaction.  This  
causes the spilled files to be removed appropriately.  
  
Author: Atsushi Torikoshi  
Reviewed-by: Kyotaro HORIGUCHI, Craig Ringer, Masahiko Sawada  
Discussion: https://postgr.es/m/54e4e488-186b-a056-6628-50628e4e4ebc@lab.ntt.co.jp  

M src/backend/replication/logical/reorderbuffer.c
M src/include/replication/reorderbuffer.h

Fix use of config-specific libraries for Windows OpenSSL

commit   : 2d03daa7b8b1b8061afa80bdec85da5d761e1023    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Wed, 3 Jan 2018 15:26:39 -0500    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Wed, 3 Jan 2018 15:26:39 -0500    

Click here for diff

Commit 614350a3 allowed for an different builds of OpenSSL libraries on  
Windows, but ignored the fact that the alternative builds don't have  
config-specific libraries. This patch fixes the Solution file to ask for  
the correct libraries.  
  
per offline discussions with Leonardo Cecchi and Marco Nenciarini,  
  
Backpatch to all live branches.  

M src/tools/msvc/Solution.pm

Make XactLockTableWait work for transactions that are not yet self-locked

commit   : fe6bdc0a38c7bdb8d94f720f7a33cfee87458f6d    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 3 Jan 2018 12:36:47 -0300    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Wed, 3 Jan 2018 12:36:47 -0300    

Click here for diff

XactLockTableWait assumed that its xid argument has already added itself  
to the lock table.  That assumption led to another assumption that if  
locking the xid has succeeded but the xid is reported as still in  
progress, then the input xid must have been a subtransaction.  
  
These assumptions hold true for the original uses of this code in  
locking related to on-disk tuples, but they break down in logical  
replication slot snapshot building -- in particular, when a standby  
snapshot logged contains an xid that's already in ProcArray but not yet  
in the lock table.  This leads to assertion failures that can be  
reproduced all the way back to 9.4, when logical decoding was  
introduced.  
  
To fix, change SubTransGetParent to SubTransGetTopmostTransaction which  
has a slightly different API: it returns the argument Xid if there is no  
parent, and it goes all the way to the top instead of moving up the  
levels one by one.  Also, to avoid busy-waiting, add a 1ms sleep to give  
the other process time to register itself in the lock table.  
  
For consistency, change ConditionalXactLockTableWait the same way.  
  
Author: Petr Jelínek  
Discussion: https://postgr.es/m/1B3E32D8-FCF4-40B4-AEF9-5C0E3AC57969@postgrespro.ru  
Reported-by: Konstantin Knizhnik  
Diagnosed-by: Stas Kelvich, Petr Jelínek  
Reviewed-by: Andres Freund, Robert Haas  

M src/backend/storage/lmgr/lmgr.c

commit   : aa7d71be4b935cd959b19b1e112792981edcdc15    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Tue, 2 Jan 2018 23:30:12 -0500    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Tue, 2 Jan 2018 23:30:12 -0500    

Click here for diff

Backpatch-through: certain files through 9.3  

M COPYRIGHT
M doc/src/sgml/legal.sgml

Fix deadlock hazard in CREATE INDEX CONCURRENTLY

commit   : 47a3a13178731feac08a262d63117fd9beabd743    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 2 Jan 2018 19:16:16 -0300    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 2 Jan 2018 19:16:16 -0300    

Click here for diff

Multiple sessions doing CREATE INDEX CONCURRENTLY simultaneously are  
supposed to be able to work in parallel, as evidenced by fixes in commit  
c3d09b3bd23f specifically to support this case.  In reality, one of the  
sessions would be aborted by a misterious "deadlock detected" error.  
  
Jeff Janes diagnosed that this is because of leftover snapshots used for  
system catalog scans -- this was broken by 8aa3e47510b9 keeping track of  
(registering) the catalog snapshot.  To fix the deadlocks, it's enough  
to de-register that snapshot prior to waiting.  
  
Backpatch to 9.4, which introduced MVCC catalog scans.  
  
Include an isolationtester spec that 8 out of 10 times reproduces the  
deadlock with the unpatched code for me (Álvaro).  
  
Author: Jeff Janes  
Diagnosed-by: Jeff Janes  
Reported-by: Jeremy Finzel  
Discussion: https://postgr.es/m/CAMa1XUhHjCv8Qkx0WOr1Mpm_R4qxN26EibwCrj0Oor2YBUFUTg%40mail.gmail.com  

M src/backend/commands/indexcmds.c
A src/test/isolation/expected/multiple-cic.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/multiple-cic.spec

Disallow UNION/INTERSECT/EXCEPT over no columns.

commit   : e1b8e0e4a61ed100510d70e63930273a6614cec4    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 22 Dec 2017 12:08:41 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 22 Dec 2017 12:08:41 -0500    

Click here for diff

Since 9.4, we've allowed the syntax "select union select" and variants  
of that.  However, the planner wasn't expecting a no-column set operation  
and ended up treating the set operation as if it were UNION ALL.  
  
Pre-v10, there seem to be some executor issues that would need to be  
fixed to support such cases, and it doesn't really seem worth expending  
much effort on.  Just disallow it, instead.  
  
Per report from Victor Yegorov.  
  
Discussion: https://postgr.es/m/CAGnEbojGJrRSOgJwNGM7JSJZpVAf8xXcVPbVrGdhbVEHZ-BUMw@mail.gmail.com  

M src/backend/optimizer/prep/prepunion.c

doc: Fix figures in example description

commit   : 02f66d8656e8fb1ba56604994149f063c7230a18    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Mon, 18 Dec 2017 16:00:35 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Mon, 18 Dec 2017 16:00:35 -0500    

Click here for diff

oversight in 244c8b466a743d1ec18a7d841bf42669699b3b56  
  
Reported-by: Blaz Merela <blaz@merela.org>  

M doc/src/sgml/perform.sgml

Perform a lot more sanity checks when freezing tuples.

commit   : ed8e1aff6acef475eb33e0212253df05cb2d4d20    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Mon, 13 Nov 2017 18:45:47 -0800    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Mon, 13 Nov 2017 18:45:47 -0800    

Click here for diff

The previous commit has shown that the sanity checks around freezing  
aren't strong enough. Strengthening them seems especially important  
because the existance of the bug has caused corruption that we don't  
want to make even worse during future vacuum cycles.  
  
The errors are emitted with ereport rather than elog, despite being  
"should never happen" messages, so a proper error code is emitted. To  
avoid superflous translations, mark messages as internal.  
  
Author: Andres Freund and Alvaro Herrera  
Reviewed-By: Alvaro Herrera, Michael Paquier  
Discussion: https://postgr.es/m/20171102112019.33wb7g5wp4zpjelu@alap3.anarazel.de  
Backpatch: 9.3-  

M src/backend/access/heap/heapam.c
M src/backend/access/heap/rewriteheap.c
M src/backend/commands/vacuumlazy.c
M src/include/access/heapam.h
M src/include/access/heapam_xlog.h

Fix pruning of locked and updated tuples.

commit   : 4eff5a8c9f848adb02af6ec2d2b4664066409de9    
  
author   : Andres Freund <andres@anarazel.de>    
date     : Fri, 3 Nov 2017 07:52:29 -0700    
  
committer: Andres Freund <andres@anarazel.de>    
date     : Fri, 3 Nov 2017 07:52:29 -0700    

Click here for diff

Previously it was possible that a tuple was not pruned during vacuum,  
even though its update xmax (i.e. the updating xid in a multixact with  
both key share lockers and an updater) was below the cutoff horizon.  
  
As the freezing code assumed, rightly so, that that's not supposed to  
happen, xmax would be preserved (as a member of a new multixact or  
xmax directly). That causes two problems: For one the tuple is below  
the xmin horizon, which can cause problems if the clog is truncated or  
once there's an xid wraparound. The bigger problem is that that will  
break HOT chains, which in turn can lead two to breakages: First,  
failing index lookups, which in turn can e.g lead to constraints being  
violated. Second, future hot prunes / vacuums can end up making  
invisible tuples visible again. There's other harmful scenarios.  
  
Fix the problem by recognizing that tuples can be DEAD instead of  
RECENTLY_DEAD, even if the multixactid has alive members, if the  
update_xid is below the xmin horizon. That's safe because newer  
versions of the tuple will contain the locking xids.  
  
A followup commit will harden the code somewhat against future similar  
bugs and already corrupted data.  
  
Author: Andres Freund, with changes by Alvaro Herrera  
Reported-By: Daniel Wood  
Analyzed-By: Andres Freund, Alvaro Herrera, Robert Haas, Peter  
   Geoghegan, Daniel Wood, Yi Wen Wong, Michael Paquier  
Reviewed-By: Alvaro Herrera, Robert Haas, Michael Paquier  
Discussion:  
    https://postgr.es/m/E5711E62-8FDF-4DCA-A888-C200BF6B5742@amazon.com  
    https://postgr.es/m/20171102112019.33wb7g5wp4zpjelu@alap3.anarazel.de  
Backpatch: 9.3-  

M src/backend/utils/time/tqual.c
M src/test/isolation/expected/freeze-the-dead.out
M src/test/isolation/isolation_schedule
M src/test/isolation/specs/freeze-the-dead.spec

Fix walsender timeouts when decoding a large transaction

commit   : f5c7e0cddf6a970012bf4cfd18f00174f4d585b3    
  
author   : Andrew Dunstan <andrew@dunslane.net>    
date     : Thu, 14 Dec 2017 11:32:25 -0500    
  
committer: Andrew Dunstan <andrew@dunslane.net>    
date     : Thu, 14 Dec 2017 11:32:25 -0500    

Click here for diff

The logical slots have a fast code path for sending data so as not to  
impose too high a per message overhead. The fast path skips checks for  
interrupts and timeouts. However, the existing coding failed to consider  
the fact that a transaction with a large number of changes may take a  
very long time to be processed and sent to the client. This causes the  
walsender to ignore interrupts for potentially a long time and more  
importantly it will result in the walsender being killed due to  
timeout at the end of such a transaction.  
  
This commit changes the fast path to also check for interrupts and only  
allows calling the fast path when the last keepalive check happened less  
than half the walsender timeout ago. Otherwise the slower code path will  
be taken.  
  
Backpatched to 9.4  
  
Petr Jelinek, reviewed by  Kyotaro HORIGUCHI, Yura Sokolov,  Craig  
Ringer and Robert Haas.  
  
Discussion: https://postgr.es/m/e082a56a-fd95-a250-3bae-0fff93832510@2ndquadrant.com  

M src/backend/replication/walsender.c

Fix corner-case coredump in _SPI_error_callback().

commit   : 239b01e313b5c3bc90d383b0a345e0549fe9ba96    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 11 Dec 2017 16:33:20 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 11 Dec 2017 16:33:20 -0500    

Click here for diff

I noticed that _SPI_execute_plan initially sets spierrcontext.arg = NULL,  
and only fills it in some time later.  If an error were to happen in  
between, _SPI_error_callback would try to dereference the null pointer.  
This is unlikely --- there's not much between those points except  
push-snapshot calls --- but it's clearly not impossible.  Tweak the  
callback to do nothing if the pointer isn't set yet.  
  
It's been like this for awhile, so back-patch to all supported branches.  

M src/backend/executor/spi.c

MSVC 2012+: Permit linking to 32-bit, MinGW-built libraries.

commit   : d78c3ca0ea1012d9c944bae981b13151e4f86dcb    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 9 Dec 2017 00:58:55 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 9 Dec 2017 00:58:55 -0800    

Click here for diff

Notably, this permits linking to the 32-bit Perl binaries advertised on  
perl.org, namely Strawberry Perl and ActivePerl.  This has a side effect  
of permitting linking to binaries built with obsolete MSVC versions.  
  
By default, MSVC 2012 and later require a "safe exception handler table"  
in each binary.  MinGW-built, 32-bit DLLs lack the relevant exception  
handler metadata, so linking to them failed with error LNK2026.  Restore  
the semantics of MSVC 2010, which omits the table from a given binary if  
some linker input lacks metadata.  This has no effect on 64-bit builds  
or on MSVC 2010 and earlier.  Back-patch to 9.3 (all supported  
versions).  
  
Reported by Victor Wagner.  
  
Discussion: https://postgr.es/m/20160326154321.7754ab8f@wagner.wagner.home  

M src/tools/msvc/MSBuildProject.pm

MSVC: Test whether 32-bit Perl needs -D_USE_32BIT_TIME_T.

commit   : aed8d41af628c2fce661f1156ce4367db41da502    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Fri, 8 Dec 2017 18:06:05 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Fri, 8 Dec 2017 18:06:05 -0800    

Click here for diff

Commits 5a5c2feca3fd858e70ea348822595547e6fa6c15 and  
b5178c5d08ca59e30f9d9428fa6fdb2741794e65 introduced support for modern  
MSVC-built, 32-bit Perl, but they broke use of MinGW-built, 32-bit Perl  
distributions like Strawberry Perl and modern ActivePerl.  Perl has no  
robust means to report whether it expects a -D_USE_32BIT_TIME_T ABI, so  
test this.  Back-patch to 9.3 (all supported versions).  
  
The chief alternative was a heuristic of adding -D_USE_32BIT_TIME_T when  
$Config{gccversion} is nonempty.  That banks on every gcc-built Perl  
using the same ABI.  gcc could change its default ABI the way MSVC once  
did, and one could build Perl with gcc and the non-default ABI.  
  
The GNU make build system could benefit from a similar test, without  
which it does not support MSVC-built Perl.  For now, just add a comment.  
Most users taking the special step of building Perl with MSVC probably  
build PostgreSQL with MSVC.  
  
Discussion: https://postgr.es/m/20171130041441.GA3161526@rfd.leadboat.com  

M config/perl.m4
M src/tools/msvc/Mkvcbuild.pm

Fix mistake in comment

commit   : 8b33b5b9dfce9b7fb94b8996f4e0bac7e5bb6c21    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Fri, 8 Dec 2017 11:16:23 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Fri, 8 Dec 2017 11:16:23 -0500    

Click here for diff

Reported-by: Masahiko Sawada <sawada.mshk@gmail.com>  

M src/backend/access/transam/xlog.c

doc: Add advice about systemd RemoveIPC

commit   : 5f82b3f7c60c6a4257c8693fbd500cbf3916fea9    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Wed, 15 Feb 2017 10:44:07 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Wed, 15 Feb 2017 10:44:07 -0500    

Click here for diff

Reviewed-by: Magnus Hagander <magnus@hagander.net>  

M doc/src/sgml/runtime.sgml

Report failure to start a background worker.

commit   : facd94e72f27992ea5488aa59364778d54f978b8    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Wed, 6 Dec 2017 08:49:30 -0500    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Wed, 6 Dec 2017 08:49:30 -0500    

Click here for diff

When a worker is flagged as BGW_NEVER_RESTART and we fail to start it,  
or if it is not marked BGW_NEVER_RESTART but is terminated before  
startup succeeds, what BgwHandleStatus should be reported?  The  
previous code really hadn't considered this possibility (as indicated  
by the comments which ignore it completely) and would typically return  
BGWH_NOT_YET_STARTED, but that's not a good answer, because then  
there's no way for code using GetBackgroundWorkerPid() to tell the  
difference between a worker that has not started but will start  
later and a worker that has not started and will never be started.  
So, when this case happens, return BGWH_STOPPED instead.  Update the  
comments to reflect this.  
  
The preceding fix by itself is insufficient to fix the problem,  
because the old code also didn't send a notification to the process  
identified in bgw_notify_pid when startup failed.  That might've  
been technically correct under the theory that the status of the  
worker was BGWH_NOT_YET_STARTED, because the status would indeed not  
change when the worker failed to start, but now that we're more  
usefully reporting BGWH_STOPPED, a notification is needed.  
  
Without these fixes, code which starts background workers and then  
uses the recommended APIs to wait for those background workers to  
start would hang indefinitely if the postmaster failed to fork a  
worker.  
  
Amit Kapila and Robert Haas  
  
Discussion: http://postgr.es/m/CAA4eK1KDfKkvrjxsKJi3WPyceVi3dH1VCkbTJji2fuwKuB=3uw@mail.gmail.com  

M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c

Mark assorted variables PGDLLIMPORT.

commit   : f4bb60ed69d8d88beef8fa22ad2fcbb99769ef4d    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Tue, 5 Dec 2017 09:24:12 -0500    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Tue, 5 Dec 2017 09:24:12 -0500    

Click here for diff

This makes life easier for extension authors who wish to support  
Windows.  
  
Brian Cloutier, slightly amended by me.  
  
Discussion: http://postgr.es/m/CAJCy68fscdNhmzFPS4kyO00CADkvXvEa-28H-OtENk-pa2OTWw@mail.gmail.com  

M src/include/access/twophase.h
M src/include/commands/extension.h
M src/include/miscadmin.h
M src/include/pgtime.h
M src/include/postmaster/postmaster.h
M src/include/storage/fd.h
M src/include/storage/proc.h
M src/include/tcop/dest.h
M src/include/tcop/tcopprot.h
M src/include/utils/guc.h
M src/include/utils/snapmgr.h

Clean up assorted messiness around AllocateDir() usage.

commit   : 225501cf754f1ce0fd0c793c0c991f40e96344e1    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 4 Dec 2017 17:02:52 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 4 Dec 2017 17:02:52 -0500    

Click here for diff

This patch fixes a couple of low-probability bugs that could lead to  
reporting an irrelevant errno value (and hence possibly a wrong SQLSTATE)  
concerning directory-open or file-open failures.  It also fixes places  
where we took shortcuts in reporting such errors, either by using elog  
instead of ereport or by using ereport but forgetting to specify an  
errcode.  And it eliminates a lot of just plain redundant error-handling  
code.  
  
In service of all this, export fd.c's formerly-static function  
ReadDirExtended, so that external callers can make use of the coding  
pattern  
  
	dir = AllocateDir(path);  
	while ((de = ReadDirExtended(dir, path, LOG)) != NULL)  
  
if they'd like to treat directory-open failures as mere LOG conditions  
rather than errors.  Also fix FreeDir to be a no-op if we reach it  
with dir == NULL, as such a coding pattern would cause.  
  
Then, remove code at many call sites that was throwing an error or log  
message for AllocateDir failure, as ReadDir or ReadDirExtended can handle  
that job just fine.  Aside from being a net code savings, this gets rid of  
a lot of not-quite-up-to-snuff reports, as mentioned above.  (In some  
places these changes result in replacing a custom error message such as  
"could not open tablespace directory" with more generic wording "could not  
open directory", but it was agreed that the custom wording buys little as  
long as we report the directory name.)  In some other call sites where we  
can't just remove code, change the error reports to be fully  
project-style-compliant.  
  
Also reorder code in restoreTwoPhaseData that was acquiring a lock  
between AllocateDir and ReadDir; in the unlikely but surely not  
impossible case that LWLockAcquire changes errno, AllocateDir failures  
would be misreported.  There is no great value in opening the directory  
before acquiring TwoPhaseStateLock, so just do it in the other order.  
  
Also fix CheckXLogRemoved to guarantee that it preserves errno,  
as quite a number of call sites are implicitly assuming.  (Again,  
it's unlikely but I think not impossible that errno could change  
during a SpinLockAcquire.  If so, this function was broken for its  
own purposes as well as breaking callers.)  
  
And change a few places that were using not-per-project-style messages,  
such as "could not read directory" when "could not open directory" is  
more correct.  
  
Back-patch the exporting of ReadDirExtended, in case we have occasion  
to back-patch some fix that makes use of it; it's not needed right now  
but surely making it global is pretty harmless.  Also back-patch the  
restoreTwoPhaseData and CheckXLogRemoved fixes.  The rest of this is  
essentially cosmetic and need not get back-patched.  
  
Michael Paquier, with a bit of additional work by me  
  
Discussion: https://postgr.es/m/CAB7nPqRpOCxjiirHmebEFhXVTK7V5Jvw4bz82p7Oimtsm3TyZA@mail.gmail.com  

M src/backend/access/transam/xlog.c
M src/backend/storage/file/fd.c
M src/include/storage/fd.h

Fix non-GNU makefiles for AIX make.

commit   : e73981cdc0fde135eefb41696584ceba8efcbf19    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Thu, 30 Nov 2017 00:57:22 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Thu, 30 Nov 2017 00:57:22 -0800    

Click here for diff

Invoking the Makefile without an explicit target was building every  
possible target instead of just the "all" target.  Back-patch to 9.3  
(all supported versions).  

M Makefile
M src/test/regress/Makefile

Fix typo in comment

commit   : 65f1623336a39f65a9d8a34e06f127cae583ff3b    
  
author   : Magnus Hagander <magnus@hagander.net>    
date     : Mon, 27 Nov 2017 09:30:03 +0100    
  
committer: Magnus Hagander <magnus@hagander.net>    
date     : Mon, 27 Nov 2017 09:30:03 +0100    

Click here for diff

Andreas Karlsson  

M src/tools/msvc/config_default.pl

Make has_sequence_privilege support WITH GRANT OPTION

commit   : d8d9c97cd118a8411ea238a07c46c4ad22767fab    
  
author   : Joe Conway <mail@joeconway.com>    
date     : Sun, 26 Nov 2017 09:50:42 -0800    
  
committer: Joe Conway <mail@joeconway.com>    
date     : Sun, 26 Nov 2017 09:50:42 -0800    

Click here for diff

The various has_*_privilege() functions all support an optional  
WITH GRANT OPTION added to the supported privilege types to test  
whether the privilege is held with grant option. That is, all except  
has_sequence_privilege() variations. Fix that.  
  
Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/005147f6-8280-42e9-5a03-dd2c1e4397ef@joeconway.com  

M src/backend/utils/adt/acl.c

Update MSVC build process for new timezone data.

commit   : 1601a9413fbfb98370c3299e3c9da514a0331c6d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 18:15:23 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 18:15:23 -0500    

Click here for diff

Missed this dependency in commits 7cce222c9 et al.  

M src/tools/msvc/Install.pm

Replace raw timezone source data with IANA’s new compact format.

commit   : 10aa064c953e5e7e7cccf545558a8336e5fdb220    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 15:30:11 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 15:30:11 -0500    

Click here for diff

Traditionally IANA has distributed their timezone data in pure source  
form, replete with extensive historical comments.  As of release 2017c,  
they've added a compact single-file format that omits comments and  
abbreviates command keywords.  This form is way shorter than the pure  
source, even before considering its allegedly better compressibility.  
Hence, let's distribute the data in that form rather than pure source.  
  
I'm pushing this now, rather than at the next timezone database update,  
so that it's easy to confirm that this data file produces compiled zic  
output that's identical to what we were getting before.  
  
Discussion: https://postgr.es/m/1915.1511210334@sss.pgh.pa.us  

M src/timezone/Makefile
M src/timezone/README
D src/timezone/data/africa
D src/timezone/data/antarctica
D src/timezone/data/asia
D src/timezone/data/australasia
D src/timezone/data/backward
D src/timezone/data/backzone
D src/timezone/data/etcetera
D src/timezone/data/europe
D src/timezone/data/factory
D src/timezone/data/northamerica
D src/timezone/data/pacificnew
D src/timezone/data/southamerica
D src/timezone/data/systemv
A src/timezone/data/tzdata.zi

Avoid formally-undefined use of memcpy() in hstoreUniquePairs().

commit   : 5c38ddebd66bc239b23fe20d489fe03c9ebd4e28    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 14:42:10 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 14:42:10 -0500    

Click here for diff

hstoreUniquePairs() often called memcpy with equal source and destination  
pointers.  Although this is almost surely harmless in practice, it's  
undefined according to the letter of the C standard.  Some versions of  
valgrind will complain about it, and some versions of libc as well  
(cf. commit ad520ec4a).  Tweak the code to avoid doing that.  
  
Noted by Tomas Vondra.  Back-patch to all supported versions because  
of the hazard of libc assertions.  
  
Discussion: https://postgr.es/m/bf84d940-90d4-de91-19dd-612e011007f4@fuzzy.cz  

M contrib/hstore/hstore_io.c

Repair failure with SubPlans in multi-row VALUES lists.

commit   : 2e105cf6db214ec69e62e4c4dfc71e39879485ae    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 14:15:48 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 25 Nov 2017 14:15:48 -0500    

Click here for diff

When nodeValuesscan.c was written, it was impossible to have a SubPlan in  
VALUES --- any sub-SELECT there would have to be uncorrelated and thereby  
would produce an InitPlan instead.  We therefore took a shortcut in the  
logic that throws away a ValuesScan's per-row expression evaluation data  
structures.  This was broken by the introduction of LATERAL however; a  
sub-SELECT containing a lateral reference produces a correlated SubPlan.  
  
The cleanest fix for this would be to give up the optimization of  
discarding the expression eval state.  But that still seems pretty  
unappetizing for long VALUES lists.  It seems to work to just prevent  
the subexpressions from hooking into the ValuesScan node's subPlan  
list, so let's do that and see how well it works.  (If this breaks,  
due to additional connections between the subexpressions and the outer  
query structures, we might consider compromises like throwing away data  
only for VALUES rows not containing SubPlans.)  
  
Per bug #14924 from Christian Duta.  Back-patch to 9.3 where LATERAL  
was introduced.  
  
Discussion: https://postgr.es/m/20171124120836.1463.5310@wrigleys.postgresql.org  

M src/backend/executor/nodeValuesscan.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql

Support linking with MinGW-built Perl.

commit   : 558f6207928b7586d0ea342c4ec05744e10f9c2a    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Thu, 23 Nov 2017 20:22:04 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Thu, 23 Nov 2017 20:22:04 -0800    

Click here for diff

This is necessary for ActivePerl 5.18 onwards and for Strawberry Perl.  
It is not sufficient for 32-bit builds with newer Visual Studio; these  
fail with error LINK2026.  Back-patch to 9.3 (all supported versions).  
  
Reported by Victor Wagner.  
  
Discussion: https://postgr.es/m/20160326154321.7754ab8f@wagner.wagner.home  

M config/perl.m4
M configure
M src/pl/plperl/plperl.h
M src/tools/msvc/Mkvcbuild.pm

Provide for forward compatibility with future minor protocol versions.

commit   : 294136d42230467b1f5496693aac847ce2199e64    
  
author   : Robert Haas <rhaas@postgresql.org>    
date     : Tue, 21 Nov 2017 13:56:24 -0500    
  
committer: Robert Haas <rhaas@postgresql.org>    
date     : Tue, 21 Nov 2017 13:56:24 -0500    

Click here for diff

Previously, any attempt to request a 3.x protocol version other than  
3.0 would lead to a hard connection failure, which made the minor  
protocol version really no different from the major protocol version  
and precluded gentle protocol version breaks.  Instead, when the  
client requests a 3.x protocol version where x is greater than 0, send  
the new NegotiateProtocolVersion message to convey that we support  
only 3.0.  This makes it possible to introduce new minor protocol  
versions without requiring a connection retry when the server is  
older.  
  
In addition, if the startup packet includes name/value pairs where  
the name starts with "_pq_.", assume that those are protocol options,  
not GUCs.  Include those we don't support (i.e. all of them, at  
present) in the NegotiateProtocolVersion message so that the client  
knows they were not understood.  This makes it possible for the  
client to request previously-unsupported features without bumping  
the protocol version at all; the client can tell from the server's  
response whether the option was understood.  
  
It will take some time before servers that support these new  
facilities become common in the wild; to speed things up and make  
things easier for a future 3.1 protocol version, back-patch to all  
supported releases.  
  
Robert Haas and Badrul Chowdhury  
  
Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com  
Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us  

M doc/src/sgml/protocol.sgml
M src/backend/postmaster/postmaster.c

Use out-of-line M68K spinlock code for OpenBSD as well as NetBSD.

commit   : 13f2bdb6394ff7f933955609bb42d4381240a602    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Nov 2017 18:05:03 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Nov 2017 18:05:03 -0500    

Click here for diff

David Carlier (from a patch being carried by OpenBSD packagers)  
  
Discussion: https://postgr.es/m/CA+XhMqzwFSGVU7MEnfhCecc8YdP98tigXzzpd0AAdwaGwaVXEA@mail.gmail.com  

M src/backend/storage/lmgr/s_lock.c

Add support for Motorola 88K to s_lock.h.

commit   : 8bd8b4b77c300c659667b16ef72a2d5c5ac63f49    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Nov 2017 17:57:46 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 20 Nov 2017 17:57:46 -0500    

Click here for diff

Apparently there are still people out there who care about this old  
architecture.  They probably care about dusty versions of Postgres  
too, so back-patch to all supported branches.  
  
David Carlier (from a patch being carried by OpenBSD packagers)  
  
Discussion: https://postgr.es/m/CA+XhMqzwFSGVU7MEnfhCecc8YdP98tigXzzpd0AAdwaGwaVXEA@mail.gmail.com  

M src/include/storage/s_lock.h

Provide modern examples of how to auto-start Postgres on macOS.

commit   : 7d98dc1332957716372ab5e9961fae480ccd2287    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Nov 2017 12:46:52 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Fri, 17 Nov 2017 12:46:52 -0500    

Click here for diff

The scripts in contrib/start-scripts/osx don't work at all on macOS  
10.10 (Yosemite) or later, because they depend on SystemStarter which  
Apple deprecated long ago and removed in 10.10.  Add a new subdirectory  
contrib/start-scripts/macos with scripts that use the newer launchd  
infrastructure.  
  
Since this problem is independent of which Postgres version you're using,  
back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/31338.1510763554@sss.pgh.pa.us  

A contrib/start-scripts/macos/README
A contrib/start-scripts/macos/org.postgresql.postgres.plist
A contrib/start-scripts/macos/postgres-wrapper.sh
M contrib/start-scripts/osx/README

MSVC: Rebuild spiexceptions.h when out of date.

commit   : ab8eae0bb5a1345eaa39aceb584623ae41691e5b    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sun, 12 Nov 2017 18:43:32 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sun, 12 Nov 2017 18:43:32 -0800    

Click here for diff

Also, add a warning to catch future instances of naming a nonexistent  
file as a prerequisite.  Back-patch to 9.3 (all supported versions).  

M src/tools/msvc/Solution.pm

Install Windows crash dump handler before all else.

commit   : e17b38db663949e7b22aa92c2c57dcdf518ecee9    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sun, 12 Nov 2017 14:31:00 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sun, 12 Nov 2017 14:31:00 -0800    

Click here for diff

Apart from calling write_stderr() on failure, the handler depends on no  
PostgreSQL facilities.  We have experienced crashes before reaching the  
former call site.  Given such an early crash, this change cannot hurt  
and may produce a helpful dump.  Absent an early crash, this change has  
no effect.  Back-patch to 9.3 (all supported versions).  
  
Takayuki Tsunakawa  
  
Discussion: https://postgr.es/m/0A3221C70F24FB45833433255569204D1F80CD13@G01JPEXMBYT05  

M src/backend/main/main.c

Don’t call pgwin32_message_to_UTF16() without CurrentMemoryContext.

commit   : 19cf9e96ae11b549073da6e63b317ec7ac1fb61e    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sun, 12 Nov 2017 13:03:15 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sun, 12 Nov 2017 13:03:15 -0800    

Click here for diff

PostgreSQL running as a Windows service crashed upon calling  
write_stderr() before MemoryContextInit().  This fix completes work  
started in 5735efee15540765315aa8c1a230575e756037f7.  Messages this  
early contain only ASCII bytes; if we removed the CurrentMemoryContext  
requirement, the ensuing conversions would have no effect.  Back-patch  
to 9.3 (all supported versions).  
  
Takayuki Tsunakawa, reviewed by Michael Paquier.  
  
Discussion: https://postgr.es/m/0A3221C70F24FB45833433255569204D1F80CC73@G01JPEXMBYT05  

M src/backend/utils/error/elog.c
M src/backend/utils/mb/mbutils.c

Add post-2010 ecpg tests to checktcp.

commit   : ae5489e1470c04e8c96999204d85d940dc737142    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 14:39:06 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 14:39:06 -0800    

Click here for diff

This suite had been a proper superset of the regular ecpg test suite,  
but the three newest tests didn't reach it.  To make this less likely to  
recur, delete the extra schedule file and pass the TCP-specific test on  
the command line.  Back-patch to 9.3 (all supported versions).  

M src/interfaces/ecpg/test/Makefile
D src/interfaces/ecpg/test/ecpg_schedule_tcp

Make connect/test1 independent of localhost IPv6.

commit   : 65fd34f7cc6c4ab729b93bef55095c0156d7c0d4    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 14:33:02 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 14:33:02 -0800    

Click here for diff

Since commit 868898739a8da9ab74c105b8349b7b5c711f265a, it has assumed  
"localhost" resolves to both ::1 and 127.0.0.1.  We gain nothing from  
that assumption, and it does not hold in a default installation of Red  
Hat Enterprise Linux 5.  Back-patch to 9.3 (all supported versions).  

M src/interfaces/ecpg/test/connect/test1.pgc
M src/interfaces/ecpg/test/expected/connect-test1.c
M src/interfaces/ecpg/test/expected/connect-test1.stderr

Fix connect/test1 expected output.

commit   : dfabce8827e65c3a48051526451c1b514b223ba1    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 14:22:29 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 14:22:29 -0800    

Click here for diff

The test runs only as part of "checktcp".  This is a back-patch to 9.5  
and 9.4 of part of commit 868898739a8da9ab74c105b8349b7b5c711f265a.  
Oversight in commit 61bee9f756ce875f3b678099a6bb9654bd2fa21a.  

M src/interfaces/ecpg/test/expected/connect-test1.stderr

Fix previous commit’s test, for non-UTF8 databases with non-XML builds.

commit   : e48fb50d8b4868de821f09f19985e35dc5f41c44    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 13:07:46 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 13:07:46 -0800    

Click here for diff

To ensure stable output, catch one more configuration-specific error.  
Back-patch to 9.3, like the commit that added the test.  

M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql

Ignore XML declaration in xpath_internal(), for UTF8 databases.

commit   : 2f4061aff504049767602927e69c85b5b4621273    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 11:10:53 -0800    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sat, 11 Nov 2017 11:10:53 -0800    

Click here for diff

When a value contained an XML declaration naming some other encoding,  
this function interpreted UTF8 bytes as the named encoding, yielding  
mojibake.  xml_parse() already has similar logic.  This would be  
necessary but not sufficient for non-UTF8 databases, so preserve  
behavior there until the xpath facility can support such databases  
comprehensively.  Back-patch to 9.3 (all supported versions).  
  
Pavel Stehule and Noah Misch  
  
Discussion: https://postgr.es/m/CAFj8pRC-dM=tT=QkGi+Achkm+gwPmjyOayGuUfXVumCxkDgYWg@mail.gmail.com  

M src/backend/utils/adt/xml.c
M src/test/regress/expected/xml.out
M src/test/regress/expected/xml_1.out
M src/test/regress/expected/xml_2.out
M src/test/regress/sql/xml.sql

Fix some null pointer dereferences in LDAP auth code

commit   : 0bcdab58e83b2ac2caf4bd78e1583c11df97d4f1    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Fri, 10 Nov 2017 14:21:32 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Fri, 10 Nov 2017 14:21:32 -0500    

Click here for diff

An LDAP URL without a host name such as "ldap://" or without a base DN  
such as "ldap://localhost" would cause a crash when reading pg_hba.conf.  
  
If no binddn is configured, an error message might end up trying to print a  
null pointer, which could crash on some platforms.  
  
Author: Thomas Munro <thomas.munro@enterprisedb.com>  
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>  

M src/backend/libpq/auth.c
M src/backend/libpq/hba.c

Add -wnet to SP invocations

commit   : 18d431bda22f1b0f9ad758338a282705ab6b21b5    
  
author   : Peter Eisentraut <peter_e@gmx.net>    
date     : Thu, 9 Nov 2017 17:06:32 -0500    
  
committer: Peter Eisentraut <peter_e@gmx.net>    
date     : Thu, 9 Nov 2017 17:06:32 -0500    

Click here for diff

This causes a warning when accidentally backpatching an XML-style  
empty-element tag like <xref linkend="abc"/>.  

M doc/src/sgml/Makefile

Fix typo in ALTER SYSTEM output.

commit   : b2ad6f9e2614f80e6dd88fe584e3aefac52cc454    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Nov 2017 11:57:20 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Nov 2017 11:57:20 -0500    

Click here for diff

The header comment written into postgresql.auto.conf by ALTER SYSTEM  
should match what initdb put there originally.  
  
Feike Steenbergen  
  
Discussion: https://postgr.es/m/CAK_s-G0KcKdO=0hqZkwb3s+tqZuuHwWqmF5BDsmoO9FtX75r0g@mail.gmail.com  

M src/backend/utils/misc/guc.c

Revert “Allow –with-bonjour to work with non-macOS implementations of Bonjour.”

commit   : 9aa6a1b29b0238802583a3b358142752a77c253d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Nov 2017 11:00:36 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Thu, 9 Nov 2017 11:00:36 -0500    

Click here for diff

Upon further review, our Bonjour code doesn't actually work with the  
Avahi not-too-compatible compatibility library.  While you can get it  
to work on non-macOS platforms if you link to Apple's own mDNSResponder  
code, there don't seem to be many people who care about that.  Leaving in  
the AC_SEARCH_LIBS call seems more likely to encourage people to build  
broken configurations than to do anything very useful.  
  
Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.  
  
Discussion: https://postgr.es/m/2D8331C5-D64F-44C1-8717-63EDC6EAF7EB@brightforge.com  

M configure
M configure.in

Allow –with-bonjour to work with non-macOS implementations of Bonjour.

commit   : f57b070943e839a08a3676ae0b09b27dcbdec8a9    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Nov 2017 17:47:14 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Nov 2017 17:47:14 -0500    

Click here for diff

On macOS the relevant functions require no special library, but elsewhere  
we need to pull in libdns_sd.  
  
Back-patch to supported branches.  No docs change since the docs do not  
suggest that this is a Mac-only feature.  
  
Luke Lonergan  
  
Discussion: https://postgr.es/m/2D8331C5-D64F-44C1-8717-63EDC6EAF7EB@brightforge.com  

M configure
M configure.in

Fix two violations of the ResourceOwnerEnlarge/Remember protocol.

commit   : 1786cdab113701f154ca804537fe1c21d23e4ce8    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Nov 2017 16:50:13 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Wed, 8 Nov 2017 16:50:13 -0500    

Click here for diff

The point of having separate ResourceOwnerEnlargeFoo and  
ResourceOwnerRememberFoo functions is so that resource allocation  
can happen in between.  Doing it in some other order is just wrong.  
  
OpenTemporaryFile() did open(), enlarge, remember, which would leak the  
open file if the enlarge step ran out of memory.  Because fd.c has its own  
layer of resource-remembering, the consequences look like they'd be limited  
to an intratransaction FD leak, but it's still not good.  
  
IncrBufferRefCount() did enlarge, remember, incr-refcount, which would blow  
up if the incr-refcount step ever failed.  It was safe enough when written,  
but since the introduction of PrivateRefCountHash, I think the assumption  
that no error could happen there is pretty shaky.  
  
The odds of real problems from either bug are probably small, but still,  
back-patch to supported branches.  
  
Thomas Munro and Tom Lane, per a comment from Andres Freund  

M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/file/fd.c

Fix unportable usage of functions.

commit   : 7a15fe5a2240244e2f80f25adfe6b9a34787a021    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 7 Nov 2017 13:49:36 -0500    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 7 Nov 2017 13:49:36 -0500    

Click here for diff

isdigit(), isspace(), etc are likely to give surprising results if passed a  
signed char.  We should always cast the argument to unsigned char to avoid  
that.  Error in commit 63d6b97fd, found by buildfarm member gaur.  
Back-patch to 9.3, like that commit.  

M src/interfaces/ecpg/ecpglib/data.c