PostgreSQL 10.10 (upcoming) commit log

Fix thinko in construction of old_conpfeqop list.

commit   : 583025c3c7e61448d1e6facca770a2171666428c    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 16 Jul 2019 18:17:47 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 16 Jul 2019 18:17:47 -0400    

Click here for diff

This should lappend the OIDs, not lcons them; the existing code produced  
a list in reversed order.  This is harmless for single-key FKs or FKs  
where all the key columns are of the same type, which probably explains  
how it went unnoticed.  But if those conditions are not met,  
ATAddForeignKeyConstraint would make the wrong decision about whether an  
existing FK needs to be revalidated.  I think it would almost always err  
in the safe direction by revalidating a constraint that didn't need it.  
You could imagine scenarios where the pfeqop check was fooled by  
swapping the types of two FK columns in one ALTER TABLE, but that case  
would probably be rejected by other tests, so it might be impossible to  
get to the worst-case scenario where an FK should be revalidated and  
isn't.  (And even then, it's likely to be fine, unless there are weird  
inconsistencies in the equality behavior of the replacement types.)  
However, this is a performance bug at least.  
  
Noted while poking around to see whether lcons calls could be converted  
to lappend.  
  
This bug is old, dating to commit cb3a7c2b9, so back-patch to all  
supported branches.  

M src/backend/commands/tablecmds.c

doc: mention pg_reload_conf() for reloading the config file

commit   : f68060cadb390a3653221a99bdc19be59ca58787    
  
author   : Bruce Momjian <bruce@momjian.us>    
date     : Mon, 15 Jul 2019 20:57:24 -0400    
  
committer: Bruce Momjian <bruce@momjian.us>    
date     : Mon, 15 Jul 2019 20:57:24 -0400    

Click here for diff

Reported-by: Ian Barwick  
  
Discussion: https://postgr.es/m/538950ec-b86a-1650-6078-beb7091c09c2@2ndquadrant.com  
  
Backpatch-through: 9.4  

M doc/src/sgml/client-auth.sgml

Fix documentation for pgbench tpcb-like.

commit   : 15f2f2759ecd0c2cf2a72644f3b1407478876968    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Sun, 14 Jul 2019 14:19:54 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Sun, 14 Jul 2019 14:19:54 +1200    

Click here for diff

We choose a random value for delta, not balance.  Back-patch to 9.6 where  
the mistake arrived.  
  
Author: Fabien Coelho  
Discussion: https://postgr.es/m/alpine.DEB.2.21.1904081752210.5867@lancre  

M doc/src/sgml/ref/pgbench.sgml

Fix variable initialization when using buffering build with GiST

commit   : 4fea3434939dfb29a16bf61baca8020e5142ccda    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 10 Jul 2019 15:15:11 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 10 Jul 2019 15:15:11 +0900    

Click here for diff

This can cause valgrind to complain, as the flag marking a buffer as a  
temporary copy was not getting initialized.  
  
While on it, fill in with zeros newly-created buffer pages.  This does  
not matter when loading a block from a temporary file, but it makes the  
push of an index tuple into a new buffer page safer.  
  
This has been introduced by 1d27dcf, so backpatch all the way down to  
9.4.  
  
Author: Alexander Lakhin  
Discussion: https://postgr.es/m/15899-0d24fb273b3dd90c@postgresql.org  
Backpatch-through: 9.4  

M src/backend/access/gist/gistbuildbuffers.c

Pass QueryEnvironment down to EvalPlanQual's EState.

commit   : 72b526779a9320e2dd8ff63237fbaf2e933b5c34    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Wed, 10 Jul 2019 10:16:02 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Wed, 10 Jul 2019 10:16:02 +1200    

Click here for diff

Otherwise the executor can't see trigger transition tables during  
EPQ evaluation.  Fixes bug #15900 and almost certainly also #15720.  
Back-patch to 10, where trigger transition tables landed.  
  
Author: Alex Aktsipetrov  
Reviewed-by: Thomas Munro, Tom Lane  
Discussion: https://postgr.es/m/15900-bc482754fe8d7415%40postgresql.org  
Discussion: https://postgr.es/m/15720-38c2b29e5d720187%40postgresql.org  

M src/backend/executor/execMain.c

doc: Clarify logical replication documentation

commit   : a5407d1bd6286c0f932514b8dc58f472981e0243    
  
author   : Peter Eisentraut <peter@eisentraut.org>    
date     : Mon, 8 Jul 2019 14:28:42 +0200    
  
committer: Peter Eisentraut <peter@eisentraut.org>    
date     : Mon, 8 Jul 2019 14:28:42 +0200    

Click here for diff

Document that the data types of replicated tables do not need to  
match.  The documentation previously claimed that they had to match.  
  
Author: Robert Treat <rob@xzilla.net>  
Discussion: https://www.postgresql.org/message-id/flat/CAJSLCQ13==D8Ka2YLyctTm0Y+8MhGYcX_zj7fU0rqRzhcV++3w@mail.gmail.com  

M doc/src/sgml/logical-replication.sgml

Don't remove surplus columns from GROUP BY for inheritance parents

commit   : 232019b79229db7a438f00f358d86659fbd6fcb9    
  
author   : David Rowley <drowley@postgresql.org>    
date     : Wed, 3 Jul 2019 23:46:06 +1200    
  
committer: David Rowley <drowley@postgresql.org>    
date     : Wed, 3 Jul 2019 23:46:06 +1200    

Click here for diff

d4c3a156c added code to remove columns that were not part of a table's  
PRIMARY KEY constraint from the GROUP BY clause when all the primary key  
columns were present in the group by.  This is fine to do since we know  
that there will only be one row per group coming from this relation.  
However, the logic failed to consider inheritance parent relations.  These  
can have child relations without a primary key, but even if they did, they  
could duplicate one of the parent's rows or one from another child  
relation.  In this case, those additional GROUP BY columns are required.  
  
Fix this by disabling the optimization for inheritance parent tables.  
In v11 and beyond, partitioned tables are fine since partitions cannot  
overlap and before v11 partitioned tables could not have a primary key.  
  
Reported-by: Manuel Rigger  
Discussion: http://postgr.es/m/CA+u7OA7VLKf_vEr6kLF3MnWSA9LToJYncgpNX2tQ-oWzYCBQAw@mail.gmail.com  
Backpatch-through: 9.6  

M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql

Add support for Visual Studio 2019 in build scripts

commit   : 0ce8e49b224300fe895f5cda6b8e45d80bbdf860    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 3 Jul 2019 08:58:04 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 3 Jul 2019 08:58:04 +0900    

Click here for diff

This adjusts the documentation and the scripts related to the versions  
of Windows SDK supported.  
  
Author: Haribabu Kommi  
Reviewed-by: Andrew Dunstan, Juan José Santamaría Flecha, Michael  
Paquier  
Discussion: https://postgr.es/m/CAJrrPGcfqXhfPyMrny9apoDU7M1t59dzVAvoJ9AeAh5BJi+UzA@mail.gmail.com  
Backpatch-through: 9.4  

M doc/src/sgml/install-windows.sgml
M src/tools/msvc/MSBuildProject.pm
M src/tools/msvc/README
M src/tools/msvc/Solution.pm
M src/tools/msvc/VSObjectFactory.pm

Fix tab completion of "SET variable TO|=" to not offer bogus completions.

commit   : 90434e6f2cf9fb629069b1b3017ea1a2c1ab07eb    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 2 Jul 2019 13:35:14 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 2 Jul 2019 13:35:14 -0400    

Click here for diff

Don't think that the context "UPDATE tab SET var =" is a GUC-setting  
command.  
  
If we have "SET var =" but the "var" is not a known GUC variable,  
don't offer any completions.  The most likely explanation is that  
we've misparsed the context and it's not really a GUC-setting command.  
  
Per gripe from Ken Tanzer.  Back-patch to 9.6.  The issue exists  
further back, but before 9.6 the code looks very different and it  
doesn't actually know whether the "var" name matches anything,  
so I desisted from trying to fix it.  
  
Discussion: https://postgr.es/m/CAD3a31XpXzrZA9TT3BqLSHghdTK+=cXjNCE+oL2Zn4+oWoc=qA@mail.gmail.com  

M src/bin/psql/tab-complete.c

Don't read fields of a misaligned ExpandedObjectHeader or AnyArrayType.

commit   : cd9d48969d944c3558a77745db389e8b68b46c4b    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Sun, 30 Jun 2019 17:34:17 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Sun, 30 Jun 2019 17:34:17 -0700    

Click here for diff

UBSan complains about this.  Instead, cast to a suitable type requiring  
only 4-byte alignment.  DatumGetAnyArrayP() already assumes one can cast  
between AnyArrayType and ArrayType, so this doesn't introduce a new  
assumption.  Back-patch to 9.5, where AnyArrayType was introduced.  
  
Reviewed by Tom Lane.  
  
Discussion: https://postgr.es/m/20190629210334.GA1244217@rfd.leadboat.com  

M src/backend/utils/adt/arrayfuncs.c
M src/include/utils/array.h
M src/include/utils/arrayaccess.h
M src/include/utils/expandeddatum.h

Repair logic for reordering grouping sets optimization.

commit   : a1637caee9c77e30aaf4afb5c51e15cb67c4f3e3    
  
author   : Andrew Gierth <rhodiumtoad@postgresql.org>    
date     : Sun, 30 Jun 2019 23:49:25 +0100    
  
committer: Andrew Gierth <rhodiumtoad@postgresql.org>    
date     : Sun, 30 Jun 2019 23:49:25 +0100    

Click here for diff

The logic in reorder_grouping_sets to order grouping set elements to  
match a pre-specified sort ordering was defective, resulting in  
unnecessary sort nodes (though the query output would still be  
correct). Repair, simplifying the code a little, and add a test.  
  
Per report from Richard Guo, though I didn't use their patch. Original  
bug seems to have been my fault.  
  
Backpatch back to 9.5 where grouping sets were introduced.  
  
Discussion: https://postgr.es/m/CAN_9JTzyjGcUjiBHxLsgqfk7PkdLGXiM=pwM+=ph2LsWw0WO1A@mail.gmail.com  

M src/backend/optimizer/plan/planner.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql

Fix misleading comment in nodeIndexonlyscan.c.

commit   : 69da8c1e69efd3a5b1b0f1d9bd8b7b79a696fbc8    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Fri, 28 Jun 2019 11:11:26 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Fri, 28 Jun 2019 11:11:26 +1200    

Click here for diff

The stated reason for acquiring predicate locks on heap pages hasn't  
existed since commit c01262a8, so fix the comment.  Perhaps in a later  
release we'll also be able to change the code to use tuple locks.  
  
Back-patch all the way.  
  
Reviewed-by: Ashwin Agrawal  
Discussion: https://postgr.es/m/CAEepm%3D2GK3FVdnt5V3d%2Bh9njWipCv_fNL%3DwjxyUhzsF%3D0PcbNg%40mail.gmail.com  

M src/backend/executor/nodeIndexonlyscan.c

Update reference to sampling algorithm in analyze.c

commit   : 0161e522cbb536d8978ca8e5b5341b16805cc6b4    
  
author   : Tomas Vondra <tomas.vondra@postgresql.org>    
date     : Thu, 27 Jun 2019 18:14:25 +0200    
  
committer: Tomas Vondra <tomas.vondra@postgresql.org>    
date     : Thu, 27 Jun 2019 18:14:25 +0200    

Click here for diff

Commit 83e176ec1 moved row sampling functions from analyze.c to  
utils/misc/sampling.c, but failed to update comment referring to  
the sampling algorithm from Jeff Vitter's paper. Correct the  
comment by pointing to utils/misc/sampling.c.  
  
Author: Etsuro Fujita  
Discussion: https://postgr.es/m/CAPmGK154gp%2BQd%3DcorQOv%2BPmbyVyZBjp_%2Bhb766UJeD1e_ie6XQ%40mail.gmail.com  

M src/backend/commands/analyze.c

Add support for OpenSSL 1.1.0 and newer versions in MSVC scripts

commit   : a559805597d5e860796d6c9430134b8e41022ef8    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Wed, 26 Jun 2019 23:05:06 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Wed, 26 Jun 2019 23:05:06 +0900    

Click here for diff

Up to now, the MSVC build scripts are able to support only one fixed  
version of OpenSSL, and they lacked logic to detect the version of  
OpenSSL a given compilation of Postgres is linking to (currently 1.0.2,  
the latest LTS of upstream which will be EOL'd at the end of 2019).  
  
This commit adds more logic to detect the version of OpenSSL used by a  
build and makes use of it to add support for compilation with OpenSSL  
1.1.0 which requires a new set of compilation flags to work properly.  
  
The supported OpenSSL installers have changed their library layer with  
various library renames with the upgrade to 1.1.0, making the logic a  
bit more complicated.  The scripts are now able to adapt to the new  
world order.  
  
Reported-by: Sergey Pashkov  
Author: Juan José Santamaría Flecha, Michael Paquier  
Reviewed-by: Álvaro Herrera  
Discussion: https://postgr.es/m/15789-8fc75dea3c5a17c8@postgresql.org  
Backpatch-through: 9.4  

M src/tools/msvc/Solution.pm

Follow the rule that regression-test-created roles are named "regress_xxx".

commit   : 80931442c9c93b6c94d56808697648e44f0625ee    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 25 Jun 2019 23:19:31 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Tue, 25 Jun 2019 23:19:31 -0400    

Click here for diff

contrib/amcheck didn't get the memo either.  

M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/sql/check_btree.sql

Fix thinkos in LookupFuncName() for function name lookups

commit   : 131e545ac304a3f5c4f8575131f4b338232cac0d    
  
author   : Michael Paquier <michael@paquier.xyz>    
date     : Tue, 25 Jun 2019 11:16:04 +0900    
  
committer: Michael Paquier <michael@paquier.xyz>    
date     : Tue, 25 Jun 2019 11:16:04 +0900    

Click here for diff

This could trigger valgrind failures when doing ambiguous function name  
lookups when no arguments are provided by the caller.  The problem has  
been introduced in aefeb68, so backpatch to v10.  HEAD is fine thanks to  
the refactoring done in bfb456c1.  
  
Reported-by: Alexander Lakhin  
Author: Alexander Lakhin, Michael Paquier  
Discussion: https://postgr.es/m/3d068be5-f617-a5ee-99f6-458a407bfd65@gmail.com  
Backpatch-through: 10  

M src/backend/parser/parse_func.c

Don't unset MAKEFLAGS in non-GNU Makefile.

commit   : 956611e4c480a2b7edeb2f1581e99f19f1983ca5    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Tue, 25 Jun 2019 09:29:53 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Tue, 25 Jun 2019 09:29:53 +1200    

Click here for diff

It's useful to be able to pass down options like -s and -j.  
  
Back-patch to 9.5, like commit a76200de.  
  
Discussion: https://postgr.es/m/CA%2BhUKG%2Be1M8-BbL%3DPqhTp6oO6XPO6%2Bs9WGQMLfbuZ%3DG9CtzyXg%40mail.gmail.com  

M Makefile

Further fix ALTER COLUMN TYPE's handling of indexes and index constraints.

commit   : cb8962ce8eb4bc8377d88424069794ad62080d4d    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 24 Jun 2019 16:43:05 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Mon, 24 Jun 2019 16:43:05 -0400    

Click here for diff

This patch reverts all the code changes of commit e76de8861, which turns  
out to have been seriously misguided.  We can't wait till later to compute  
the definition string for an index; we must capture that before applying  
the data type change for any column it depends on, else ruleutils.c will  
deliverr wrong/misleading results.  (This fine point was documented  
nowhere, of course.)  
  
I'd also managed to forget that ATExecAlterColumnType executes once per  
ALTER COLUMN TYPE clause, not once per statement; which resulted in the  
code being basically completely broken for any case in which multiple ALTER  
COLUMN TYPE clauses are applied to a table having non-constraint indexes  
that must be rebuilt.  Through very bad luck, none of the existing test  
cases nor the ones added by e76de8861 caught that, but of course it was  
soon found in the field.  
  
The previous patch also had an implicit assumption that if a constraint's  
index had a dependency on a table column, so would the constraint --- but  
that isn't actually true, so it didn't fix such cases.  
  
Instead of trying to delete unneeded index dependencies later, do the  
is-there-a-constraint lookup immediately on seeing an index dependency,  
and switch to remembering the constraint if so.  In the unusual case of  
multiple column dependencies for a constraint index, this will result in  
duplicate constraint lookups, but that's not that horrible compared to all  
the other work that happens here.  Besides, such cases did not work at all  
before, so it's hard to argue that they're performance-critical for anyone.  
  
Per bug #15865 from Keith Fiske.  As before, back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/15865-17940eacc8f8b081@postgresql.org  

M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql

Fix spinlock assembly code for MIPS so it works on MIPS r6.

commit   : 05399b14889e32f2714b428836c841b08d1d5258    
  
author   : Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 22 Jun 2019 20:31:50 -0400    
  
committer: Tom Lane <tgl@sss.pgh.pa.us>    
date     : Sat, 22 Jun 2019 20:31:50 -0400    

Click here for diff

Original MIPS-I processors didn't have the LL/SC instructions (nor any  
other userland synchronization primitive).  If the build toolchain  
targets that ISA variant by default, as an astonishingly large fraction  
of MIPS platforms still do, the assembler won't take LL/SC without  
coercion in the form of a ".set mips2" instruction.  But we issued that  
unconditionally, making it an ISA downgrade for chips later than MIPS2.  
That breaks things for the latest MIPS r6 ISA, which encodes these  
instructions differently.  Adjust the code so we don't change ISA level  
if it's >= 2.  
  
Note that this patch doesn't change what happens on an actual MIPS-I  
processor: either the kernel will emulate these instructions  
transparently, or you'll get a SIGILL failure.  That tradeoff seemed  
fine in 2002 when this code was added (cf 3cbe6b247), and it's even  
more so today when MIPS-I is basically extinct.  But let's add a  
comment about that.  
  
YunQiang Su (with cosmetic adjustments by me).  Back-patch to all  
supported branches.  
  
Discussion: https://postgr.es/m/15844-8f62fe7e163939b3@postgresql.org  

M src/include/storage/s_lock.h

Consolidate methods for translating a Perl path to a Windows path.

commit   : 6121ba9d18efa30d09db80708c1e222c81daff97    
  
author   : Noah Misch <noah@leadboat.com>    
date     : Fri, 21 Jun 2019 20:34:23 -0700    
  
committer: Noah Misch <noah@leadboat.com>    
date     : Fri, 21 Jun 2019 20:34:23 -0700    

Click here for diff

This fixes some TAP suites when using msys Perl and a builddir located  
in an msys mount point other than "/".  For example, builddir=/c/pg  
exhibited the problem, since /c/pg falls in mount point "/c".  
Back-patch to 9.6, where tests first started to perform such  
translations.  In back branches, offer both new and old APIs.  
  
Reviewed by Andrew Dunstan.  
  
Discussion: https://postgr.es/m/20190610045838.GA238501@rfd.leadboat.com  

M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm

Remove obsolete comments about sempahores from proc.c.

commit   : df098c371b882eeafa168501f0d20330c9a186fc    
  
author   : Thomas Munro <tmunro@postgresql.org>    
date     : Fri, 21 Jun 2019 10:57:07 +1200    
  
committer: Thomas Munro <tmunro@postgresql.org>    
date     : Fri, 21 Jun 2019 10:57:07 +1200    

Click here for diff

Commit 6753333f switched from a semaphore-based wait to a latch-based  
wait for ProcSleep()/ProcWakeup(), but left behind some stray references  
to semaphores.  
  
Back-patch to 9.5.  
  
Reviewed-by: Daniel Gustafsson, Michael Paquier  
Discussion: https://postgr.es/m/CA+hUKGLs5H6zhmgTijZ1OaJvC1sG0=AFXc1aHuce32tKiQrdEA@mail.gmail.com  

M src/backend/storage/lmgr/proc.c

Avoid spurious deadlocks when upgrading a tuple lock

commit   : 0772d8a00eb88b50ac81f379955d30660cecdeeb    
  
author   : Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 18 Jun 2019 18:23:16 -0400    
  
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>    
date     : Tue, 18 Jun 2019 18:23:16 -0400    

Click here for diff

This puts back reverted commit de87a084c0a5, with some bug fixes.  
  
When two (or more) transactions are waiting for transaction T1 to release a  
tuple-level lock, and transaction T1 upgrades its lock to a higher level, a  
spurious deadlock can be reported among the waiting transactions when T1  
finishes.  The simplest example case seems to be:  
  
T1: select id from job where name = 'a' for key share;  
Y: select id from job where name = 'a' for update; -- starts waiting for T1  
Z: select id from job where name = 'a' for key share;  
T1: update job set name = 'b' where id = 1;  
Z: update job set name = 'c' where id = 1; -- starts waiting for T1  
T1: rollback;  
  
At this point, transaction Y is rolled back on account of a deadlock: Y  
holds the heavyweight tuple lock and is waiting for the Xmax to be released,  
while Z holds part of the multixact and tries to acquire the heavyweight  
lock (per protocol) and goes to sleep; once T1 releases its part of the  
multixact, Z is awakened only to be put back to sleep on the heavyweight  
lock that Y is holding while sleeping.  Kaboom.  
  
This can be avoided by having Z skip the heavyweight lock acquisition.  As  
far as I can see, the biggest downside is that if there are multiple Z  
transactions, the order in which they resume after T1 finishes is not  
guaranteed.  
  
Backpatch to 9.6.  The patch applies cleanly on 9.5, but the new tests don't  
work there (because isolationtester is not smart enough), so I'm not going  
to risk it.  
  
Author: Oleksii Kliukin  
Discussion: https://postgr.es/m/B9C9D7CD-EB94-4635-91B6-E558ACEC0EC3@hintbits.com  
Discussion: https://postgr.es/m/2815.1560521451@sss.pgh.pa.us  

M src/backend/access/heap/README.tuplock
M src/backend/access/heap/heapam.c
A src/test/isolation/expected/tuplelock-upgrade-no-deadlock.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/tuplelock-upgrade-no-deadlock.spec