PostgreSQL 9.6.21 commit log

Stamp 9.6.21.

commit   : 43b434edefdae5607000bb044caec1cf5e0c25c4    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 8 Feb 2021 17:01:30 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 8 Feb 2021 17:01:30 -0500    

Click here for diff

M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc

Translation updates

commit   : e641bce57deb2bcbd25f04fe8f56fddaeb7757ea    
  
author   : Peter Eisentraut <[email protected]>    
date     : Mon, 8 Feb 2021 18:10:23 +0100    
  
committer: Peter Eisentraut <[email protected]>    
date     : Mon, 8 Feb 2021 18:10:23 +0100    

Click here for diff

Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git  
Source-Git-Hash: c1022f89a3531073349e8954e4f4fae64fe34552  

M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/bin/pg_rewind/po/ru.po
M src/bin/psql/po/de.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po

Release notes for 13.2, 12.6, 11.11, 10.16, 9.6.21, 9.5.25.

commit   : 625e04fa28787739b04af1b2df45a1e20c65a703    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 7 Feb 2021 15:46:38 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 7 Feb 2021 15:46:38 -0500    

Click here for diff

M doc/src/sgml/release-9.6.sgml

Revert "Propagate CTE property flags when copying a CTE list into a rule."

commit   : 6f8e393a79e4d28c4b9cbad47b417a63fb4b95fa    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 7 Feb 2021 12:54:08 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 7 Feb 2021 12:54:08 -0500    

Click here for diff

This reverts commit ed290896335414c6c069b9ccae1f3dcdd2fac6ba and  
equivalent back-branch commits.  The issue is subtler than I thought,  
and it's far from new, so just before a release deadline is no time  
to be fooling with it.  We'll consider what to do at a bit more  
leisure.  
  
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com  

M src/backend/rewrite/rewriteHandler.c

Propagate CTE property flags when copying a CTE list into a rule.

commit   : 01e3fe3275af6b6f4b246dae8b8d3c169b1e3f8c    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 6 Feb 2021 19:28:39 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 6 Feb 2021 19:28:39 -0500    

Click here for diff

rewriteRuleAction() neglected this step, although it was careful to  
propagate other similar flags such as hasSubLinks or hasRowSecurity.  
Omitting to transfer hasRecursive is just cosmetic at the moment,  
but omitting hasModifyingCTE is a live bug, since the executor  
certainly looks at that.  
  
The proposed test case only fails back to v10, but since the executor  
examines hasModifyingCTE in 9.x as well, I suspect that a test case  
could be devised that fails in older branches.  Given the nearness  
of the release deadline, though, I'm not going to spend time looking  
for a better test.  
  
Report and patch by Greg Nancarrow, cosmetic changes by me  
  
Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com  

M src/backend/rewrite/rewriteHandler.c

Disallow converting an inheritance child table to a view.

commit   : 7736ab05fe839ee09386701ca1c992834f9af56a    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 6 Feb 2021 15:17:02 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 6 Feb 2021 15:17:02 -0500    

Click here for diff

Generally, members of inheritance trees must be plain tables (or,  
in more recent versions, foreign tables).  ALTER TABLE INHERIT  
rejects creating an inheritance relationship that has a view at  
either end.  When DefineQueryRewrite attempts to convert a relation  
to a view, it already had checks prohibiting doing so for partitioning  
parents or children as well as traditional-inheritance parents ...  
but it neglected to check that a traditional-inheritance child wasn't  
being converted.  Since the planner assumes that any inheritance  
child is a table, this led to making plans that tried to do a physical  
scan on a view, causing failures (or even crashes, in recent versions).  
  
One could imagine trying to support such a case by expanding the view  
normally, but since the rewriter runs before the planner does  
inheritance expansion, it would take some very fundamental refactoring  
to make that possible.  There are probably a lot of other parts of the  
system that don't cope well with such a situation, too.  For now,  
just forbid it.  
  
Per bug #16856 from Yang Lin.  Back-patch to all supported branches.  
(In versions before v10, this includes back-patching the portion of  
commit 501ed02cf that added has_superclass().  Perhaps the lack of  
that infrastructure partially explains the missing check.)  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/catalog/pg_inherits.c
M src/backend/rewrite/rewriteDefine.c
M src/include/catalog/pg_inherits_fn.h
M src/test/regress/expected/rules.out
M src/test/regress/sql/rules.sql

Fix backslash-escaping multibyte chars in COPY FROM.

commit   : e152ccc7fd5932cfa9371a7c4cb7ca9a7733c5ab    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Fri, 5 Feb 2021 11:14:56 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Fri, 5 Feb 2021 11:14:56 +0200    

Click here for diff

If a multi-byte character is escaped with a backslash in TEXT mode input,  
and the encoding is one of the client-only encodings where the bytes after  
the first one can have an ASCII byte "embedded" in the char, we didn't  
skip the character correctly. After a backslash, we only skipped the first  
byte of the next character, so if it was a multi-byte character, we would  
try to process its second byte as if it was a separate character. If it  
was one of the characters with special meaning, like '\n', '\r', or  
another '\\', that would cause trouble.  
  
One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.  
That's supposed to be [backslash][two-byte character][.][f][o][o], but  
because the second byte of the two-byte character is 0x5c, we incorrectly  
treat it as another backslash. And because the next character is a dot, we  
parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"  
error.  
  
Backpatch to all supported versions.  
  
Reviewed-by: John Naylor, Kyotaro Horiguchi  
Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi  

M src/backend/commands/copy.c

Fix ancient memory leak in contrib/auto_explain.

commit   : 608cf2bfd9ec33e741ba6a1c07c78c7e8f1cb171    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 2 Feb 2021 13:49:08 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 2 Feb 2021 13:49:08 -0500    

Click here for diff

The ExecutorEnd hook is invoked in a context that could be quite  
long-lived, not the executor's own per-query context as I think  
we were sort of assuming.  Thus, any cruft generated while producing  
the EXPLAIN output could accumulate over multiple queries.  This can  
result in spectacular leakage if log_nested_statements is on, and  
even without that I'm surprised nobody complained before.  
  
To fix, just switch into the executor's context so that anything we  
allocate will be released when standard_ExecutorEnd frees the executor  
state.  We might as well nuke the code's retail pfree of the explain  
output string, too; that's laughably inadequate to the need.  
  
Japin Li, per report from Jeff Janes.  This bug is old, so  
back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/CAMkU=1wCVtbeRn0s9gt12KwQ7PLXovbpM8eg25SYocKW3BT4hg@mail.gmail.com  

M contrib/auto_explain/auto_explain.c

Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions.

commit   : d683d6528dba56669f24e5fac85c7692fc9ffd43    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 30 Jan 2021 00:00:27 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 30 Jan 2021 00:00:27 -0800    

Click here for diff

In a cluster having used CREATE INDEX CONCURRENTLY while having enabled  
prepared transactions, queries that use the resulting index can silently  
fail to find rows.  Fix this for future CREATE INDEX CONCURRENTLY by  
making it wait for prepared transactions like it waits for ordinary  
transactions.  This expands the VirtualTransactionId structure domain to  
admit prepared transactions.  It may be necessary to reindex to recover  
from past occurrences.  Back-patch to 9.5 (all supported versions).  
  
Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael  
Paquier.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lock.c
M src/include/storage/lock.h
M src/test/isolation/Makefile
M src/test/isolation/README
A src/test/isolation/expected/prepared-transactions-cic.out
A src/test/isolation/specs/prepared-transactions-cic.spec

Silence another gcc 11 warning.

commit   : ea3164aae4b2d2edfb1dc3bd450bfed86d3266ea    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 28 Jan 2021 17:18:23 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 28 Jan 2021 17:18:23 -0500    

Click here for diff

Per buildfarm and local experimentation, bleeding-edge gcc isn't  
convinced that the MemSet in reorder_function_arguments() is safe.  
Shut it up by adding an explicit check that pronargs isn't negative,  
and by changing MemSet to memset.  (It appears that either change is  
enough to quiet the warning at -O2, but let's do both to be sure.)  

M src/backend/optimizer/util/clauses.c

Make ecpg's rjulmdy() and rmdyjul() agree with their declarations.

commit   : edb39e321049c7a12abc6bb78df5ee2ccf2e6437    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 28 Jan 2021 11:17:13 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 28 Jan 2021 11:17:13 -0500    

Click here for diff

We had "short *mdy" in the extern declarations, but "short mdy[3]"  
in the actual function definitions.  Per C99 these are equivalent,  
but recent versions of gcc have started to issue warnings about  
the inconsistency.  Clean it up before the warnings get any more  
widespread.  
  
Back-patch, in case anyone wants to build older PG versions with  
bleeding-edge compilers.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/interfaces/ecpg/compatlib/informix.c

pgbench: Remove dead code

commit   : 4162f91d1c201f5f76c34910e35d32aeb673b219    
  
author   : Alvaro Herrera <[email protected]>    
date     : Thu, 28 Jan 2021 12:50:40 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Thu, 28 Jan 2021 12:50:40 -0300    

Click here for diff

doConnect() never returns connections in state CONNECTION_BAD, so  
checking for that is pointless.  Remove the code that does.  
  
This code has been dead since ba708ea3dc84, 20 years ago.  
  
Discussion: https://postgr.es/m/[email protected]  
Reviewed-by: Tom Lane <[email protected]>  

M src/bin/pgbench/pgbench.c

Report the true database name on connection errors

commit   : bcae842b962a485524a41575810caf231400e3db    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 26 Jan 2021 16:42:13 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 26 Jan 2021 16:42:13 -0300    

Click here for diff

When reporting connection errors, we might show a database name in the  
message that's not the one we actually tried to connect to, if the  
database was taken from libpq defaults instead of from user parameters.  
Fix such error messages to use PQdb(), which reports the correct name.  
  
(But, per commit 2930c05634bc, make sure not to try to print NULL.)  
  
Apply to branches 9.5 through 13.  Branch master has already been  
changed differently by commit 58cd8dca3de0.  
  
Reported-by: Robert Haas <[email protected]>  
Discussion: https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8VcRBjmj-b9UDqRjw@mail.gmail.com  

M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pgbench/pgbench.c

Code review for psql's helpSQL() function.

commit   : 2c2e134b735f893d7840278a54d92c8def61819f    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 26 Jan 2021 13:04:52 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 26 Jan 2021 13:04:52 -0500    

Click here for diff

The loops to identify word boundaries could access past the end of  
the input string.  Likely that would never result in an actual  
crash, but it makes valgrind unhappy.  
  
The logic to try different numbers of words didn't work when the  
input has two words but we only have a match to the first, eg  
"\h with select".  (We must "continue" the pass loop, not "break".)  
  
The logic to compute nl_count was bizarrely managed, and in at  
least two code paths could end up calling PageOutput with  
nl_count = 0, resulting in failing to paginate output that should  
have been fed to the pager.  Also, in v12 and up, the nl_count  
calculation hadn't been updated to account for the addition of a URL.  
  
The PQExpBuffer holding the command syntax details wasn't freed,  
resulting in a session-lifespan memory leak.  
  
While here, improve some comments, choose a more descriptive name  
for a variable, fix inconsistent datatype choice for another variable.  
  
Per bug #16837 from Alexander Lakhin.  This code is very old,  
so back-patch to all supported branches.  
  
Kyotaro Horiguchi and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

M src/bin/psql/help.c

Fix broken ruleutils support for function TRANSFORM clauses.

commit   : 57a7d6f4956915de80084b687d4e0385c01ddd16    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 25 Jan 2021 13:03:12 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 25 Jan 2021 13:03:12 -0500    

Click here for diff

I chanced to notice that this dumped core due to a faulty Assert.  
To add insult to injury, the output has been misformatted since v11.  
Obviously we need some regression testing here.  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/hstore_plpython/expected/hstore_plpython.out
M contrib/hstore_plpython/sql/hstore_plpython.sql
M src/backend/utils/fmgr/funcapi.c

Fix hypothetical bug in heap backward scans

commit   : 317de9c45d88dd67718b4e411e2918ffbc96f5ae    
  
author   : David Rowley <[email protected]>    
date     : Mon, 25 Jan 2021 20:01:00 +1300    
  
committer: David Rowley <[email protected]>    
date     : Mon, 25 Jan 2021 20:01:00 +1300    

Click here for diff

Both heapgettup() and heapgettup_pagemode() incorrectly set the first page  
to scan in a backward scan in which the number of pages to scan was  
specified by heap_setscanlimits().  The code incorrectly started the scan  
at the end of the relation when startBlk was 0, or otherwise at  
startBlk - 1, neither of which is correct when only scanning a subset of  
pages.  
  
The fix here checks if heap_setscanlimits() has changed the number of  
pages to scan and if so we set the first page to scan as the final page in  
the specified range during backward scans.  
  
Proper adjustment of this code was forgotten when heap_setscanlimits() was  
added in 7516f5259 back in 9.5.  However, practice, nowhere in core code  
performs backward scans after having used heap_setscanlimits(), yet, it is  
possible an extension uses the heap functions in this way, hence  
backpatch.  
  
An upcoming patch does use heap_setscanlimits() with backward scans, so  
this must be fixed before that can go in.  
  
Author: David Rowley  
Discussion: https://postgr.es/m/CAApHDvpGc9h0_oVD2CtgBcxCS1N-qDYZSeBRnUh+0CWJA9cMaA@mail.gmail.com  
Backpatch-through: 9.5, all supported versions  

M src/backend/access/heap/heapam.c

Update time zone data files to tzdata release 2021a.

commit   : 7e078675168befe07b40b472a315de5423c2600a    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 24 Jan 2021 16:29:48 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 24 Jan 2021 16:29:48 -0500    

Click here for diff

DST law changes in Russia (Volgograd zone) and South Sudan.  
Historical corrections for Australia, Bahamas, Belize, Bermuda,  
Ghana, Israel, Kenya, Nigeria, Palestine, Seychelles, and Vanuatu.  
Notably, the Australia/Currie zone has been corrected to the point  
where it is identical to Australia/Hobart.  

M src/timezone/data/tzdata.zi

Make check-prepared-txns use temp-install binaries.

commit   : cedbca85d28ec88cf763146c8c9653750a91da15    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 23 Jan 2021 14:09:59 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 23 Jan 2021 14:09:59 -0800    

Click here for diff

It tested the installed binaries.  This fixes 9.6 and 9.5 to follow  
later versions and "make -C src/test/isolation check" in this respect.  

M src/test/isolation/Makefile

Doc: improve directions for building on macOS.

commit   : 4dd2c3b1c3e940b09c3b26d30cd37c66c4deda4d    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 22 Jan 2021 18:58:25 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 22 Jan 2021 18:58:25 -0500    

Click here for diff

In light of recent discussions, we should instruct people to  
install Apple's command line tools; installing Xcode is secondary.  
  
Also, fix sample command for finding out the default sysroot,  
as we now know that the command originally recommended can give  
a result that doesn't match your OS version.  
  
Also document the workaround to use if you really don't want  
configure to select a sysroot at all.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/installation.sgml

Further tweaking of PG_SYSROOT heuristics for macOS.

commit   : cbcf7b130f5e53e405c5b366e78a4b3d8ff5f642    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 20 Jan 2021 12:07:23 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 20 Jan 2021 12:07:23 -0500    

Click here for diff

It emerges that in some phases of the moon (perhaps to do with  
directory entry order?), xcrun will report that the SDK path is  
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk  
which is normally a symlink to a version-numbered sibling directory.  
Our heuristic to skip non-version-numbered pathnames was rejecting  
that, which is the wrong thing to do.  We'd still like to end up  
with a version-numbered PG_SYSROOT value, but we can have that by  
dereferencing the symlink.  
  
Like the previous fix, back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/template/darwin

Fix ALTER DEFAULT PRIVILEGES with duplicated objects

commit   : 7dc3be9df86cfd2515e76c078347dccc9a929d0b    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 20 Jan 2021 11:39:31 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 20 Jan 2021 11:39:31 +0900    

Click here for diff

Specifying duplicated objects in this command would lead to unique  
constraint violations in pg_default_acl or "tuple already updated by  
self" errors.  Similarly to GRANT/REVOKE, increment the command ID after  
each subcommand processing to allow this case to work transparently.  
  
A regression test is added by tweaking one of the existing queries of  
privileges.sql to stress this case.  
  
Reported-by: Andrus  
Author: Michael Paquier  
Reviewed-by: Álvaro Herrera  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.5  

M src/backend/catalog/aclchk.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql

Remove faulty support for MergeAppend plan with WHERE CURRENT OF.

commit   : fe8edbb8267adf24ba3b392ac6229b96c4287f93    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 19 Jan 2021 13:25:33 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 19 Jan 2021 13:25:33 -0500    

Click here for diff

Somebody extended search_plan_tree() to treat MergeAppend exactly  
like Append, which is 100% wrong, because unlike Append we can't  
assume that only one input node is actively returning tuples.  
Hence a cursor using a MergeAppend across a UNION ALL or inheritance  
tree could falsely match a WHERE CURRENT OF query at a row that  
isn't actually the cursor's current output row, but coincidentally  
has the same TID (in a different table) as the current output row.  
  
Delete the faulty code; this means that such a case will now return  
an error like 'cursor "foo" is not a simply updatable scan of table  
"bar"', instead of silently misbehaving.  Users should not find that  
surprising though, as the same cursor query could have failed that way  
already depending on the chosen plan.  (It would fail like that if the  
sort were done with an explicit Sort node instead of MergeAppend.)  
  
Expand the clearly-inadequate commentary to be more explicit about  
what this code is doing, in hopes of forestalling future mistakes.  
  
It's been like this for awhile, so back-patch to all supported  
branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/executor/execCurrent.c

Avoid crash with WHERE CURRENT OF and a custom scan plan.

commit   : ffbf1746354a44a16d4d525d62226aa65fdb4e69    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 18 Jan 2021 18:32:30 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 18 Jan 2021 18:32:30 -0500    

Click here for diff

execCurrent.c's search_plan_tree() assumed that ForeignScanStates  
and CustomScanStates necessarily have a valid ss_currentRelation.  
This is demonstrably untrue for postgres_fdw's remote join and  
remote aggregation plans, and non-leaf custom scans might not have  
an identifiable scan relation either.  Avoid crashing by ignoring  
such nodes when the field is null.  
  
This solution will lead to errors like 'cursor "foo" is not a  
simply updatable scan of table "bar"' in cases where maybe we  
could have allowed WHERE CURRENT OF to work.  That's not an issue  
for postgres_fdw's usages, since joins or aggregations would render  
WHERE CURRENT OF invalid anyway.  But an otherwise-transparent  
upper level custom scan node might find this annoying.  When and if  
someone cares to expend work on such a scenario, we could invent a  
custom-scan-provider callback to determine what's safe.  
  
Report and patch by David Geier, commentary by me.  It's been like  
this for awhile, so back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/executor/execCurrent.c

Fix pg_dump for GRANT OPTION among initial privileges.

commit   : ad2b7c9bb061e85d1afe0debb916e00ca8cf8d5b    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 16 Jan 2021 12:21:35 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 16 Jan 2021 12:21:35 -0800    

Click here for diff

The context is an object that no longer bears some aclitem that it bore  
initially.  (A user issued REVOKE or GRANT statements upon the object.)  
pg_dump is forming SQL to reproduce the object ACL.  Since initdb  
creates no ACL bearing GRANT OPTION, reaching this bug requires an  
extension where the creation script establishes such an ACL.  No PGXN  
extension does that.  If an installation did reach the bug, pg_dump  
would have omitted a semicolon, causing a REVOKE and the next SQL  
statement to fail.  Separately, since the affected code exists to  
eliminate an entire aclitem, it wants plain REVOKE, not REVOKE GRANT  
OPTION FOR.  Back-patch to 9.6, where commit  
23f34fa4ba358671adab16773e79c17c92cbc870 first appeared.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/bin/pg_dump/dumputils.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql

Prevent excess SimpleLruTruncate() deletion.

commit   : 1a31d8c52db4220f162ed3c4812569843daf950e    
  
author   : Noah Misch <[email protected]>    
date     : Sat, 16 Jan 2021 12:21:35 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Sat, 16 Jan 2021 12:21:35 -0800    

Click here for diff

Every core SLRU wraps around.  With the exception of pg_notify, the wrap  
point can fall in the middle of a page.  Account for this in the  
PagePrecedes callback specification and in SimpleLruTruncate()'s use of  
said callback.  Update each callback implementation to fit the new  
specification.  This changes SerialPagePrecedesLogically() from the  
style of asyncQueuePagePrecedes() to the style of CLOGPagePrecedes().  
(Whereas pg_clog and pg_serial share a key space, pg_serial is nothing  
like pg_notify.)  The bug fixed here has the same symptoms and user  
followup steps as 592a589a04bd456410b853d86bd05faa9432cbbb.  Back-patch  
to 9.5 (all supported versions).  
  
Reviewed by Andrey Borodin and (in earlier versions) by Tom Lane.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/storage/lmgr/predicate.c
M src/include/access/slru.h

Improve our heuristic for selecting PG_SYSROOT on macOS.

commit   : fc6d08b27a3395f9a1b30a28f60ad60a01d1523b    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 15 Jan 2021 11:28:51 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 15 Jan 2021 11:28:51 -0500    

Click here for diff

In cases where Xcode is newer than the underlying macOS version,  
asking xcodebuild for the SDK path will produce a pointer to the  
SDK shipped with Xcode, which may end up building code that does  
not work on the underlying macOS version.  It appears that in  
such cases, xcodebuild's answer also fails to match the default  
behavior of Apple's compiler: assuming one has installed Xcode's  
"command line tools", there will be an SDK for the OS's own version  
in /Library/Developer/CommandLineTools, and the compiler will  
default to using that.  This is all pretty poorly documented,  
but experimentation suggests that "xcrun --show-sdk-path" gives  
the sysroot path that the compiler is actually using, at least  
in some cases.  Hence, try that first, but revert to xcodebuild  
if xcrun fails (in very old Xcode, it is missing or lacks the  
--show-sdk-path switch).  
  
Also, "xcrun --show-sdk-path" may give a path that is valid but lacks  
any OS version identifier.  We don't really want that, since most  
of the motivation for wiring -isysroot into the build flags at all  
is to ensure that all parts of a PG installation are built against  
the same SDK, even when considering extensions built later and/or on  
a different machine.  Insist on finding "N.N" in the directory name  
before accepting the result.  (Adding "--sdk macosx" to the xcrun  
call seems to produce the same answer as xcodebuild, but usually  
more quickly because it's cached, so we also try that as a fallback.)  
  
The core reason why we don't want to use Xcode's default SDK in cases  
like this is that Apple's technology for introducing new syscalls  
does not play nice with Autoconf: for example, configure will think  
that preadv/pwritev exist when using a Big Sur SDK, even when building  
on an older macOS version where they don't exist.  It'd be nice to  
have a better solution to that problem, but this patch doesn't attempt  
to fix that.  
  
Per report from Sergey Shinderuk.  Back-patch to all supported versions.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/template/darwin

Doc: clarify behavior of back-half options in pg_dump.

commit   : 7343a155925107aec07676ab5c5247b4c16f4bdc    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 13 Jan 2021 13:30:04 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 13 Jan 2021 13:30:04 -0500    

Click here for diff

Options that change how the archive data is converted to SQL text  
are ignored when dumping to archive formats.  The documentation  
previously said "not meaningful", which is not helpful.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/ref/pg_dump.sgml

Fix memory leak in SnapBuildSerialize.

commit   : f2b268ee0626286d15a1eaa78fe1704811d51cd4    
  
author   : Amit Kapila <[email protected]>    
date     : Wed, 13 Jan 2021 08:31:45 +0530    
  
committer: Amit Kapila <[email protected]>    
date     : Wed, 13 Jan 2021 08:31:45 +0530    

Click here for diff

The memory for the snapshot was leaked while serializing it to disk during  
logical decoding. This memory will be freed only once walsender stops  
streaming the changes. This can lead to a huge memory increase when master  
logs Standby Snapshot too frequently say when the user is trying to create  
many replication slots.  
  
Reported-by: [email protected]  
Diagnosed-by: [email protected]  
Author: Amit Kapila  
Backpatch-through: 9.5  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/replication/logical/snapbuild.c

Fix thinko in comment

commit   : 594a7dcd674cdc1596d74edca1b80c41679ab499    
  
author   : Alvaro Herrera <[email protected]>    
date     : Tue, 12 Jan 2021 11:48:45 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Tue, 12 Jan 2021 11:48:45 -0300    

Click here for diff

This comment has been wrong since its introduction in commit  
2c03216d8311.  
  
Author: Masahiko Sawada <[email protected]>  
Discussion: https://postgr.es/m/CAD21AoAzz6qipFJBbGEaHmyWxvvNDp8httbwLR9tUQWaTjUs2Q@mail.gmail.com  

M src/backend/access/transam/xlogreader.c

doc: expand description of how non-SELECT queries are processed

commit   : e8973d8bdd178c3899f8d44aaeef4ac3492b95e1    
  
author   : Bruce Momjian <[email protected]>    
date     : Sat, 9 Jan 2021 12:11:15 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Sat, 9 Jan 2021 12:11:15 -0500    

Click here for diff

The previous description of how the executor processes non-SELECT  
queries was very dense, causing lack of clarity.  This expanded text  
spells it out more simply.  
  
Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

M doc/src/sgml/arch-dev.sgml

Fix ancient bug in parsing of BRE-mode regular expressions.

commit   : 085a1cfb3d5fd767eb66e7c5bd7ed7818a7d717f    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 8 Jan 2021 12:16:00 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 8 Jan 2021 12:16:00 -0500    

Click here for diff

brenext(), when parsing a '*' quantifier, forgot to return any "value"  
for the token; per the equivalent case in next(), it should return  
value 1 to indicate that greedy rather than non-greedy behavior is  
wanted.  The result is that the compiled regexp could behave like 'x*?'  
rather than the intended 'x*', if we were unlucky enough to have  
a zero in v->nextvalue at this point.  That seems to happen with some  
reliability if we have '.*' at the beginning of a BRE-mode regexp,  
although that depends on the initial contents of a stack-allocated  
struct, so it's not guaranteed to fail.  
  
Found by Alexander Lakhin using valgrind testing.  This bug seems  
to be aboriginal in Spencer's code, so back-patch all the way.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/regex/regc_lex.c

Further second thoughts about idle_session_timeout patch.

commit   : 388ec04f14366d5df0c9b7958c5d9c09d4327f3a    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 7 Jan 2021 11:45:09 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 7 Jan 2021 11:45:09 -0500    

Click here for diff

On reflection, the order of operations in PostgresMain() is wrong.  
These timeouts ought to be shut down before, not after, we do the  
post-command-read CHECK_FOR_INTERRUPTS, to guarantee that any  
timeout error will be detected there rather than at some ill-defined  
later point (possibly after having wasted a lot of work).  
  
This is really an error in the original idle_in_transaction_timeout  
patch, so back-patch to 9.6 where that was introduced.  

M src/backend/tcop/postgres.c

doc: Remove reference to recovery params for divergence lookup in pg_rewind

commit   : dd312101efc1b0836c00230951a135edecbdb0e5    
  
author   : Michael Paquier <[email protected]>    
date     : Thu, 7 Jan 2021 20:50:56 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Thu, 7 Jan 2021 20:50:56 +0900    

Click here for diff

The documentation of pg_rewind mentioned the use of restore_command and  
primary_conninfo as options available at startup to fetch missing WAL  
segments that could be used to find the point of divergence for the  
rewind.  This is confusing because when finding the point of divergence  
the target cluster is offline, so this option is not available.  
  
Issue introduced by 878bd9a, so backpatch down to 9.6.  The  
documentation of 13 and HEAD was already right as this sentence has been  
changed by a7e8ece when introducing -c/--restore-target-wal.  
  
Reported-by: Amine Tengilimoglu  
Discussion: https://postgr.es/m/CADTdw-w_0MP=iQrfizeU4QU5fcZb+w8P_oPeLL+WznWf0kbn3w@mail.gmail.com  
Backpatch-through: 9.6  

M doc/src/sgml/ref/pg_rewind.sgml

Detect the deadlocks between backends and the startup process.

commit   : 0307b98d8f5f464de622853be77a28ab6a8e1038    
  
author   : Fujii Masao <[email protected]>    
date     : Wed, 6 Jan 2021 12:33:28 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Wed, 6 Jan 2021 12:33:28 +0900    

Click here for diff

The deadlocks that the recovery conflict on lock is involved in can  
happen between hot-standby backends and the startup process.  
If a backend takes an access exclusive lock on the table and which  
finally triggers the deadlock, that deadlock can be detected  
as expected. On the other hand, previously, if the startup process  
took an access exclusive lock and which finally triggered the deadlock,  
that deadlock could not be detected and could remain even after  
deadlock_timeout passed. This is a bug.  
  
The cause of this bug was that the code for handling the recovery  
conflict on lock didn't take care of deadlock case at all. It assumed  
that deadlocks involving the startup process and backends were able  
to be detected by the deadlock detector invoked within backends.  
But this assumption was incorrect. The startup process also should  
have invoked the deadlock detector if necessary.  
  
To fix this bug, this commit makes the startup process invoke  
the deadlock detector if deadlock_timeout is reached while handling  
the recovery conflict on lock. Specifically, in that case, the startup  
process requests all the backends holding the conflicting locks to  
check themselves for deadlocks.  
  
Back-patch to v9.6. v9.5 has also this bug, but per discussion we decided  
not to back-patch the fix to v9.5. Because v9.5 doesn't have some  
infrastructure codes (e.g., 37c54863cf) that this bug fix patch depends on.  
We can apply those codes for the back-patch, but since the next minor  
version release is the final one for v9.5, it's risky to do that. If we  
unexpectedly introduce new bug to v9.5 by the back-patch, there is no  
chance to fix that. We determined that the back-patch to v9.5 would give  
more risk than gain.  
  
Author: Fujii Masao  
Reviewed-by: Bertrand Drouvot, Masahiko Sawada, Kyotaro Horiguchi  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/storage/procarray.h

doc: improve NLS instruction wording

commit   : 72c2187eb7df51f73cdbc034f1ddf31f01611155    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 5 Jan 2021 14:26:37 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 5 Jan 2021 14:26:37 -0500    

Click here for diff

Reported-by: "Tang, Haiying"  
  
Discussion: https://postgr.es/m/bbbccf7a3c2d436e85d45869d612fd6b@G08CNEXMBPEKD05.g08.fujitsu.local  
  
Author: "Tang, Haiying"  
  
Backpatch-through: 9.5  

M doc/src/sgml/nls.sgml

Add an explicit cast to double when using fabs().

commit   : f0b9bada56f801fc9e70befb7206a72d5444eb8e    
  
author   : Dean Rasheed <[email protected]>    
date     : Tue, 5 Jan 2021 11:45:17 +0000    
  
committer: Dean Rasheed <[email protected]>    
date     : Tue, 5 Jan 2021 11:45:17 +0000    

Click here for diff

Commit bc43b7c2c0 used fabs() directly on an int variable, which  
apparently requires an explicit cast on some platforms.  
  
Per buildfarm.  

M src/backend/utils/adt/numeric.c

Fix numeric_power() when the exponent is INT_MIN.

commit   : 9a299dff25a5f945cc2ee0c00b705857f966e169    
  
author   : Dean Rasheed <[email protected]>    
date     : Tue, 5 Jan 2021 11:02:46 +0000    
  
committer: Dean Rasheed <[email protected]>    
date     : Tue, 5 Jan 2021 11:02:46 +0000    

Click here for diff

In power_var_int(), the computation of the number of significant  
digits to use in the computation used log(Abs(exp)), which isn't safe  
because Abs(exp) returns INT_MIN when exp is INT_MIN. Use fabs()  
instead of Abs(), so that the exponent is cast to a double before the  
absolute value is taken.  
  
Back-patch to 9.6, where this was introduced (by 7d9a4737c2).  
  
Discussion: https://postgr.es/m/CAEZATCVd6pMkz=BrZEgBKyqqJrt2xghr=fNc8+Z=5xC6cgWrWA@mail.gmail.com  

M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql

commit   : 0709b644239ea5beeb21a53220c5b8eceae9a317    
  
author   : Bruce Momjian <[email protected]>    
date     : Sat, 2 Jan 2021 13:06:24 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Sat, 2 Jan 2021 13:06:24 -0500    

Click here for diff

Backpatch-through: 9.5  

M COPYRIGHT
M doc/src/sgml/legal.sgml

Doc: improve explanation of EXTRACT(EPOCH) for timestamp without tz.

commit   : 1cfdeda1bd1fbc71cea497ee9acb3813e9d75907    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 1 Jan 2021 15:51:09 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 1 Jan 2021 15:51:09 -0500    

Click here for diff

Try to be clearer about what computation is actually happening here.  
  
Per bug #16797 from Dana Burd.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/func.sgml

Doc: spell out comparison behaviors for the date/time types.

commit   : d56ea24166e981432786dc0be37ca7993e5c210d    
  
author   : Tom Lane <[email protected]>    
date     : Wed, 30 Dec 2020 17:48:43 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Wed, 30 Dec 2020 17:48:43 -0500    

Click here for diff

The behavior of cross-type comparisons among date/time data types was  
not really explained anywhere.  You could probably infer it if you  
recognized the applicability of comments elsewhere about datatype  
conversions, but it seems worthy of explicit documentation.  
  
Per bug #16797 from Dana Burd.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/func.sgml

doc: Improve some grammar and sentences

commit   : b25c106df71fc317e65814e4f301afa05b7b2304    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 29 Dec 2020 18:19:25 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 29 Dec 2020 18:19:25 +0900    

Click here for diff

90fbf7c has taken care of that for HEAD.  This includes the portion of  
the fixes that applies to the documentation, where needed depending on  
the branch.  
  
Author: Justin Pryzby  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.5  

M doc/src/sgml/ref/pg_dump.sgml

Fix inconsistent code with shared invalidations of snapshots

commit   : 7bf20c1b8a14022c59631a2b1ce6d1e41e9494ad    
  
author   : Michael Paquier <[email protected]>    
date     : Mon, 28 Dec 2020 22:17:16 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Mon, 28 Dec 2020 22:17:16 +0900    

Click here for diff

The code in charge of processing a single invalidation message has been  
using since 568d413 the structure for relation mapping messages.  This  
had fortunately no consequence as both locate the database ID at the  
same location, but it could become a problem in the future if this area  
of the code changes.  
  
Author: Konstantin Knizhnik  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.5  

M src/backend/utils/cache/inval.c

postgres_fdw: Fix connection leak.

commit   : 03b7a1ee7b7ff037156c2357eaf545e515ad6455    
  
author   : Fujii Masao <[email protected]>    
date     : Mon, 28 Dec 2020 20:00:15 +0900    
  
committer: Fujii Masao <[email protected]>    
date     : Mon, 28 Dec 2020 20:00:15 +0900    

Click here for diff

In postgres_fdw, the cached connections to foreign servers will not be  
closed until the local session exits if the user mappings or foreign servers  
that those connections depend on are dropped. Those connections can be  
leaked.  
  
To fix that connection leak issue, after a change to a pg_foreign_server  
or pg_user_mapping catalog entry, this commit makes postgres_fdw close  
the connections depending on that entry immediately if current  
transaction has not used those connections yet. Otherwise, mark those  
connections as invalid and then close them at the end of current transaction,  
since they cannot be closed in the midst of the transaction using them.  
Closed connections will be remade at the next opportunity if necessary.  
  
Back-patch to all supported branches.  
  
Author: Bharath Rupireddy  
Reviewed-by: Zhihong Yu, Zhijie Hou, Fujii Masao  
Discussion: https://postgr.es/m/CALj2ACVNcGH_6qLY-4_tXz8JLvA+4yeBThRfxMz7Oxbk1aHcpQ@mail.gmail.com  

M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql

Fix back-patch of "Invalidate acl.c caches when pg_authid changes."

commit   : 3f920e876bdc045959ebb82e44245cb839da0df4    
  
author   : Noah Misch <[email protected]>    
date     : Fri, 25 Dec 2020 11:02:56 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Fri, 25 Dec 2020 11:02:56 -0800    

Click here for diff

Test script role names and error messages differed in v10, 9.6 and 9.5.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql

Invalidate acl.c caches when pg_authid changes.

commit   : b81d3791ed8f52bf2507c306bda922dc272f20ce    
  
author   : Noah Misch <[email protected]>    
date     : Fri, 25 Dec 2020 10:41:59 -0800    
  
committer: Noah Misch <[email protected]>    
date     : Fri, 25 Dec 2020 10:41:59 -0800    

Click here for diff

This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as  
quickly as they have been reflecting "GRANT role_name".  Back-patch to  
9.5 (all supported versions).  
  
Reviewed by Nathan Bossart.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql

Fix portability issues with parsing of recovery_target_xid

commit   : 0b54a80a38c79e2fda049c1a2f43de6ef7d45d7a    
  
author   : Michael Paquier <[email protected]>    
date     : Wed, 23 Dec 2020 12:51:51 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Wed, 23 Dec 2020 12:51:51 +0900    

Click here for diff

The parsing of this parameter has been using strtoul(), which is not  
portable across platforms.  On most Unix platforms, unsigned long has a  
size of 64 bits, while on Windows it is 32 bits.  It is common in  
recovery scenarios to rely on the output of txid_current() or even the  
newer pg_current_xact_id() to get a transaction ID for setting up  
recovery_target_xid.  The value returned by those functions includes the  
epoch in the computed result, which would cause strtoul() to fail where  
unsigned long has a size of 32 bits once the epoch is incremented.  
  
WAL records and 2PC data include only information about 32-bit XIDs and  
it is not possible to have XIDs across more than one epoch, so  
discarding the high bits from the transaction ID set has no impact on  
recovery.  On the contrary, the use of strtoul() prevents a consistent  
behavior across platforms depending on the size of unsigned long.  
  
This commit changes the parsing of recovery_target_xid to use  
pg_strtouint64() instead, available down to 9.6.  There is one TAP test  
stressing recovery with recovery_target_xid, where a tweak based on  
pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets  
tested, as per an idea from Alexander Lakhin.  
  
Reported-by: Alexander Lakhin  
Discussion: https://postgr.es/m/[email protected]  
Backpatch-through: 9.6  

M src/backend/access/transam/xlog.c
M src/test/recovery/t/003_recovery_targets.pl

Remove "invalid concatenation of jsonb objects" error case.

commit   : 1d5f3f976b26271b51a619f02436ec5c263db9f3    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 21 Dec 2020 13:11:30 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 21 Dec 2020 13:11:30 -0500    

Click here for diff

The jsonb || jsonb operator arbitrarily rejected certain combinations  
of scalar and non-scalar inputs, while being willing to concatenate  
other combinations.  This was of course quite undocumented.  Rather  
than trying to document it, let's just remove the restriction,  
creating a uniform rule that unless we are handling an object-to-object  
concatenation, non-array inputs are converted to one-element arrays,  
resulting in an array-to-array concatenation.  (This does not change  
the behavior for any case that didn't throw an error before.)  
  
Per complaint from Joel Jacobson.  Back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/func.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql

Doc: fix description of how to use src/tutorial files.

commit   : c336e90b266bb92a7c782eaefbeb038522e41c4b    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 20 Dec 2020 15:28:22 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 20 Dec 2020 15:28:22 -0500    

Click here for diff

The separate "cd" command before invoking psql made sense (or at least  
I thought so) when it was added in commit ed1939332.  But 4e3a61635  
removed the supporting text that explained when to use it, making it  
just confusing.  So drop it.  
  
Also switch from four-dot to three-dot filler for the unsupplied  
part of the path, since at least one person has read the four-dot  
filler as a typo for "../..".  And fix these/those inconsistency.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/query.sgml

Doc: improve description of pgbench script weights.

commit   : c7a0b94c452826c6c87ce670ff252b8623088741    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 20 Dec 2020 13:37:25 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 20 Dec 2020 13:37:25 -0500    

Click here for diff

Point out the workaround to be used if you want to write a script  
file name that includes "@".  Clean up the text a little.  
  
Fabien Coelho, additional wordsmithing by me  
  
Discussion: https://postgr.es/m/1c4e81550d214741827a03292222db8d@G08CNEXMBPEKD06.g08.fujitsu.local  

M doc/src/sgml/ref/pgbench.sgml

Avoid memcpy() with same source and destination during relmapper init.

commit   : 48786740094ca527d381587ab909c23ca6d5a855    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 18 Dec 2020 15:46:44 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 18 Dec 2020 15:46:44 -0500    

Click here for diff

A narrow reading of the C standard says that memcpy(x,x,n) is undefined,  
although it's hard to envision an implementation that would really  
misbehave.  However, analysis tools such as valgrind might whine about  
this; accordingly, let's band-aid relmapper.c to not do it.  
  
See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes.  
Apparently, none of those folk tried valgrinding initdb?  This has been  
like this for long enough that I'm surprised it hasn't been reported  
before.  
  
Back-patch, just in case anybody wants to use a back branch on a platform  
that complains about this; we back-patched those earlier fixes too.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/utils/cache/relmapper.c

Use native methods to open input in TestLib::slurp_file on Windows.

commit   : b5eb07ed78a0e52b392c9b64e01e044a45c1fdb6    
  
author   : Andrew Dunstan <[email protected]>    
date     : Tue, 15 Dec 2020 10:00:18 -0500    
  
committer: Andrew Dunstan <[email protected]>    
date     : Tue, 15 Dec 2020 10:00:18 -0500    

Click here for diff

This is a backport of commits 114541d58e and 6f59826f0 to the remaining  
live branches.  

M src/test/perl/TestLib.pm

Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE."

commit   : f10da1e9c970019ad9427cbff1f340d91c093603    
  
author   : Jeff Davis <[email protected]>    
date     : Mon, 14 Dec 2020 23:49:57 -0800    
  
committer: Jeff Davis <[email protected]>    
date     : Mon, 14 Dec 2020 23:49:57 -0800    

Click here for diff

This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327.  
  
Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked  
around.  
  
This enables proper pipelining of commands after terminating  
replication, eliminating an undocumented limitation.  
  
Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi  
Backpatch-through: 9.5  

M src/backend/replication/walsender.c

initdb: complete getopt_long alphabetization

commit   : 348dd28a0df7c56e88cc83cedeb92a43db39233c    
  
author   : Bruce Momjian <[email protected]>    
date     : Sat, 12 Dec 2020 12:59:09 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Sat, 12 Dec 2020 12:59:09 -0500    

Click here for diff

Backpatch-through: 9.5  

M src/bin/initdb/initdb.c

initdb: properly alphabetize getopt_long options in C string

commit   : 5ec8a2e7c3bf5af67e2fff68a709d01add16ebfc    
  
author   : Bruce Momjian <[email protected]>    
date     : Sat, 12 Dec 2020 12:51:16 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Sat, 12 Dec 2020 12:51:16 -0500    

Click here for diff

Backpatch-through: 9.5  

M src/bin/initdb/initdb.c

Teach contain_leaked_vars that assignment SubscriptingRefs are leaky.

commit   : 17c77c8c90f7c566a8f42e0809e425072d8f26b7    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 8 Dec 2020 17:50:54 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 8 Dec 2020 17:50:54 -0500    

Click here for diff

array_get_element and array_get_slice qualify as leakproof, since  
they will silently return NULL for bogus subscripts.  But  
array_set_element and array_set_slice throw errors for such cases,  
making them clearly not leakproof.  contain_leaked_vars was evidently  
written with only the former case in mind, as it gave the wrong answer  
for assignment SubscriptingRefs (nee ArrayRefs).  
  
This would be a live security bug, were it not that assignment  
SubscriptingRefs can only occur in INSERT and UPDATE target lists,  
while we only care about leakproofness for qual expressions; so the  
wrong answer can't occur in practice.  Still, that's a rather shaky  
answer for a security-related question; and maybe in future somebody  
will want to ask about leakproofness of a tlist.  So it seems wise to  
fix and even back-patch this correction.  
  
(We would need some change here anyway for the upcoming  
generic-subscripting patch, since extensions might make different  
tradeoffs about whether to throw errors.  Commit 558d77f20 attempted  
to lay groundwork for that by asking check_functions_in_node whether a  
SubscriptingRef contains leaky functions; but that idea fails now that  
the implementation methods of a SubscriptingRef are not SQL-visible  
functions that could be marked leakproof or not.)  
  
Back-patch to 9.6.  While 9.5 has the same issue, the code's a bit  
different.  It seems quite unlikely that we'd introduce any actual bug  
in the short time 9.5 has left to live, so the work/risk/reward balance  
isn't attractive for changing 9.5.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/optimizer/util/clauses.c

Doc: clarify that CREATE TABLE discards redundant unique constraints.

commit   : 2bf5d1a74a33ed45939ed4c16054aee51ab65540    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 8 Dec 2020 13:09:48 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 8 Dec 2020 13:09:48 -0500    

Click here for diff

The SQL standard says that redundant unique constraints are disallowed,  
but we long ago decided that throwing an error would be too  
user-unfriendly, so we just drop redundant ones.  The docs weren't very  
clear about that though, as this behavior was only explained for PRIMARY  
KEY vs UNIQUE, not UNIQUE vs UNIQUE.  
  
While here, I couldn't resist doing some copy-editing and markup-fixing  
on the adjacent text about INCLUDE options.  
  
Per bug #16767 from Matthias vd Meent.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/ref/create_table.sgml

Doc: explain that the string types can't store \0 (ASCII NUL).

commit   : 3410a9b57062609ecd823532847f622d862ba9ec    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 8 Dec 2020 12:06:19 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 8 Dec 2020 12:06:19 -0500    

Click here for diff

This restriction was mentioned in connection with string literals,  
but it wasn't made clear that it's a general restriction not just  
a syntactic limitation in query strings.  
  
Per unsigned documentation comment.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/datatype.sgml

pgcrypto: Detect errors with EVP calls from OpenSSL

commit   : 95992e5ed93d142d64efb020a51148d39678df6a    
  
author   : Michael Paquier <[email protected]>    
date     : Tue, 8 Dec 2020 15:22:59 +0900    
  
committer: Michael Paquier <[email protected]>    
date     : Tue, 8 Dec 2020 15:22:59 +0900    

Click here for diff

The following routines are called within pgcrypto when handling digests  
but there were no checks for failures:  
- EVP_MD_CTX_size (can fail with -1 as of 3.0.0)  
- EVP_MD_CTX_block_size (can fail with -1 as of 3.0.0)  
- EVP_DigestInit_ex  
- EVP_DigestUpdate  
- EVP_DigestFinal_ex  
  
A set of elog(ERROR) is added by this commit to detect such failures,  
that should never happen except in the event of a processing failure  
internal to OpenSSL.  
  
Note that it would be possible to use ERR_reason_error_string() to get  
more context about such errors, but these refer mainly to the internals  
of OpenSSL, so it is not really obvious how useful that would be.  This  
is left out for simplicity.  
  
Per report from Coverity.  Thanks to Tom Lane for the discussion.  
  
Backpatch-through: 9.5  

M contrib/pgcrypto/openssl.c

Fix more race conditions in the newly-added pg_rewind test.

commit   : 3ea8e660c0e050826ce06b0ad4dc34ae64c8f850    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Mon, 7 Dec 2020 14:44:34 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Mon, 7 Dec 2020 14:44:34 +0200    

Click here for diff

pg_rewind looks at the control file to check what timeline a server is on.  
But promotion doesn't immediately write a checkpoint, it merely writes  
an end-of-recovery WAL record. If pg_rewind runs immediately after  
promotion, before the checkpoint has completed, it will think think that  
the server is still on the earlier timeline. We ran into this issue a long  
time ago already, see commit 484a848a73f.  
  
It's a bit bogus that pg_rewind doesn't determine the timeline correctly  
until the end-of-recovery checkpoint has completed. We probably should  
fix that. But for now work around it by waiting for the checkpoint  
to complete before running pg_rewind, like we did in commit 484a848a73f.  
  
In the passing, tidy up the new test a little bit. Rerder the INSERTs so  
that the comments make more sense, remove a spurious CHECKPOINT call after  
pg_rewind has already run, and add --debug option, so that if this fails  
again, we'll have more data.  
  
Per buildfarm failure at https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2020-12-06%2018%3A32%3A19&stg=pg_rewind-check.  
Backpatch to all supported versions.  
  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M src/bin/pg_rewind/t/008_min_recovery_point.pl

Fix race conditions in newly-added test.

commit   : a075c84f2ce17cad75c3afe57de2ecb05a180288    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Fri, 4 Dec 2020 18:20:18 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Fri, 4 Dec 2020 18:20:18 +0200    

Click here for diff

Buildfarm has been failing sporadically on the new test.  I was able to  
reproduce this by adding a random 0-10 s delay in the walreceiver, just  
before it connects to the primary. There's a race condition where node_3  
is promoted before it has fully caught up with node_1, leading to diverged  
timelines. When node_1 is later reconfigured as standby following node_3,  
it fails to catch up:  
  
LOG:  primary server contains no more WAL on requested timeline 1  
LOG:  new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0  
  
That's the situation where you'd need to use pg_rewind, but in this case  
it happens already when we are just setting up the actual pg_rewind  
scenario we want to test, so change the test so that it waits until  
node_3 is connected and fully caught up before promoting it, so that you  
get a clean, controlled failover.  
  
Also rewrite some of the comments, for clarity. The existing comments  
detailed what each step in the test did, but didn't give a good overview  
of the situation the steps were trying to create.  
  
For reasons I don't understand, the test setup had to be written slightly  
differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version  
needed node 1 to be reinitialized from backup, whereas in later versions  
it could be shut down and reconfigured to be a standby. But even 9.5 should  
support "clean switchover", where primary makes sure that pending WAL is  
replicated to standby on shutdown. It would be nice to figure out what's  
going on there, but that's independent of pg_rewind and the scenario that  
this test tests.  
  
Discussion: https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi  

M src/bin/pg_rewind/t/008_min_recovery_point.pl

docs: list single-letter options first in command-line summary

commit   : 89cdf1b65e6b64c3bd784717ee4429e9db5eaf8d    
  
author   : Bruce Momjian <[email protected]>    
date     : Thu, 3 Dec 2020 10:28:58 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Thu, 3 Dec 2020 10:28:58 -0500    

Click here for diff

In a few places, the long-version options were listed before the  
single-letter ones in the command summary of a few commands.  This  
didn't match other commands, and didn't match the option ordering later  
in the same reference page.  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml

Fix pg_rewind bugs when rewinding a standby server.

commit   : 0740857de7804609783eb9f11f055776c1fff4e8    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Thu, 3 Dec 2020 15:57:48 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Thu, 3 Dec 2020 15:57:48 +0200    

Click here for diff

If the target is a standby server, its WAL doesn't end at the last  
checkpoint record, but at minRecoveryPoint. We must scan all the  
WAL from the last common checkpoint all the way up to minRecoveryPoint  
for modified pages, and also consider that portion when determining  
whether the server needs rewinding.  
  
Backpatch to all supported versions.  
  
Author: Ian Barwick and me  
Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com  

M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
A src/bin/pg_rewind/t/008_min_recovery_point.pl

Ensure that expandTableLikeClause() re-examines the same table.

commit   : f00c4400270f21905305a4648d1f3d57400a2e15    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 1 Dec 2020 14:02:28 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 1 Dec 2020 14:02:28 -0500    

Click here for diff

As it stood, expandTableLikeClause() re-did the same relation_openrv  
call that transformTableLikeClause() had done.  However there are  
scenarios where this would not find the same table as expected.  
We hold lock on the LIKE source table, so it can't be renamed or  
dropped, but another table could appear before it in the search path.  
This explains the odd behavior reported in bug #16758 when cloning a  
table as a temp table of the same name.  This case worked as expected  
before commit 502898192 introduced the need to open the source table  
twice, so we should fix it.  
  
To make really sure we get the same table, let's re-open it by OID not  
name.  That requires adding an OID field to struct TableLikeClause,  
which is a little nervous-making from an ABI standpoint, but as long  
as it's at the end I don't think there's any serious risk.  
  
Per bug #16758 from Marc Boeren.  Like the previous patch,  
back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql

Free disk space for dropped relations on commit.

commit   : d0bbe212209924e664a783a767e827df5b5c08d3    
  
author   : Thomas Munro <[email protected]>    
date     : Tue, 1 Dec 2020 13:46:27 +1300    
  
committer: Thomas Munro <[email protected]>    
date     : Tue, 1 Dec 2020 13:46:27 +1300    

Click here for diff

When committing a transaction that dropped a relation, we previously  
truncated only the first segment file to free up disk space (the one  
that won't be unlinked until the next checkpoint).  
  
Truncate higher numbered segments too, even though we unlink them on  
commit.  This frees the disk space immediately, even if other backends  
have open file descriptors and might take a long time to get around to  
handling shared invalidation events and closing them.  Also extend the  
same behavior to the first segment, in recovery.  
  
Back-patch to all supported releases.  
  
Bug: #16663  
Reported-by: Denis Patron <[email protected]>  
Reviewed-by: Pavel Borisov <[email protected]>  
Reviewed-by: Neil Chen <[email protected]>  
Reviewed-by: David Zhang <[email protected]>  
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org  

M src/backend/storage/smgr/md.c

Document concurrent indexes waiting on each other

commit   : b3d33bf598dd64c95f48111081b6672ec0dc380e    
  
author   : Alvaro Herrera <[email protected]>    
date     : Mon, 30 Nov 2020 18:24:55 -0300    
  
committer: Alvaro Herrera <[email protected]>    
date     : Mon, 30 Nov 2020 18:24:55 -0300    

Click here for diff

Because regular CREATE INDEX commands are independent, and there's no  
logical data dependency, it's not immediately obvious that transactions  
held by concurrent index builds on one table will block the second phase  
of concurrent index creation on an unrelated table, so document this  
caveat.  
  
Backpatch this all the way back.  In branch master, mention that only  
some indexes are involved.  
  
Author: James Coleman <[email protected]>  
Reviewed-by: David Johnston <[email protected]>  
Discussion: https://postgr.es/m/CAAaqYe994=PUrn8CJZ4UEo_S-FfRr_3ogERyhtdgHAb2WG_Ufg@mail.gmail.com  

M doc/src/sgml/ref/create_index.sgml

Fix miscomputation of direct_lateral_relids for join relations.

commit   : ab4cbb4bceff6f939f12d0edbe84004d85ebea16    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 30 Nov 2020 12:22:43 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 30 Nov 2020 12:22:43 -0500    

Click here for diff

If a PlaceHolderVar is to be evaluated at a join relation, but  
its value is only needed there and not at higher levels, we neglected  
to update the joinrel's direct_lateral_relids to include the PHV's  
source rel.  This causes problems because join_is_legal() then won't  
allow joining the joinrel to the PHV's source rel at all, leading  
to "failed to build any N-way joins" planner failures.  
  
Per report from Andreas Seltenreich.  Back-patch to 9.5  
where the problem originated.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/optimizer/util/placeholder.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql

Fix recently-introduced breakage in psql's \connect command.

commit   : 3f59a05f0fa8c4f5f43fe0f5fed3291cf2cf224a    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 29 Nov 2020 15:22:04 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 29 Nov 2020 15:22:04 -0500    

Click here for diff

Through my misreading of what the existing code actually did,  
commits 85c54287a et al. broke psql's behavior for the case where  
"\c connstring" provides a password in the connstring.  We should  
use that password in such a case, but as of 85c54287a we ignored it  
(and instead, prompted for a password).  
  
Commit 94929f1cf fixed that in HEAD, but since I thought it was  
cleaning up a longstanding misbehavior and not one I'd just created,  
I didn't back-patch it.  
  
Hence, back-patch the portions of 94929f1cf having to do with  
password management.  In addition to fixing the introduced bug,  
this means that "\c -reuse-previous=on connstring" will allow  
re-use of an existing connection's password if the connstring  
doesn't change user/host/port.  That didn't happen before, but  
it seems like a bug fix, and anyway I'm loath to have significant  
differences in this code across versions.  
  
Also fix an error with the same root cause about whether or not to  
override a connstring's setting of client_encoding.  As of 85c54287a  
we always did so; restore the previous behavior of overriding only  
when stdin/stdout are a terminal and there's no environment setting  
of PGCLIENTENCODING.  (I find that definition a bit surprising, but  
right now doesn't seem like the time to revisit it.)  
  
Per bug #16746 from Krzysztof Gradek.  As with the previous patch,  
back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c

Doc: clarify behavior of PQconnectdbParams().

commit   : 7fbd1442807ad62ee0e71e2ff3f8b0eefb0af5b8    
  
author   : Tom Lane <[email protected]>    
date     : Sun, 29 Nov 2020 13:58:30 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sun, 29 Nov 2020 13:58:30 -0500    

Click here for diff

The documentation omitted the critical tidbit that a keyword-array entry  
is simply ignored if its corresponding value-array entry is NULL or an  
empty string; it will *not* override any previously-obtained value for  
the parameter.  (See conninfo_array_parse().)  I'd supposed that would  
force the setting back to default, which is what led me into bug #16746;  
but it doesn't.  
  
While here, I couldn't resist the temptation to do some copy-editing,  
both in the description of PQconnectdbParams() and in the section  
about connection URI syntax.  
  
Discussion: https://postgr.es/m/[email protected]  

M doc/src/sgml/libpq.sgml

Fix a recently-introduced race condition in LISTEN/NOTIFY handling.

commit   : 8a4069766079df3b0aa781bc2a0d67b105cda34a    
  
author   : Tom Lane <[email protected]>    
date     : Sat, 28 Nov 2020 14:03:40 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Sat, 28 Nov 2020 14:03:40 -0500    

Click here for diff

Commit 566372b3d fixed some race conditions involving concurrent  
SimpleLruTruncate calls, but it introduced new ones in async.c.  
A newly-listening backend could attempt to read Notify SLRU pages that  
were in process of being truncated, possibly causing an error.  Also,  
the QUEUE_TAIL pointer could become set to a value that's not equal to  
the queue position of any backend.  While that's fairly harmless in  
v13 and up (thanks to commit 51004c717), in older branches it resulted  
in near-permanent disabling of the queue truncation logic, so that  
continued use of NOTIFY led to queue-fill warnings and eventual  
inability to send any more notifies.  (A server restart is enough to  
make that go away, but it's still pretty unpleasant.)  
  
The core of the problem is confusion about whether QUEUE_TAIL  
represents the "logical" tail of the queue (i.e., the oldest  
still-interesting data) or the "physical" tail (the oldest data we've  
not yet truncated away).  To fix, split that into two variables.  
QUEUE_TAIL regains its definition as the logical tail, and we  
introduce a new variable to track the oldest un-truncated page.  
  
Per report from Mikael Gustavsson.  Like the previous patch,  
back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/commands/async.c

doc: Fix typos

commit   : 272ace098dd365b7aba2dfa93fbf35e1ba55d984    
  
author   : Peter Eisentraut <[email protected]>    
date     : Wed, 25 Nov 2020 09:49:00 +0100    
  
committer: Peter Eisentraut <[email protected]>    
date     : Wed, 25 Nov 2020 09:49:00 +0100    

Click here for diff

Author: Justin Pryzby <[email protected]>  
Discussion: https://www.postgresql.org/message-id/[email protected]  

M doc/src/sgml/contrib.sgml

Properly check index mark/restore in ExecSupportsMarkRestore.

commit   : 6402afd9865e3218b1f3835108a1e156157b6e08    
  
author   : Andrew Gierth <[email protected]>    
date     : Tue, 24 Nov 2020 20:58:32 +0000    
  
committer: Andrew Gierth <[email protected]>    
date     : Tue, 24 Nov 2020 20:58:32 +0000    

Click here for diff

Previously this code assumed that all IndexScan nodes supported  
mark/restore, which is not true since it depends on optional index AM  
support functions. This could lead to errors about missing support  
functions in rare edge cases of mergejoins with no sort keys, where an  
unordered non-btree index scan was placed on the inner path without a  
protecting Materialize node. (Normally, the fact that merge join  
requires ordered input would avoid this error.)  
  
Backpatch all the way since this bug is ancient.  
  
Per report from Eugen Konkov on irc.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/executor/execAmi.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/relation.h

Skip allocating hash table in EXPLAIN-only mode.

commit   : 02a2dbe91d37e521fb0dbeebc79ee9edd051f950    
  
author   : Heikki Linnakangas <[email protected]>    
date     : Fri, 20 Nov 2020 14:41:14 +0200    
  
committer: Heikki Linnakangas <[email protected]>    
date     : Fri, 20 Nov 2020 14:41:14 +0200    

Click here for diff

This is a backpatch of commit 2cccb627f1, backpatched due to popular  
demand. Backpatch to all supported versions.  
  
Author: Alexey Bashtanov  
Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc  

M src/backend/executor/nodeAgg.c

commit   : e7abc1111634f51dfb653fc95bb13cbf2468c466    
  
author   : Tom Lane <[email protected]>    
date     : Fri, 20 Nov 2020 00:58:26 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Fri, 20 Nov 2020 00:58:26 -0500    

Click here for diff

We previously put the -isysroot switch only into CPPFLAGS, theorizing  
that it was only needed to find the right copies of include files.  
However, it seems that we also need to use it while linking programs,  
to find the right stub ".tbd" files for libraries.  We got away  
without that up to now, but apparently that was mostly luck.  It may  
also be that failures are only observed when the Xcode version is  
noticeably out of sync with the host macOS version; the case that's  
prompting action right now is that builds fail when using latest Xcode  
(12.2) on macOS Catalina, even though it's fine on Big Sur.  
  
Hence, add -isysroot to LDFLAGS as well.  (It seems that the more  
common practice is to put it in CFLAGS, whence it'd be included at  
both compile and link steps.  However, we can't mess with CFLAGS in  
the platform template file without confusing configure's logic for  
choosing default CFLAGS.)  
  
Back-patch of 49407dc32 into all supported branches.  
  
Report and patch by James Hilliard (some cosmetic mods by me)  
  
Discussion: https://postgr.es/m/[email protected]  

M configure
M configure.in
M src/template/darwin

Further fixes for CREATE TABLE LIKE: cope with self-referential FKs.

commit   : 159b6775fad9b040fb181e5660a8149e57aa0609    
  
author   : Tom Lane <[email protected]>    
date     : Thu, 19 Nov 2020 15:03:17 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Thu, 19 Nov 2020 15:03:17 -0500    

Click here for diff

Commit 502898192 was too careless about the order of execution of the  
additional ALTER TABLE operations generated by expandTableLikeClause.  
It just stuck them all at the end, which seems okay for most purposes.  
But it falls down in the case where LIKE is importing a primary key  
or unique index and the outer CREATE TABLE includes a FOREIGN KEY  
constraint that needs to depend on that index.  Weird as that is,  
it used to work, so we ought to keep it working.  
  
To fix, make parse_utilcmd.c insert LIKE clauses between index-creation  
and FK-creation commands in the transformed list of commands, and change  
utility.c so that the commands generated by expandTableLikeClause are  
executed immediately not at the end.  One could imagine scenarios where  
this wouldn't work either; but currently expandTableLikeClause only  
makes column default expressions, CHECK constraints, and indexes, and  
this ordering seems fine for those.  
  
Per bug #16730 from Sofoklis Papasofokli.  Like the previous patch,  
back-patch to all supported branches.  
  
Discussion: https://postgr.es/m/[email protected]  

M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql

Don't Insert() a VFD entry until it's fully built.

commit   : d726e44fb9c0a8e3c0f405b88d28d55ece1837c6    
  
author   : Tom Lane <[email protected]>    
date     : Mon, 16 Nov 2020 20:32:35 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Mon, 16 Nov 2020 20:32:35 -0500    

Click here for diff

Otherwise, if FDDEBUG is enabled, the debugging output fails because  
it tries to read the fileName, which isn't set up yet (and should in  
fact always be NULL).  
  
AFAICT, this has been wrong since Berkeley.  Before 96bf88d52,  
it would accidentally fail to crash on platforms where snprintf()  
is forgiving about being passed a NULL pointer for %s; but the  
file name intended to be included in the debug output wouldn't  
ever have shown up.  
  
Report and fix by Greg Nancarrow.  Although this is only visibly  
broken in custom-made builds, it still seems worth back-patching  
to all supported branches, as the FDDEBUG code is pretty useless  
as it stands.  
  
Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com  

M src/backend/storage/file/fd.c

doc: update bgwriter description

commit   : de56e96ac83d2745566d36fd03617561ee026a2b    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 13:13:43 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 13:13:43 -0500    

Click here for diff

This clarifies exactly what the bgwriter does, which should help with  
tuning.  
  
Reported-by: Chris Wilson  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

M doc/src/sgml/config.sgml

doc: clarify how to find pg_type_d.h in the install tree

commit   : 5491d7829e7ed6726381cfbc90b2ce27c5016f8b    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 12:36:16 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 12:36:16 -0500    

Click here for diff

Followup to patch 152ed04799.  
  
Reported-by: Alvaro Herrera  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

M doc/src/sgml/libpq.sgml

doc: adjust expression index analyze working some more

commit   : 7bf7e5caad76b829dd46301501f751c87083bc66    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 11:14:54 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 11:14:54 -0500    

Click here for diff

This applies the fix bcbd771332 to skipped branches.  
  
Reported-by: Erik Rijkers  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5-11  

M doc/src/sgml/ref/create_index.sgml

doc: improve wording of the need for analyze of exp. indexes

commit   : 708853a8a72ff96ebd35cfde78187a7e135da806    
  
author   : Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 10:26:16 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Mon, 16 Nov 2020 10:26:16 -0500    

Click here for diff

This is a followup commit on 3370207986.  
  
Reported-by: Justin Pryzby  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/create_index.sgml

doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h

commit   : fe709f3e0f178fc2d79fb8d4a16ce88497b702c9    
  
author   : Bruce Momjian <[email protected]>    
date     : Thu, 12 Nov 2020 15:13:01 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Thu, 12 Nov 2020 15:13:01 -0500    

Click here for diff

These files are in compiled directories and install directories.  
  
Reported-by: [email protected]  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

M doc/src/sgml/libpq.sgml

docs: mention that expression indexes need analyze

commit   : 05a719ceb0aada6d1a1a9b432227841bbaf280ae    
  
author   : Bruce Momjian <[email protected]>    
date     : Thu, 12 Nov 2020 15:00:44 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Thu, 12 Nov 2020 15:00:44 -0500    

Click here for diff

Expression indexes can't benefit from pre-computed statistics on  
columns.  
  
Reported-by: Nikolay Samokhvalov  
  
Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com  
  
Author: Nikolay Samokhvalov, modified  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/create_index.sgml

doc: wire protocol data type for history file content is bytea

commit   : 86913ba9abaf2a56ea19b61ccbcf9d35e894d5d0    
  
author   : Bruce Momjian <[email protected]>    
date     : Thu, 12 Nov 2020 14:33:28 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Thu, 12 Nov 2020 14:33:28 -0500    

Click here for diff

Document that though the history file content is marked as bytea, it is  
the same a text, and neither is btyea-escaped or encoding converted.  
  
Reported-by: Brar Piening  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 13 - 9.5 (not master)  

M doc/src/sgml/protocol.sgml
M src/backend/replication/walsender.c

pg_trgm: fix crash in 2-item picksplit

commit   : 245a35f96e3a12b7fe19bad88671d75dcbec133c    
  
author   : Andrew Gierth <[email protected]>    
date     : Thu, 12 Nov 2020 14:34:37 +0000    
  
committer: Andrew Gierth <[email protected]>    
date     : Thu, 12 Nov 2020 14:34:37 +0000    

Click here for diff

Whether from size overflow in gistSplit or from secondary splits,  
picksplit is (rarely) called with exactly two items to split.  
  
Formerly, due to special-case handling of the last item, this would  
lead to access to an uninitialized cache entry; prior to PG 13 this  
might have been harmless or at worst led to an incorrect union datum,  
but in 13 onwards it can cause a backend crash from using an  
uninitialized pointer.  
  
Repair by removing the special case, which was deemed not to have been  
appropriate anyway. Backpatch all the way, because this bug has  
existed since pg_trgm was added.  
  
Per report on IRC from user "ftzdomino". Analysis and testing by me,  
patch from Alexander Korotkov.  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/pg_trgm/trgm_gist.c

Fix and simplify some usages of TimestampDifference().

commit   : cd39c23a21ac46009d21a51db7438bba9d5d1941    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 10 Nov 2020 22:51:19 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 10 Nov 2020 22:51:19 -0500    

Click here for diff

Introduce TimestampDifferenceMilliseconds() to simplify callers  
that would rather have the difference in milliseconds, instead of  
the select()-oriented seconds-and-microseconds format.  This gets  
rid of at least one integer division per call, and it eliminates  
some apparently-easy-to-mess-up arithmetic.  
  
Two of these call sites were in fact wrong:  
  
* pg_prewarm's autoprewarm_main() forgot to multiply the seconds  
by 1000, thus ending up with a delay 1000X shorter than intended.  
That doesn't quite make it a busy-wait, but close.  
  
* postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute  
microseconds not milliseconds, thus ending up with a delay 1000X longer  
than intended.  Somebody along the way had noticed this problem but  
misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather  
than fixing the units.  This was relatively harmless in context, because  
we don't care that much about exactly how long this delay is; still,  
it's wrong.  
  
There are a few more callers of TimestampDifference() that don't  
have a direct need for seconds-and-microseconds, but can't use  
TimestampDifferenceMilliseconds() either because they do need  
microsecond precision or because they might possibly deal with  
intervals long enough to overflow 32-bit milliseconds.  It might be  
worth inventing another API to improve that, but that seems outside  
the scope of this patch; so those callers are untouched here.  
  
Given the fact that we are fixing some bugs, and the likelihood  
that future patches might want to back-patch code that uses this  
new API, back-patch to all supported branches.  
  
Alexey Kondratov and Tom Lane  
  
Discussion: https://postgr.es/m/[email protected]  

M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xlog.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/timestamp.h

doc: fix spelling "connction" to "connection"

commit   : efc306935de1b7de0ab696a54193f57b26268207    
  
author   : Bruce Momjian <[email protected]>    
date     : Tue, 10 Nov 2020 19:18:35 -0500    
  
committer: Bruce Momjian <[email protected]>    
date     : Tue, 10 Nov 2020 19:18:35 -0500    

Click here for diff

Was wrong in commit 1a9388bd0f.  
  
Reported-by: Tom Lane, Justin Pryzby  
  
Discussion: https://postgr.es/m/[email protected]  
  
Backpatch-through: 9.5  

M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dumpall.sgml

Work around cross-version-upgrade issues created by commit 9e38c2bb5.

commit   : cea97d98f1674fa4d0c1174b32dcdc44d9d5e620    
  
author   : Tom Lane <[email protected]>    
date     : Tue, 10 Nov 2020 18:32:36 -0500    
  
committer: Tom Lane <[email protected]>    
date     : Tue, 10 Nov 2020 18:32:36 -0500    

Click here for diff

Summarily changing the STYPE of regression-test aggregates that  
depend on array_append or array_cat is an issue for the buildfarm's  
cross-version-upgrade tests, because those aggregates (as defined  
in the back branches) now won't load into HEAD.  Although this seems  
like only a minimal risk for genuine user-defined aggregates, we  
need to do something for the buildfarm.  Hence, adjust the aggregate  
definitions, in both HEAD and the back branches.  
  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/[email protected]  

M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/polymorphism.sql