Stamp 9.6.20.
commit : e4554425e6d3b429a207ac1a109af9d7e1d14da6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 17:32:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 17:32:22 -0500
M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : 0425151233e018edfc03d2c3fe38f94ef3a67e77
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 13:02:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 13:02:14 -0500
Security: CVE-2020-25694, CVE-2020-25695, CVE-2020-25696
M doc/src/sgml/release-9.6.sgml
Doc: clarify data type behavior of COALESCE and NULLIF.
commit : d4fb509d07c36e1839ba2a1c0d00ff18606937f6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 12:02:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 12:02:24 -0500
After studying the code, NULLIF is a lot more subtle than you might
have guessed.
Discussion: https://postgr.es/m/160486028730.25500.15740897403028593550@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
M doc/src/sgml/typeconv.sgml
Ignore attempts to \gset into specially treated variables.
commit : 12fd81cb7fab49ba0a5f004596e52271275deec0
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
If an interactive psql session used \gset when querying a compromised
server, the attacker could execute arbitrary code as the operating
system account running psql. Using a prefix not found among specially
treated variables, e.g. every lowercase string, precluded the attack.
Fix by issuing a warning and setting no variable for the column in
question. Users wanting the old behavior can use a prefix and then a
meta-command like "\set HISTSIZE :prefix_HISTSIZE". Back-patch to 9.5
(all supported versions).
Reviewed by Robert Haas. Reported by Nick Cleaton.
Security: CVE-2020-25696
M src/bin/psql/common.c
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
In security-restricted operations, block enqueue of at-commit user code.
commit : ff3de4c21a462f2da773848c53faedac88d6e54d
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
Specifically, this blocks DECLARE ... WITH HOLD and firing of deferred
triggers within index expressions and materialized view queries. An
attacker having permission to create non-temp objects in at least one
schema could execute arbitrary SQL functions under the identity of the
bootstrap superuser. One can work around the vulnerability by disabling
autovacuum and not manually running ANALYZE, CLUSTER, REINDEX, CREATE
INDEX, VACUUM FULL, or REFRESH MATERIALIZED VIEW. (Don't restore from
pg_dump, since it runs some of those commands.) Plain VACUUM (without
FULL) is safe, and all commands are fine when a trusted user owns the
target object. Performance may degrade quickly under this workaround,
however. Back-patch to 9.5 (all supported versions).
Reviewed by Robert Haas. Reported by Etienne Stalmans.
Security: CVE-2020-25695
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xact.c
M src/backend/commands/portalcmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Translation updates
commit : d8a9722bd39c4eced4b13551f0401b37ff72ef74
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 Nov 2020 12:47:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 Nov 2020 12:47:52 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: f09d69720b2d48f37d3b555c38501b6529c0c6ac
M src/backend/po/de.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/initdb/po/cs.po
M src/bin/initdb/po/fr.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/sv.po
M src/bin/pg_basebackup/po/de.po
M src/bin/pg_basebackup/po/fr.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_basebackup/po/sv.po
M src/bin/pg_config/po/ru.po
M src/bin/pg_ctl/po/cs.po
M src/bin/pg_ctl/po/fr.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/sv.po
M src/bin/pg_dump/po/cs.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
M src/bin/psql/po/cs.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/cs.po
M src/bin/scripts/po/ru.po
M src/interfaces/ecpg/ecpglib/po/sv.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/sv.po
M src/interfaces/libpq/po/ru.po
M src/pl/plperl/po/sv.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/sv.po
M src/pl/plpython/po/sv.po
M src/pl/tcl/po/sv.po
Doc: suppress PDF build warning in 9.6 branch.
commit : 7ebea4e224ca826e6c7abc6c682701485cd634f6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 16:22:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 16:22:39 -0500
Fractional colwidths seem not to work in the openjade toolchain.
Since we have no colwidth specs anywhere else in pre-v10 branches,
let's just drop the ones introduced by commits ee59f669b/35b12948a.
M doc/src/sgml/config.sgml
Release notes for 13.1, 12.5, 11.10, 10.15, 9.6.20, 9.5.24.
commit : a39e23b8c8d7ae0744619a5e7a14bcc907b696f9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 15:16:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 15:16:12 -0500
M doc/src/sgml/release-9.6.sgml
Fix redundant error messages in client tools
commit : 8096be26178befa690841e449abb39e134f8d415
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 22:15:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 22:15:52 +0100
A few client tools duplicate error messages already provided by libpq.
Discussion: https://www.postgresql.org/message-id/flat/3e937641-88a1-e697-612e-99bba4b8e5e4%40enterprisedb.com
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_rewind/libpq_fetch.c
Properly detoast data in brin_form_tuple
commit : bae31e75f7777f2398eccbd40c5676e3dd487d1a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 7 Nov 2020 00:41:36 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 7 Nov 2020 00:41:36 +0100
brin_form_tuple failed to consider the values may be toasted, inserting
the toast pointer into the index. This may easily result in index
corruption, as the toast data may be deleted and cleaned up by vacuum.
The cleanup however does not care about indexes, leaving invalid toast
pointers behind, which triggers errors like this:
ERROR: missing chunk number 0 for toast value 16433 in pg_toast_16426
A less severe consequence are inconsistent failures due to the index row
being too large, depending on whether brin_form_tuple operated on plain
or toasted version of the row. For example
CREATE TABLE t (val TEXT);
INSERT INTO t VALUES ('... long value ...')
CREATE INDEX idx ON t USING brin (val);
would likely succeed, as the row would likely include toast pointer.
Switching the order of INSERT and CREATE INDEX would likely fail:
ERROR: index row size 8712 exceeds maximum 8152 for index "idx"
because this happens before the row values are toasted.
The bug exists since PostgreSQL 9.5 where BRIN indexes were introduced.
So backpatch all the way back.
Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Backpatch-through: 9.5
Discussion: https://postgr.es/m/20201001184133.oq5uq75sb45pu3aw@development
Discussion: https://postgr.es/m/20201104010544.zexj52mlldagzowv%40development
M src/backend/access/brin/brin_tuple.c
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
Revert "Accept relations of any kind in LOCK TABLE".
commit : 9e555180f23dd4d1a85a8b2b93350358322a540a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 16:17:57 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 16:17:57 -0500
Revert 59ab4ac32, as well as the followup fix 33862cb9c, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Revert "pg_dump: Lock all relations, not just plain tables".
commit : 768ab4d676aa84cfcb8f59e843e4e97559b6cc5e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 15:48:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 15:48:21 -0500
Revert 403a3d91c, as well as the followup fix 7f4235032, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
Doc: undo mistaken adjustment to LOCK TABLE docs in back branches.
commit : 583fcf2f57594cc73fe537b2bf18cfbc6b13336f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 12:14:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 12:14:46 -0500
Commits 59ab4ac32 et al mistakenly copied-and-pasted some text about
how LOCK on a view recurses to referenced tables into pre-v11 branches,
which in fact don't do that. Undo that, and instead state clearly
that they don't. (I also chose to add a note that this behavior
changed in v11. We usually don't back-patch such statements, but
since it's easy to add the warning now, might as well.)
Noted while considering followup fixes for bug #16703.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M doc/src/sgml/ref/lock.sgml
Allow users with BYPASSRLS to alter their own passwords.
commit : 04c4b495b1c93c40bc989c450ca8fb5bea965e27
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:41:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:41:32 -0500
The intention in commit 491c029db was to require superuserness to
change the BYPASSRLS property, but the actual effect of the coding
in AlterRole() was to require superuserness to change anything at all
about a BYPASSRLS role. Other properties of a BYPASSRLS role should
be changeable under the same rules as for a normal role, though.
Fix that, and also take care of some documentation omissions related
to BYPASSRLS and REPLICATION role properties.
Tom Lane and Stephen Frost, per bug report from Wolfgang Walther.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/a5548a9f-89ee-3167-129d-162b5985fcf8@technowledgy.de
M doc/src/sgml/ref/alter_role.sgml
M doc/src/sgml/ref/create_role.sgml
M src/backend/commands/user.c
Fix some grammar and typos in comments and docs
commit : c6671cdbde93b9b344d182dcbedeebff745a67c8
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 15:15:37 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 15:15:37 +0900
The documentation fixes are backpatched down to where they apply.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20201031020801.GD3080@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/ref/pg_rewind.sgml
Avoid null pointer dereference if error result lacks SQLSTATE.
commit : 1127377a578bde76ed09edd7cd5d8965365c8aaa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 11:26:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 11:26:16 -0500
Although error results received from the backend should always have
a SQLSTATE field, ones generated by libpq won't, making this code
vulnerable to a crash after, say, untimely loss of connection.
Noted by Coverity.
Oversight in commit 403a3d91c. Back-patch to 9.5, as that was.
M src/bin/pg_dump/pg_backup_db.c
Use mode "r" for popen() in psql's evaluate_backtick().
commit : 204d779695607eae5c6c0fa314516201b0af357f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 14:35:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 14:35:53 -0400
In almost all other places, we use plain "r" or "w" mode in popen()
calls (the exceptions being for COPY data). This one has been
overlooked (possibly because it's buried in a ".l" flex file?),
but it's using PG_BINARY_R.
Kensuke Okamura complained in bug #16688 that we fail to strip \r
when stripping the trailing newline from a backtick result string.
That's true enough, but we'd also fail to convert embedded \r\n
cleanly, which also seems undesirable. Fixing the popen() mode
seems like the best way to deal with this.
It's been like this for a long time, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/16688-c649c7b69cd7e6f8@postgresql.org
M src/bin/psql/psqlscanslash.l
Fix use-after-free bug with event triggers and ALTER TABLE.
commit : cc623ed2f4ac09a97c5ef24dee28d91482ae2e13
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 27 Oct 2020 15:37:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 27 Oct 2020 15:37:13 -0400
EventTriggerAlterTableEnd neglected to make sure that it built its
output list in the right context. In simple cases this was masked
because the function is called in PortalContext which will be
sufficiently long-lived anyway; but that doesn't make it not a bug.
Commit ced138e8c fixed this in HEAD and v13, but mistakenly chose
not to back-patch further. Back-patch the same code change all
the way (I didn't bother with the test case though, as it would
prove nothing in pre-v13 branches).
Per report from Arseny Sher.
Original fix by Jehan-Guillaume de Rorthais.
Discussion: https://postgr.es/m/877drcyprb.fsf@ars-thinkpad
Discussion: https://postgr.es/m/20200902193715.6e0269d4@firost
M src/backend/commands/event_trigger.c
Makefile comment: remove reference to tools/thread/thread_test
commit : 00e4788f206452168a1b9505c22956074722e930
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 14:00:38 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 14:00:38 -0400
You can't compile thread_test alone anymore, and the location moved too.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/1062278.1603819969@sss.pgh.pa.us
Backpatch-through: 9.5
M src/template/netbsd
pg_dump: Lock all relations, not just plain tables
commit : a575a1ab27ef8362f5e885b41092824929961288
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 14:31:37 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 14:31:37 -0300
Now that LOCK TABLE can take any relation type, acquire lock on all
relations that are to be dumped. This prevents schema changes or
deadlock errors that could cause a dump to fail after expending much
effort. The server is tested to have the capability and the feature
disabled if it doesn't, so that a patched pg_dump doesn't fail when
connecting to an unpatched server.
Backpatch to 9.5.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
Accept relations of any kind in LOCK TABLE
commit : 53de141f975187621c1c50e473c562ee63cd9b36
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 13:49:19 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 13:49:19 -0300
The restriction that only tables and views can be locked by LOCK TABLE
is quite arbitrary, since the underlying mechanism can lock any relation
type. Drop the restriction so that programs such as pg_dump can lock
all relations they're interested in, preventing schema changes that
could cause a dump to fail after expending much effort.
Backpatch to 9.5.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
docs: remove reference to src/tools/thread
commit : 67b3c3b8c4cbe31d51bc7f68162054f2f33b185a
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 12:43:11 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 12:43:11 -0400
This directory and the ability to build the thread test independently
were removed in commit 8a2121185b.
Reported-by: e.indrupskaya@postgrespro.ru
Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
doc: simplify wording of function error affects
commit : 58947869528dad323196bf79bd0b08c52c5e33c5
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 22:38:11 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 22:38:11 -0400
Reported-by: bob.henkel@gmail.com
Discussion: https://postgr.es/m/160324449781.693.8298142858847611071@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/plpgsql.sgml
doc: make blooms docs match reality
commit : eb435768889467fb7e4bb84321f4347e25377e5a
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 19:17:05 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 19:17:05 -0400
Parallel execution changed the way bloom queries are executed, so update
the EXPLAIN output, and restructure the docs to be clearer and more
accurate.
Reported-by: Daniel Westermann
Discussion: https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM
Author: Daniel Westermann and me
Backpatch-through: 9.6
M doc/src/sgml/bloom.sgml
Fix ancient bug in ecpg's pthread_once() emulation for Windows.
commit : 379c43bdad1d18786b87f38e8761f61a478b5875
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Oct 2020 13:12:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Oct 2020 13:12:08 -0400
We must not set the "done" flag until after we've executed the
initialization function. Otherwise, other threads can fall through
the initial unlocked test before initialization is really complete.
This has been seen to cause rare failures of ecpg's thread/descriptor
test, and it could presumably cause other sorts of misbehavior in
threaded ECPG-using applications, since ecpglib relies on
pthread_once() in several places.
Diagnosis and patch by me, based on investigation by Alexander Lakhin.
Back-patch to all supported branches (the bug dates to 2007).
Discussion: https://postgr.es/m/16685-d6cd241872c101d3@postgresql.org
M src/interfaces/ecpg/ecpglib/misc.c
Update time zone data files to tzdata release 2020d.
commit : 05a36321a70e180168cd58a2e20313163c007623
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:23:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:23:47 -0400
DST law changes in Palestine, with a whopping 120 hours' notice.
Also some historical corrections for Palestine.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020d.
commit : 58f9f52a3791e760550ac6a32aa085d3ed5ee667
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:15:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:15:22 -0400
There's no functional change at all here, but I'm curious to see
whether this change successfully shuts up Coverity's warning about
a useless strcmp(), which appeared with the previous update.
Discussion: http://mm.icann.org/pipermail/tz/2020-October/029370.html
M src/timezone/README
M src/timezone/zic.c
Fix connection string handling in psql's \connect command.
commit : 870a2323033e8b96b27cdf65a53f3e0c7c7b9a88
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 16:18:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 16:18:41 -0400
psql's \connect claims to be able to re-use previous connection
parameters, but in fact it only re-uses the database name, user name,
host name (and possibly hostaddr, depending on version), and port.
This is problematic for assorted use cases. Notably, pg_dump[all]
emits "\connect databasename" commands which we would like to have
re-use all other parameters. If such a script is loaded in a psql run
that initially had "-d connstring" with some non-default parameters,
those other parameters would be lost, potentially causing connection
failure. (Thus, this is the same kind of bug addressed in commits
a45bc8a4f and 8e5793ab6, although the details are much different.)
To fix, redesign do_connect() so that it pulls out all properties
of the old PGconn using PQconninfo(), and then replaces individual
properties in that array. In the case where we don't wish to re-use
anything, get libpq's default settings using PQconndefaults() and
replace entries in that, so that we don't need different code paths
for the two cases.
This does result in an additional behavioral change for cases where
the original connection parameters allowed multiple hosts, say
"psql -h host1,host2", and the \connect request allows re-use of the
host setting. Because the previous coding relied on PQhost(), it
would only permit reconnection to the same host originally selected.
Although one can think of scenarios where that's a good thing, there
are others where it is not. Moreover, that behavior doesn't seem to
meet the principle of least surprise, nor was it documented; nor is
it even clear it was intended, since that coding long pre-dates the
addition of multi-host support to libpq. Hence, this patch is content
to drop it and re-use the host list as given.
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Avoid invalid alloc size error in shm_mq
commit : d4e654d15467d68c85e05eaaa49291de9f81b3ed
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:52:25 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:52:25 +0200
In shm_mq_receive(), a huge payload could trigger an unjustified
"invalid memory alloc request size" error due to the way the buffer
size is increased.
Add error checks (documenting the upper limit) and avoid the error by
limiting the allocation size to MaxAllocSize.
Author: Markus Wanner <markus.wanner@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com
M src/backend/storage/ipc/shm_mq.c
Fix connection string handling in src/bin/scripts/ programs.
commit : 5c78f797701b90546279b994791fc07af5bbb5c9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 19:03:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 19:03:47 -0400
When told to process all databases, clusterdb, reindexdb, and vacuumdb
would reconnect by replacing their --maintenance-db parameter with the
name of the target database. If that parameter is a connstring (which
has been allowed for a long time, though we failed to document that
before this patch), we'd lose any other options it might specify, for
example SSL or GSS parameters, possibly resulting in failure to connect.
Thus, this is the same bug as commit a45bc8a4f fixed in pg_dump and
pg_restore. We can fix it in the same way, by using libpq's rules for
handling multiple "dbname" parameters to add the target database name
separately. I chose to apply the same refactoring approach as in that
patch, with a struct to handle the command line parameters that need to
be passed through to connectDatabase. (Maybe someday we can unify the
very similar functions here and in pg_dump/pg_restore.)
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/createdb.sgml
M doc/src/sgml/ref/dropdb.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createlang.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/droplang.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
Misc documentation fixes.
commit : 67a6af69870d1c95e6e78295849fee807ba5691b
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:28:54 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:28:54 +0300
- Misc grammar and punctuation fixes.
- Stylistic cleanup: use spaces between function arguments and JSON fields
in examples. For example "foo(a,b)" -> "foo(a, b)". Add semicolon after
last END in a few PL/pgSQL examples that were missing them.
- Make sentence that talked about "..." and ".." operators more clear,
by avoiding to end the sentence with "..". That makes it look the same
as "..."
- Fix syntax description for HAVING: HAVING conditions cannot be repeated
Patch by Justin Pryzby, per Yaroslav Schekin's report. Backpatch to all
supported versions, to the extent that the patch applies easily.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/dblink.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/ref/select_into.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml
Fix output of tsquery example in docs.
commit : 07d46e7e077f5a1c1d6b5d7c9c8fff51c66d54e5
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 18:50:33 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 18:50:33 +0300
The output for this query changed in commit 4e2477b7b8. Backport to 9.6
like that commit.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
In libpq for Windows, call WSAStartup once and WSACleanup not at all.
commit : cdc7ace161d39ed788e4e169763c80b58d5bd9c1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 11:23:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 11:23:52 -0400
The Windows documentation insists that every WSAStartup call should
have a matching WSACleanup call. However, if that ever had actual
relevance, it wasn't in this century. Every remotely-modern Windows
kernel is capable of cleaning up when a process exits without doing
that, and must be so to avoid resource leaks in case of a process
crash. Moreover, Postgres backends have done WSAStartup without
WSACleanup since commit 4cdf51e64 in 2004, and we've never seen any
indication of a problem with that.
libpq's habit of doing WSAStartup during connection start and
WSACleanup during shutdown is also rather inefficient, since a
series of non-overlapping connection requests leads to repeated,
quite expensive DLL unload/reload cycles. We document a workaround
for that (having the application call WSAStartup for itself), but
that's just a kluge. It's also worth noting that it's far from
uncommon for applications to exit without doing PQfinish, and
we've not heard reports of trouble from that either.
However, the real reason for acting on this is that recent
experiments by Alexander Lakhin show that calling WSACleanup
during PQfinish is triggering the symptom we occasionally see
that a process using libpq fails to emit expected stdio output.
Therefore, let's change libpq so that it calls WSAStartup only
once per process, during the first connection attempt, and never
calls WSACleanup at all.
While at it, get rid of the only other WSACleanup call in our code
tree, in pg_dump/parallel.c; that presumably is equally useless.
Back-patch of HEAD commit 7d00a6b2d.
Discussion: https://postgr.es/m/ac976d8c-03df-d6b8-025c-15a2de8d9af1@postgrespro.ru
M doc/src/sgml/libpq.sgml
M src/bin/pg_dump/parallel.c
M src/interfaces/libpq/fe-connect.c
Fix doc for full text search distance operator.
commit : 8ae447e2f7cd1a60e715f624dc7fe1e1a768e7ab
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 17:58:38 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 17:58:38 +0300
Commit 028350f619 changed its behavior from "at most" to "exactly", but
forgot to update the documentation. Backpatch to 9.6.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
Relax some asserts in merge join costing code
commit : 00fecc24c140b965e0bdd00942f07f0d43983d7d
author : David Rowley <drowley@postgresql.org>
date : Tue, 20 Oct 2020 00:03:16 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 20 Oct 2020 00:03:16 +1300
In the planner, it was possible, given an extreme enough case containing a
large number of joins for the number of estimated rows to become infinite.
This could cause problems in initial_cost_mergejoin() where we perform
some calculations based on those row estimates.
A problem case, presented by Onder Kalaci showed an Assert failure from
an Assert checking outerstartsel <= outerendsel. In his test case this
was effectively NaN <= Inf, which is false. The NaN outerstartsel came
from multiplying the infinite outer_path_rows by 0.0.
In master, this problem was fixed by a90c950fc, however, that fix was too
invasive for the backbranches. Here we just relax the Asserts to allow
them to pass. The worst that appears to happen from this is that we show
NaN cost values and infinite row estimates in EXPLAIN. add_path() would
have had a hard time doing anything useful with such costs, but that does
not really matter as if the row estimates were even close to accurate,
such plan would not complete this side of the heat death of the universe.
Reported-by: Onder Kalaci
Backpatch: 9.5 to 13
Discussion: https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com
M src/backend/optimizer/path/costsize.c
Fix potential memory leak in pgcrypto
commit : 994a02f7f77c760db5c28aa9f34c7e7fd88993cf
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Oct 2020 09:38:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Oct 2020 09:38:13 +0900
When allocating a EVP context, it would have been possible to leak some
memory allocated directly by OpenSSL, that PostgreSQL lost track of if
the initialization of the context allocated failed. The cleanup can be
done with EVP_MD_CTX_destroy().
Note that EVP APIs exist since OpenSSL 0.9.7 and we have in the tree
equivalent implementations for older versions since ce9b75d (code
removed with 9b7cd59a as of 10~). However, in 9.5 and 9.6, the existing
code makes use of EVP_MD_CTX_destroy() and EVP_MD_CTX_create() without
an equivalent implementation when building the tree with OpenSSL 0.9.6
or older, meaning that this code is in reality broken with such versions
since it got introduced in e2838c5. As we have heard no complains about
that, it does not seem worth bothering with in 9.5 and 9.6, so I have
left that out for simplicity.
Author: Michael Paquier
Discussion: https://postgr.es/m/20201015072212.GC2305@paquier.xyz
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
Doc: caution against misuse of 'now' and related datetime literals.
commit : c1a3188baeda2414a7a7453917d73aa5db511067
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 17 Oct 2020 16:02:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 17 Oct 2020 16:02:47 -0400
Section 8.5.1.4, which defines these literals, made only a vague
reference to the fact that they might be evaluated too soon to be
safe in non-interactive contexts. Provide a more explicit caution
against misuse. Also, generalize the wording in the related tip in
section 9.9.4: while it clearly described this problem, it implied
(or really, stated outright) that the problem only applies to table
DEFAULT clauses.
Per gripe from Tijs van Dam. Back-patch to all supported branches.
Discussion: https://postgr.es/m/c2LuRv9BiRT3bqIo5mMQiVraEXey_25B4vUn0kDqVqilwOEu_iVF1tbtvLnyQK7yDG3PFaz_GxLLPil2SDkj1MCObNRVaac-7j1dVdFERk8=@thalex.com
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
Update time zone data files to tzdata release 2020c.
commit : 13dbf4ab82ea4ebe32aa178fcc7535f95a08de2f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:53:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:53:33 -0400
DST law changes in Morocco, Canadian Yukon, Fiji, Macquarie Island,
Casey Station (Antarctica). Historical corrections for France,
Hungary, Monaco.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020c.
commit : 5515c73a610568e9572e01137a0153a98c1f6088
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:40:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:40:16 -0400
This changes zic's default output format from "-b fat" to "-b slim".
We were already using "slim" in v13/HEAD, so those branches drop
the explicit -b switch in the Makefiles. Instead, add an explicit
"-b fat" in v12 and before, so that we don't change the output file
format in those branches. (This is perhaps excessively conservative,
but we decided not to do so in a12079109, and I'll stick with that.)
Other non-cosmetic changes are to drop support for zic's long-obsolete
"-y" switch, and to ensure that strftime() does not change errno
unless it fails.
As usual with tzcode changes, back-patch to all supported branches.
M src/timezone/Makefile
M src/timezone/README
M src/timezone/strftime.c
M src/timezone/zic.c
M src/tools/msvc/Install.pm
Add missing error check in pgcrypto/crypt-md5.c.
commit : e15115b4d204989447203b12b905c66dfb6c0a78
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:59:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:59:13 -0400
In theory, the second px_find_digest call in px_crypt_md5 could fail
even though the first one succeeded, since resource allocation is
required. Don't skip testing for a failure. (If one did happen,
the likely result would be a crash rather than clean recovery from
an OOM failure.)
The code's been like this all along, so back-patch to all supported
branches.
Daniel Gustafsson
Discussion: https://postgr.es/m/AA8D6FE9-4AB2-41B4-98CB-AE64BA668C03@yesql.se
M contrib/pgcrypto/crypt-md5.c
Doc: tweak column widths in synchronous-commit-matrix table.
commit : 35b12948ac880a3adcf61be1297809b9c19f34d3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:36:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:36:34 -0400
Commit a97e85f2b caused "exceed the available area" warnings in PDF
builds. Fine-tune colwidth values to avoid that.
Back-patch to 9.6, like the prior patch. (This is of dubious value
before v13, since we were far from free of such warnings in older
branches. But we might as well keep the SGML looking the same in all
branches.)
Per buildfarm.
M doc/src/sgml/config.sgml
pg_upgrade: remove C99 compiler req. from commit 3c0471b5fd
commit : 9dcffe69a93183bba607527279270eac486463e8
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 20:37:19 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 20:37:19 -0400
This commit required support for inline variable definition, which is
not a requirement.
RELEASE NOTE AUTHOR: the author of commit 3c0471b5fd
(pg_upgrade/tablespaces) was Justin Pryzby, not me.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20201016001959.h24fkywfubkv2pc5@alap3.anarazel.de
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
pg_upgrade: generate check error for left-over new tablespace
commit : 39c23c199d0aec58d0018a2ca52cbccbf8ea18e4
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 19:33:36 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 19:33:36 -0400
Previously, if pg_upgrade failed, and the user recreated the cluster but
did not remove the new cluster tablespace directory, a later pg_upgrade
would fail since the new tablespace directory would already exists.
This adds error reporting for this during check.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20200925005531.GJ23631@telsasoft.com
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
doc: improve description of synchronous_commit modes
commit : ee59f669b666e62b7e769ece97ef74ef621acd8d
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 15:15:28 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 15:15:28 -0400
Previously it wasn't clear exactly what each of the synchronous_commit
modes accomplished. This clarifies that, and adds a table describing it.
Only backpatched through 9.6 since 9.5 doesn't have all the options.
Reported-by: kghost0@gmail.com
Discussion: https://postgr.es/m/159741195522.14321.13812604195366728976@wrigleys.postgresql.org
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M src/include/access/xact.h
In the postmaster, rely on the signal infrastructure to block signals.
commit : 7753ca49d3580454fc55df0ef77cfa87621b56e3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Oct 2020 12:50:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 15 Oct 2020 12:50:57 -0400
POSIX sigaction(2) can be told to block a set of signals while a
signal handler executes. Make use of that instead of manually
blocking and unblocking signals in the postmaster's signal handlers.
This should save a few cycles, but more importantly it prevents
recursive invocation of signal handlers when many signals arrive in
close succession. (Assuming that the platform's signal infrastructure
is designed to avoid consuming stack space in that case, but this is
demonstrably true at least on Linux.) The existing code has been seen
to recurse to the point of stack overflow, either in the postmaster
or in a forked-off child.
Back-patch of commit 9abb2bfc0. At the time, we'd only seen excess
postmaster stack consumption in the buildfarm; but we now have a
user report of it, and that commit has aged enough to have a fair
amount of confidence that it doesn't break anything.
This still doesn't change anything about the way that it works on
Windows. Perhaps someone else would like to fix that?
Per bug #16673 from David Geier. Back-patch to 9.6. Although
the problem exists in principle before that, we've only seen it
actually materialize in connection with heavy use of parallel
workers, so it doesn't seem necessary to do anything in 9.5;
and the relevant code is different there, too.
Discussion: https://postgr.es/m/16673-d278c604f8e34ec0@postgresql.org
Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
M src/backend/libpq/pqsignal.c
M src/backend/postmaster/postmaster.c
M src/include/libpq/pqsignal.h
M src/include/port.h
M src/port/pqsignal.c
Fix memory leak when guc.c decides a setting can't be applied now.
commit : c7573ab1eca766591ca6fbb1dc64067540094c62
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Oct 2020 13:31:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Oct 2020 13:31:24 -0400
The prohibitValueChange code paths in set_config_option(), which
are executed whenever we re-read a PGC_POSTMASTER variable from
postgresql.conf, neglected to free anything before exiting. Thus
we'd leak the proposed new value of a PGC_STRING variable, as noted
by BoChen in bug #16666. For all variable types, if the check hook
creates an "extra" chunk, we'd also leak that.
These are malloc not palloc chunks, so there is no mechanism for
recovering the leaks before process exit. Fortunately, the values
are typically not very large, meaning you'd have to go through an
awful lot of SIGHUP configuration-reload cycles to make the leakage
amount to anything. Still, for a long-lived postmaster process it
could potentially be a problem.
Oversight in commit 2594cf0e8. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16666-2c41a4eec61b03e1@postgresql.org
M src/backend/utils/misc/guc.c
Fix optimization hazard in gram.y's makeOrderedSetArgs(), redux.
commit : eb48619d01d77d8273f4effa64f7ad544db8bb35
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 18:41:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 18:41:39 -0400
It appears that commit cf63c641c, which intended to prevent
misoptimization of the result-building step in makeOrderedSetArgs,
didn't go far enough: buildfarm member hornet's version of xlc
is now optimizing back to the old, broken behavior in which
list_length(directargs) is fetched only after list_concat() has
changed that value. I'm not entirely convinced whether that's
an undeniable compiler bug or whether it can be justified by a
sufficiently aggressive interpretation of C sequence points.
So let's just change the code to make it harder to misinterpret.
Back-patch to all supported versions, just in case.
Discussion: https://postgr.es/m/1830491.1601944935@sss.pgh.pa.us
M src/backend/parser/gram.y
Rethink recent fix for pg_dump's handling of extension config tables.
commit : 710c0a66d6e9cc2817f6b7e8ad694498bafca843
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 12:50:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 12:50:55 -0400
Commit 3eb3d3e78 was a few bricks shy of a load: while it correctly
set the table's "interesting" flag when deciding to dump the data of
an extension config table, it was not correct to clear that flag
if we concluded we shouldn't dump the data. This led to the crash
reported in bug #16655, because in fact we'll traverse dumpTableSchema
anyway for all extension tables (to see if they have user-added
seclabels or RLS policies).
The right thing to do is to force "interesting" true in makeTableDataInfo,
and otherwise leave the flag alone. (Doing it there is more future-proof
in case additional calls are added, and it also avoids setting the flag
unnecessarily if that function decides the table is non-dumpable.)
This investigation also showed that while only the --inserts code path
had an obvious failure in the case considered by 3eb3d3e78, the COPY
code path also has a problem with not having loaded table subsidiary
data. That causes fmtCopyColumnList to silently return an empty string
instead of the correct column list. That accidentally mostly works,
which perhaps is why we didn't notice this before. It would only fail
if the restore column order is different from the dump column order,
which only happens in weird inheritance cases, so it's not surprising
nobody had hit the case with an extension config table. Nonetheless,
it's a bug, and it goes a long way back, not just to v12 where the
--inserts code path started to have a problem with this.
In hopes of catching such cases a bit sooner in future, add some
Asserts that "interesting" has been set in both dumpTableData and
dumpTableSchema. Adjust the test case added by 3eb3d3e78 so that it
checks the COPY rather than INSERT form of that bug, allowing it to
detect the longer-standing symptom.
Per bug #16655 from Cameron Daniel. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/16655-5c92d6b3a9438137@postgresql.org
Discussion: https://postgr.es/m/18048b44-3414-b983-8c7c-9165b177900d@2ndQuadrant.com
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
pg_upgrade: remove pre-8.4 code and >= 8.4 check
commit : 700b7002f2727cd1f9e01c5def6023b78017b958
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 14:31:21 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 14:31:21 -0400
We only support upgrading from >= 8.4 so no need for this code or tests.
Reported-by: Magnus Hagander
Discussion: https://postgr.es/m/CABUevEx-D0PNVe00tkeQRGennZQwDtBJn=493MJt-x6sppbUxA@mail.gmail.com
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/relfilenode.c
pg_upgrade; change major version comparisons to use <=, not <
commit : 0e78dcaf73b6b9d4c74b858ebb4a0464b7c726de
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 12:12:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 12:12:09 -0400
This makes checking for older major versions more consistent.
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/function.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/server.c
doc: show functions returning record types and use of ROWS FROM
commit : 997188b6e0a11c60ddab369969ea2e57202941b8
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 Oct 2020 16:27:33 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 Oct 2020 16:27:33 -0400
Previously it was unclear exactly how ROWS FROM behaved and how to cast
the data types of columns returned by FROM functions. Also document
that only non-OUT record functions can have their columns cast to data
types.
Reported-by: guyren@gmail.com
Discussion: https://postgr.es/m/158638264419.662.2482095087061084020@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/queries.sgml
Fix two latent(?) bugs in equivclass.c.
commit : f6a23344f8c8ec155399375d048a890afaa575ab
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Oct 2020 13:15:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Oct 2020 13:15:39 -0400
get_eclass_for_sort_expr() computes expr_relids and nullable_relids
early on, even though they won't be needed unless we make a new
EquivalenceClass, which we often don't. Aside from the probably-minor
inefficiency, there's a memory management problem: these bitmapsets will
be built in the caller's context, leading to dangling pointers if that
is shorter-lived than root->planner_cxt. This would be a live bug if
get_eclass_for_sort_expr() could be called with create_it = true during
GEQO join planning. So far as I can find, the core code never does
that, but it's hard to be sure that no extensions do, especially since
the comments make it clear that that's supposed to be a supported case.
Fix by not computing these values until we've switched into planner_cxt
to build the new EquivalenceClass.
generate_join_implied_equalities() uses inner_rel->relids to look up
relevant eclasses, but it ought to be using nominal_inner_relids.
This is presently harmless because a child RelOptInfo will always have
exactly the same eclass_indexes as its topmost parent; but that might
not be true forever, and anyway it makes the code confusing.
The first of these is old (introduced by me in f3b3b8d5b), so back-patch
to all supported branches. The second only dates to v13, but we might
as well back-patch it to keep the code looking similar across branches.
Discussion: https://postgr.es/m/1508010.1601832581@sss.pgh.pa.us
M src/backend/optimizer/path/equivclass.c
doc: libpq connection options can override command-line flags
commit : cf42550d513169b3687785015d241c87b9cf87cd
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 22:19:30 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 22:19:30 -0400
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16486-b9c93d71c02c4907@postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_isready.sgml
M doc/src/sgml/ref/pg_recvlogical.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
doc: clarify the use of ssh port forwarding
commit : d449ec6d52fc257e9f35b9841dbe98b92845faee
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 21:39:33 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 21:39:33 -0400
Reported-by: karimelghazouly@gmail.com
Discussion: https://postgr.es/m/159854511172.24991.4373145230066586863@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/runtime.sgml
Fix handling of BC years in to_date/to_timestamp.
commit : 19e7982681df74e20aa062d5605cb0f7e04c5d51
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Sep 2020 15:40:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Sep 2020 15:40:23 -0400
Previously, a conversion such as
to_date('-44-02-01','YYYY-MM-DD')
would result in '0045-02-01 BC', as the code attempted to interpret
the negative year as BC, but failed to apply the correction needed
for our internal handling of BC years. Fix the off-by-one problem.
Also, arrange for the combination of a negative year and an
explicit "BC" marker to cancel out and produce AD. This is how
the negative-century case works, so it seems sane to do likewise.
Continue to read "year 0000" as 1 BC. Oracle would throw an error,
but we've accepted that case for a long time so I'm hesitant to
change it in a back-patch.
Per bug #16419 from Saeed Hubaishan. Back-patch to all supported
branches.
Dar Alathar-Yemen and Tom Lane
Discussion: https://postgr.es/m/16419-d8d9db0a7553f01b@postgresql.org
M doc/src/sgml/func.sgml
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
Archive timeline history files in standby if archive_mode is set to "always".
commit : 7be02a3bf03a8bec605d089f34ceb5f441e754a6
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 29 Sep 2020 16:21:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 29 Sep 2020 16:21:46 +0900
Previously the standby server didn't archive timeline history files
streamed from the primary even when archive_mode is set to "always",
while it archives the streamed WAL files. This could cause the PITR to
fail because there was no required timeline history file in the archive.
The cause of this issue was that walreceiver didn't mark those files as
ready for archiving.
This commit makes walreceiver mark those streamed timeline history
files as ready for archiving if archive_mode=always. Then the archiver
process archives the marked timeline history files.
Back-patch to all supported versions.
Reported-by: Grigory Smolkin
Author: Grigory Smolkin, Fujii Masao
Reviewed-by: David Zhang, Anastasia Lubennikova
Discussion: https://postgr.es/m/54b059d4-2b48-13a4-6f43-95a087c92367@postgrespro.ru
M doc/src/sgml/high-availability.sgml
M src/backend/replication/walreceiver.c
Revise RelationBuildRowSecurity() to avoid memory leaks.
commit : acab757ce9ead47a39b4d2d25b91ed2532b8444a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Sep 2020 16:04:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Sep 2020 16:04:06 -0400
This function leaked some memory while loading qual clauses for
an RLS policy. While ordinarily negligible, that could build up
in some repeated-reload cases, as reported by Konstantin Knizhnik.
We can improve matters by borrowing the coding long used in
RelationBuildRuleLock: build stringToNode's result directly in
the target context, and remember to explicitly pfree the
input string.
This patch by no means completely guarantees zero leaks within
this function, since we have no real guarantee that the catalog-
reading subroutines it calls don't leak anything. However,
practical tests suggest that this is enough to resolve the issue.
In any case, any remaining leaks are similar to those risked by
RelationBuildRuleLock and other relcache-loading subroutines.
If we need to fix them, we should adopt a more global approach
such as that used by the RECOVER_RELATION_BUILD_MEMORY hack.
While here, let's remove the need for an expensive PG_TRY block by
using MemoryContextSetParent to reparent an initially-short-lived
context for the RLS data.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/21356c12-8917-8249-b35f-1c447231922b@postgrespro.ru
M src/backend/commands/policy.c
Fix handling of -d "connection string" in pg_dump/pg_restore.
commit : 7c154f2fd246f7a5be40f3c6780b84e5eaba08de
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Sep 2020 18:19:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Sep 2020 18:19:39 -0400
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_restore.c
Fix missing fsync of SLRU directories.
commit : d38e15979ba484380cc3c54648e4985d44fb0b5e
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 24 Sep 2020 09:26:09 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 24 Sep 2020 09:26:09 +1200
Harmonize behavior by moving reponsibility for fsyncing directories down
into slru.c. In 10 and later, only the multixact directories were
missed (see commit 1b02be21), and in older branches all SLRUs were
missed.
Back-patch to all supported releases.
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKGLtsTUOScnNoSMZ-2ZLv%2BwGh01J6kAo_DM8mTRq1sKdSQ%40mail.gmail.com
M src/backend/access/transam/slru.c
Avoid possible dangling-pointer access in tsearch_readline_callback.
commit : b1fbb7d08f22290d63d7918274d55a5b8e364be3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Sep 2020 11:36:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Sep 2020 11:36:13 -0400
tsearch_readline() saves the string pointer it returns to the caller
for possible use in the associated error context callback. However,
the caller will usually pfree that string sometime before it next
calls tsearch_readline(), so that there is a window where an ereport
will try to print an already-freed string.
The built-in users of tsearch_readline() happen to all do that pfree
at the bottoms of their loops, so that the window is effectively
empty for them. However, this is not documented as a requirement,
and contrib/dict_xsyn doesn't do it like that, so it seems likely
that third-party dictionaries might have live bugs here.
The practical consequences of this seem pretty limited in any case,
since production builds wouldn't clobber the freed string immediately,
besides which you'd not expect syntax errors in dictionary files
being used in production. Still, it's clearly a bug waiting to bite
somebody.
Fix by pstrdup'ing the string to be saved for the error callback,
and then pfree'ing it next time through. It's been like this for
a long time, so back-patch to all supported branches.
Discussion: https://postgr.es/m/48A4FA71-524E-41B9-953A-FD04EF36E2E7@yesql.se
M src/backend/tsearch/ts_locale.c
Use factorial rather than numeric_fac in create_operator.sql.
commit : 1e00333ffa2eebd4cd76526dbd087bee3b96a015
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Sep 2020 18:03:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Sep 2020 18:03:44 -0400
These two SQL functions are aliases for the same C function, so this
change has no semantic effect. However, because we dropped the
numeric_fac alias in HEAD (commit 76f412ab3), operator definitions
based on that one don't port forward, causing problems for cross-version
upgrade tests based on the regression database.
Patch all active back branches to dodge the problem.
Discussion: https://postgr.es/m/449144.1600439950@sss.pgh.pa.us
M src/test/regress/expected/create_operator.out
M src/test/regress/sql/create_operator.sql
Fix race in test of pg_switch_wal().
commit : e09391a0a042a631de05faa195aea86acc66ba43
author : Noah Misch <noah@leadboat.com>
date : Sun, 13 Sep 2020 23:13:44 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 13 Sep 2020 23:13:44 -0700
The test failed when something added WAL between pg_switch_wal() and
pg_current_wal_lsn(), seen on buildfarm members hornet and sungazer.
Fix v10, v9.6 and v9.5 by making this code mirror its v13+ counterpart.
v12 and v11 lack a counterpart.
M src/test/recovery/t/020_archive_status.pl
Use the properly transformed RangeVar for expandTableLikeClause().
commit : 1a9c93ec3b9ae2766b60e97957d07028e2828796
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 13 Sep 2020 12:51:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 13 Sep 2020 12:51:21 -0400
transformCreateStmt() adjusts the transformed statement's RangeVar
to specify the target schema explicitly, for the express reason
of making sure that auxiliary statements derived by parse
transformation operate on the right table. But the refactoring
I did in commit 502898192 got this wrong and passed the untransformed
RangeVar to expandTableLikeClause(). This could lead to assertion
failures or weird misbehavior if the wrong table was accessed.
Per report from Alexander Lakhin. Like the previous patch, back-patch
to all supported branches.
Discussion: https://postgr.es/m/05051f9d-b32b-cb35-6735-0e9f2ab86b5f@gmail.com
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Use _exit(2) for SIGQUIT during ProcessStartupPacket, too.
commit : dc71c640985fd07b2e2ff3df7e57752a44e828ed
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 12:06:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 12:06:26 -0400
Bring the signal handling for startup-packet collection into line
with the policy established in commits bedadc732 and 8e19a8264,
namely don't risk running atexit callbacks when handling SIGQUIT.
Ideally, we'd not do so for SIGTERM or timeout interrupts either,
but that change seems a bit too risky for the back branches.
For now, just improve the comments in this area to describe the risk.
Also relocate where BackendInitialize re-disables these interrupts,
to minimize the code span where they're active. This doesn't buy
a whole lot of safety, but it can't hurt.
In passing, rename startup_die() to remove confusion about whether
it is for the startup process.
Like the previous commits, back-patch to all supported branches.
Discussion: https://postgr.es/m/1850884.1599601164@sss.pgh.pa.us
M src/backend/postmaster/postmaster.c
Fix title in reference section
commit : 4f737ca15b92abd4081cf400da2d4498154ae106
author : Magnus Hagander <magnus@hagander.net>
date : Thu, 10 Sep 2020 14:15:26 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Thu, 10 Sep 2020 14:15:26 +0200
Reported-by: Robert Kahlert
Author: Daniel Gustafsson
M doc/src/sgml/biblio.sgml
doc: Fix some grammar and inconsistencies
commit : a4c0dbc447c7c42e1483e2d3bcc0856a5b9ba5dd
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Sep 2020 15:51:00 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Sep 2020 15:51:00 +0900
Some comments are fixed while on it.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20200818171702.GK17022@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/sources.sgml
M src/backend/storage/lmgr/proc.c
Make archiver's SIGQUIT handler exit via _exit().
commit : b2eaddd9b00acad80d634ec671235a080269bd72
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 15:32:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 15:32:34 -0400
Commit 8e19a8264 changed the SIGQUIT handlers of almost all server
processes not to run atexit callbacks. The archiver process was
skipped, perhaps because it's not connected to shared memory; but
it's just as true here that running atexit callbacks in a signal
handler is unsafe. So let's make it work like the rest.
In HEAD and v13, we can use the common SignalHandlerForCrashExit
handler. Before that, just tweak pgarch_exit to use _exit(2)
explicitly.
Like the previous commit, back-patch to all supported branches.
Kyotaro Horiguchi, back-patching by me
Discussion: https://postgr.es/m/1850884.1599601164@sss.pgh.pa.us
M src/backend/postmaster/pgarch.c
Use return instead of exit() in configure
commit : 27acbd51e60c98cba6441d7a88210a1c33bf9db4
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 8 Sep 2020 10:09:43 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 8 Sep 2020 10:09:43 +0200
Using exit() requires stdlib.h, which is not included. Use return
instead. Also add return type for main().
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Thomas Munro <thomas.munro@enterprisedb.com>
Backpatched because Apple macOS 10.16/11 (Big Sur) compiler makes
calling undeclared functions an error, so these configure tests would
fail.
Reported-by: Thomas Gilligan <thomas.gilligan@icloud.com>
Reported-by: Jesse Zhang <sbjesse@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/09A4B554-82B1-4536-B191-2461342EE0BB%40icloud.com
M config/c-compiler.m4
M config/c-library.m4
M configure
Fix misleading error message about inconsistent moving-aggregate types.
commit : 2939f613fa91f12b43d51eb8c4e595d5164e5254
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 12:55:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 12:55:13 -0400
We reported the wrong types when complaining that an aggregate's
moving-aggregate implementation is inconsistent with its regular
implementation.
This was wrong since the feature was introduced, so back-patch
to all supported branches.
Jeff Janes
Discussion: https://postgr.es/m/CAMkU=1x808LH=LPhZp9mNSP0Xd1xDqEd+XeGcvEe48dfE6xV=A@mail.gmail.com
M src/backend/catalog/pg_aggregate.c
Remove useless lstat() call in pg_rewind.
commit : 1f8c163c5acdef34dff14615337332cecc21d70b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 11:50:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 11:50:41 -0400
This is duplicative of an lstat that was just done by the calling
function (traverse_datadir), besides which we weren't really doing
anything with the results. There's not much point in checking to
see if someone removed the file since the previous lstat, since the
FILE_ACTION_REMOVE code would have to deal with missing-file cases
anyway. Moreover, the "exists = false" assignment was a dead store;
nothing was done with that value later.
A syscall saved is a syscall earned, so back-patch to 9.5
where this code was introduced.
Discussion: https://postgr.es/m/1221796.1599329320@sss.pgh.pa.us
M src/bin/pg_rewind/filemap.c
C comment: correct use of 64-"byte" cache line size
commit : 7f79f9582185eb7086ac3c59fa79b47a37cd7d7a
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 4 Sep 2020 13:27:52 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 4 Sep 2020 13:27:52 -0400
Reported-by: Kelly Min
Discussion: https://postgr.es/m/CAPSbxatOiQO90LYpSC3+svAU9-sHgDfEP4oFhcEUt_X=DqFA9g@mail.gmail.com
Backpatch-through: 9.5
M src/include/storage/buf_internals.h
Fix rare deadlock failure in create_am regression test.
commit : b2fcaed66a4aa88331b375409ef208b12f97c144
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 12:40:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 12:40:28 -0400
The "DROP ACCESS METHOD gist2" test will require locking the index
to be dropped and then its table; while most ordinary operations
lock a table first then its index. While no concurrent test scripts
should be touching fast_emp4000, autovacuum might chance to be
processing that table when the DROP runs, resulting in a deadlock
failure. This is pretty rare but we see it in the buildfarm from
time to time.
To fix, acquire a lock on fast_emp4000 before issuing the DROP.
Since the point of the exercise is mostly to prevent buildfarm
failures, back-patch to 9.6 where this test was introduced.
Discussion: https://postgr.es/m/839004.1599185607@sss.pgh.pa.us
M src/test/regress/expected/create_am.out
M src/test/regress/sql/create_am.sql
Avoid lockup of a parallel worker when reporting a long error message.
commit : 2500e51e7bdc6c4f01312db276d3460644684112
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Sep 2020 16:52:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Sep 2020 16:52:09 -0400
Because sigsetjmp() will restore the initial state with signals blocked,
the code path in bgworker.c for reporting an error and exiting would
execute that way. Usually this is fairly harmless; but if a parallel
worker had an error message exceeding the shared-memory communication
buffer size (16K) it would lock up, because it would wait for a
resume-sending signal from its parallel leader which it would never
detect.
To fix, just unblock signals at the appropriate point.
This can be shown to fail back to 9.6. The lack of parallel query
infrastructure makes it difficult to provide a simple test case for
9.5; but I'm pretty sure the issue exists in some form there as well,
so apply the code change there too.
Vignesh C, reviewed by Bharath Rupireddy, Robert Haas, and myself
Discussion: https://postgr.es/m/CALDaNm1d1hHPZUg3xU4XjtWBOLCrA+-2cJcLpw-cePZ=GgDVfA@mail.gmail.com
M src/backend/postmaster/bgworker.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
doc: clarify that max_wal_size is "during" checkpoints
commit : 34071aa1a23a949eecd49c6c765e14bf057cacb0
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Sep 2020 17:00:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Sep 2020 17:00:09 -0400
Previous wording was "between".
Reported-by: Pavel Luzanov
Discussion: https://postgr.es/m/26906a54-d7cb-2f8e-eed7-e31660024694@postgrespro.ru
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
Teach libpq to handle arbitrary-length lines in .pgpass files.
commit : ba23174dd9163b9ebc48791f57e45fae727f8922
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Sep 2020 13:14:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Sep 2020 13:14:44 -0400
Historically there's been a hard-wired assumption here that no line of
a .pgpass file could be as long as NAMEDATALEN*5 bytes. That's a bit
shaky to start off with, because (a) there's no reason to suppose that
host names fit in NAMEDATALEN, and (b) this figure fails to allow for
backslash escape characters. However, it fails completely if someone
wants to use a very long password, and we're now hearing reports of
people wanting to use "security tokens" that can run up to several
hundred bytes. Another angle is that the file is specified to allow
comment lines, but there's no reason to assume that long comment lines
aren't possible.
Rather than guessing at what might be a more suitable limit, let's
replace the fixed-size buffer with an expansible PQExpBuffer. That
adds one malloc/free cycle to the typical use-case, but that's surely
pretty cheap relative to the I/O this code has to do.
Also, add TAP test cases to exercise this code, because there was no
test coverage before.
This reverts most of commit 2eb3bc588, as there's no longer a need for
a warning message about overlength .pgpass lines. (I kept the explicit
check for comment lines, though.)
In HEAD and v13, this also fixes an oversight in 74a308cf5: there's not
much point in explicit_bzero'ing the line buffer if we only do so in two
of the three exit paths.
Back-patch to all supported branches, except that the test case only
goes back to v10 where src/test/authentication/ was added.
Discussion: https://postgr.es/m/4187382.1598909041@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
doc: add commas after 'i.e.' and 'e.g.'
commit : 5394b87a30ce1abd65aee3d2ba339135aa58c920
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 18:33:36 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 18:33:36 -0400
This follows the American format,
https://jakubmarian.com/comma-after-i-e-and-e-g/. There is no intention
of requiring this format for future text, but making existing text
consistent every few years makes sense.
Discussion: https://postgr.es/m/20200825183619.GA22369@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/docguide.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/queries.sgml
M doc/src/sgml/recovery-config.sgml
M doc/src/sgml/ref/create_database.sgml
M doc/src/sgml/ref/create_event_trigger.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/grant.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/release-9.6.sgml
M doc/src/sgml/replication-origins.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/sepgsql.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/sslinfo.sgml
M doc/src/sgml/wal.sgml
M doc/src/sgml/xfunc.sgml
M doc/src/sgml/xml2.sgml
C comment: remove mention of use of t_hoff WAL structure member
commit : 160c8a4d382204ca2983aba56cc9e5db7167354e
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:51:31 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:51:31 -0400
Reported-by: Antonin Houska
Discussion: https://postgr.es/m/21643.1595353537@antos
Backpatch-through: 9.5
M src/include/access/heapam_xlog.h
pg_upgrade doc: mention saving postgresql.conf.auto files
commit : cda4397f2bc32976f774bf0ec0b3b7d40a77d64e
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:36:22 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:36:22 -0400
Also mention files included by postgresql.conf.
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/08AD4526-75AB-457B-B2DD-099663F28040@yesql.se
Backpatch-through: 9.5
M doc/src/sgml/ref/pgupgrade.sgml
docs: in mapping SQL to C data types, timestamp isn't a pointer
commit : 511d51acb33c780f3be018ace68d3f7fb963ea63
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:05:53 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:05:53 -0400
It is an int64.
Reported-by: ajulien@shaktiware.fr
Discussion: https://postgr.es/m/159845038271.24995.15682121015698255155@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/xfunc.sgml
doc: cross-link file-fdw and CSV config log sections
commit : e80e7891d35ec94659c3fe8f1d69007d7b05af42
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:59:58 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:59:58 -0400
There is an file-fdw example that reads the server config file, so cross
link them.
Reported-by: Oleg Samoilov
Discussion: https://postgr.es/m/159800192078.2886.10431506404995508950@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
M doc/src/sgml/file-fdw.sgml
docs: clarify intermediate certificate creation instructions
commit : 361217659dc80e8f7763eaba047d19391d908f9d
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:21:03 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:21:03 -0400
Specifically, explain the v3_ca openssl specification.
Discussion: https://postgr.es/m/20200824175653.GA32411@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/runtime.sgml
docs: replace "stable storage" with "durable" in descriptions
commit : aa02d352b53329287b22092ed41a9a9ec23b0ef8
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 15:23:18 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 15:23:18 -0400
For PG, "durable storage" has a clear meaning, while "stable storage"
does not, so use the former.
Discussion: https://postgr.es/m/20200817165222.GA31806@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
doc: improve description of subscripting of arrays
commit : dd00de928c3fe59f48510f6d02e5e9b8d72c8ae5
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:49:17 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:49:17 -0400
It wasn't clear the non-integers are cast to integers for subscripting,
rather than throwing an error.
Reported-by: sean@materialize.io
Discussion: https://postgr.es/m/159538675800.624.7728794628229799531@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/syntax.sgml
docs: improve 'capitals' inheritance example
commit : 4d93773798d68915b3b7d870bdceae9869b62f03
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:43:04 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:43:04 -0400
Adds constraints and improves wording.
Reported-by: 2552891@gmail.com
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/advanced.sgml
Fix docs bug stating file_fdw requires absolute paths
commit : fd0f8a6c3a9e30cfd3d1c1ccfb4bb214639b18b4
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 31 Aug 2020 13:03:54 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 31 Aug 2020 13:03:54 +0200
It has always (since the first commit) worked with relative paths, so
use the same wording as other parts of the documentation.
Author: Bruce Momjian
Discussion: https://postgr.es/m/CABUevExx-hm=cit+A9LeKBH39srvk8Y2tEZeEAj5mP8YfzNKUg@mail.gmail.com
M doc/src/sgml/file-fdw.sgml
Mark factorial operator, and postfix operators in general, as deprecated.
commit : 1ce20826ce80af757658b682c94acb584916e5fe
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 30 Aug 2020 16:03:19 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 30 Aug 2020 16:03:19 -0400
Back-patch key parts of 4c5cf5431 and 6ca547cf7 into stable branches.
I didn't touch pg_description entries here, so it's purely a docs
change; and I didn't fool with any examples either. The main point
is so that anyone who's wondering if factorial() exists in the stable
branches will be reassured.
Mark Dilger and John Naylor, with some adjustments by me
Discussion: https://postgr.es/m/BE2DF53D-251A-4E26-972F-930E523580E9@enterprisedb.com
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/create_operator.sgml
Fix code for re-finding scan position in a multicolumn GIN index.
commit : 0dfec43c829af7e48a159fa5da13f5e48025f985
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Aug 2020 17:36:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Aug 2020 17:36:13 -0400
collectMatchBitmap() needs to re-find the index tuple it was previously
looking at, after transiently dropping lock on the index page it's on.
The tuple should still exist and be at its prior position or somewhere
to the right of that, since ginvacuum never removes tuples but
concurrent insertions could add one. However, there was a thinko in
that logic, to the effect of expecting any inserted tuples to have the
same index "attnum" as what we'd been scanning. Since there's no
physical separation of tuples with different attnums, it's not terribly
hard to devise scenarios where this fails, leading to transient "lost
saved point in index" errors. (While I've duplicated this with manual
testing, it seems impossible to make a reproducible test case with our
available testing technology.)
Fix by just continuing the scan when the attnum doesn't match.
While here, improve the error message used if we do fail, so that it
matches the wording used in btree for a similar case.
collectMatchBitmap()'s posting-tree code path was previously not
exercised at all by our regression tests. While I can't make
a regression test that exhibits the bug, I can at least improve
the code coverage here, so do that. The test case I made for this
is an extension of one added by 4b754d6c1, so it only works in
HEAD and v13; didn't seem worth trying hard to back-patch it.
Per bug #16595 from Jesse Kinkead. This has been broken since
multicolumn capability was added to GIN (commit 27cb66fdf),
so back-patch to all supported branches.
Discussion: https://postgr.es/m/16595-633118be8eef9ce2@postgresql.org
M src/backend/access/gin/ginget.c
docs: client certificates are always sent to the server
commit : 6efa6aa96276297a937945feee777d7902a1deb2
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 25 Aug 2020 09:53:12 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 25 Aug 2020 09:53:12 -0400
They are not "requested" by the server.
Reported-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20200825.155320.986648039251743210.horikyota.ntt@gmail.com
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
Avoid pushing quals down into sub-queries that have grouping sets.
commit : d3701bc8a25bfcc9904e9b11f0a354b1fc0947bd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Aug 2020 14:46:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Aug 2020 14:46:40 -0400
The trouble with doing this is that an apparently-constant subquery
output column isn't really constant if it is a grouping column that
appears in only some of the grouping sets. A qual using such a
column would be subject to incorrect const-folding after push-down,
as seen in bug #16585 from Paul Sivash.
To fix, just disable qual pushdown altogether if the sub-query has
nonempty groupingSets. While we could imagine far less restrictive
solutions, there is not much point in working harder right now,
because subquery_planner() won't move HAVING clauses to WHERE within
such a subquery. If the qual stays in HAVING it's not going to be
a lot more useful than if we'd kept it at the outer level.
Having said that, this restriction could be removed if we used a
parsetree representation that distinguished such outputs from actual
constants, which is something I hope to do in future. Hence, make
the patch a minimal addition rather than integrating it more tightly
(e.g. by renumbering the existing items in subquery_is_pushdown_safe's
comment).
Back-patch to 9.5 where grouping sets were introduced.
Discussion: https://postgr.es/m/16585-9d8c340d23ade8c1@postgresql.org
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
docs: improve description of how to handle multiple databases
commit : 190fb57c4423f2ef64358f090a828e5fc29a1566
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Aug 2020 20:23:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Aug 2020 20:23:09 -0400
This is a redesign of the intro to the managing databases chapter.
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org
Author: David G. Johnston
Backpatch-through: 9.5
M doc/src/sgml/manage-ag.sgml
Fix handling of CREATE TABLE LIKE with inheritance.
commit : 37681d7e942e31256efdd9b562a750f1742ac111
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Aug 2020 15:00:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Aug 2020 15:00:43 -0400
If a CREATE TABLE command uses both LIKE and traditional inheritance,
Vars in CHECK constraints and expression indexes that are absorbed
from a LIKE parent table tended to get mis-numbered, resulting in
wrong answers and/or bizarre error messages (though probably not any
actual crashes, thanks to validation occurring in the executor).
In v12 and up, the same could happen to Vars in GENERATED expressions,
even in cases with no LIKE clause but multiple traditional-inheritance
parents.
The cause of the problem for LIKE is that parse_utilcmd.c supposed
it could renumber such Vars correctly during transformCreateStmt(),
which it cannot since we have not yet accounted for columns added via
inheritance. Fix that by postponing processing of LIKE INCLUDING
CONSTRAINTS, DEFAULTS, GENERATED, INDEXES till after we've performed
DefineRelation().
The error with GENERATED and multiple inheritance is a simple oversight
in MergeAttributes(); it knows it has to renumber Vars in inherited
CHECK constraints, but forgot to apply the same processing to inherited
GENERATED expressions (a/k/a defaults).
Per bug #16272 from Tom Gottfried. The non-GENERATED variants of the
issue are ancient, presumably dating right back to the addition of
CREATE TABLE LIKE; hence back-patch to all supported branches.
Discussion: https://postgr.es/m/16272-6e32da020e9a9381@postgresql.org
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/include/parser/parse_utilcmd.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Disable autovacuum for BRIN test table
commit : d4d21490376e3dc863484b97c60cce08096218ce
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 17 Aug 2020 16:20:05 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 17 Aug 2020 16:20:05 -0400
This should improve stability in the tests.
Per buildfarm member hyrax (CLOBBER_CACHE_ALWAYS) via Tom Lane.
Discussion: https://postgr.es/m/871534.1597503261@sss.pgh.pa.us
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
Doc: fix description of UNION/CASE/etc type unification.
commit : 722564cf0d7cbcd56b05d8945c360f6e56ba715d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Aug 2020 15:40:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Aug 2020 15:40:07 -0400
The description of what select_common_type() does was not terribly
accurate. Improve it.
David Johnston and Tom Lane
Discussion: https://postgr.es/m/1019930.1597613200@sss.pgh.pa.us
M doc/src/sgml/typeconv.sgml
doc: Fix description about bgwriter and checkpoint in HA section
commit : 5b39f772e2c1670eea2cfde0cc8b08183b010d0e
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Aug 2020 10:24:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Aug 2020 10:24:42 +0900
Since 806a2ae, the work of the bgwriter is split the checkpointer, but a
portion of the documentation did not get the message.
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CA+fd4k6jXxjAtjMVC=wG3=QGpauZBtcgN3Jhw+oV7zXGKVLKzQ@mail.gmail.com
Backpatch-through: 9.5
M doc/src/sgml/high-availability.sgml
Move new LOCKTAG_DATABASE_FROZEN_IDS to end of enum LockTagType.
commit : 85aa22cb120d0ee91d41dfa27d307a95f01df89f
author : Noah Misch <noah@leadboat.com>
date : Sat, 15 Aug 2020 16:15:59 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 15 Aug 2020 16:15:59 -0700
Several PGXN modules reference LockTagType values; renumbering would
force a recompile of those modules. Oversight in back-patch of today's
commit 566372b3d6435639e4cc4476d79b8505a0297c87. Back-patch to released
branches, v12 through 9.5.
Reported by Tom Lane.
Discussion: https://postgr.es/m/921383.1597523945@sss.pgh.pa.us
M src/backend/utils/adt/lockfuncs.c
M src/include/storage/lock.h
Prevent concurrent SimpleLruTruncate() for any given SLRU.
commit : 5ae01df6f6ef590fd73299050c7112f69c03f3de
author : Noah Misch <noah@leadboat.com>
date : Sat, 15 Aug 2020 10:15:53 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 15 Aug 2020 10:15:53 -0700
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/commands/vacuum.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/utils/adt/lockfuncs.c
M src/include/storage/lmgr.h
M src/include/storage/lock.h
Be more careful about the shape of hashable subplan clauses.
commit : 7241edc4fa8cda23d04bfe3cd139ed27904eac04
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 22:14:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 22:14:03 -0400
nodeSubplan.c expects that the testexpr for a hashable ANY SubPlan
has the form of one or more OpExprs whose LHS is an expression of the
outer query's, while the RHS is an expression over Params representing
output columns of the subquery. However, the planner only went as far
as verifying that the clauses were all binary OpExprs. This works
99.99% of the time, because the clauses have the right shape when
emitted by the parser --- but it's possible for function inlining to
break that, as reported by PegoraroF10. To fix, teach the planner
to check that the LHS and RHS contain the right things, or more
accurately don't contain the wrong things. Given that this has been
broken for years without anyone noticing, it seems sufficient to just
give up hashing when it happens, rather than go to the trouble of
commuting the clauses back again (which wouldn't necessarily work
anyway).
While poking at that, I also noticed that nodeSubplan.c had a baked-in
assumption that the number of hash clauses is identical to the number
of subquery output columns. Again, that's fine as far as parser output
goes, but it's not hard to break it via function inlining. There seems
little reason for that assumption though --- AFAICS, the only thing
it's buying us is not having to store the number of hash clauses
explicitly. Adding code to the planner to reject such cases would take
more code than getting nodeSubplan.c to cope, so I fixed it that way.
This has been broken for as long as we've had hashable SubPlans,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/1549209182255-0.post@n3.nabble.com
M src/backend/executor/nodeSubplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix postmaster's behavior during smart shutdown.
commit : b4d8be9f6dc892bd69cf99b587a7d6f925d43ca3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 13:26:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 13:26:57 -0400
Up to now, upon receipt of a SIGTERM ("smart shutdown" command), the
postmaster has immediately killed all "optional" background processes,
and subsequently refused to launch new ones while it's waiting for
foreground client processes to exit. No doubt this seemed like an OK
policy at some point; but it's a pretty bad one now, because it makes
for a seriously degraded environment for the remaining clients:
* Parallel queries are killed, and new ones fail to launch. (And our
parallel-query infrastructure utterly fails to deal with the case
in a reasonable way --- it just hangs waiting for workers that are
not going to arrive. There is more work needed in that area IMO.)
* Autovacuum ceases to function. We can tolerate that for awhile,
but if bulk-update queries continue to run in the surviving client
sessions, there's eventually going to be a mess. In the worst case
the system could reach a forced shutdown to prevent XID wraparound.
* The bgwriter and walwriter are also stopped immediately, likely
resulting in performance degradation.
Hence, let's rearrange things so that the only immediate change in
behavior is refusing to let in new normal connections. Once the last
normal connection is gone, shut everything down as though we'd received
a "fast" shutdown. To implement this, remove the PM_WAIT_BACKUP and
PM_WAIT_READONLY states, instead staying in PM_RUN or PM_HOT_STANDBY
while normal connections remain. A subsidiary state variable tracks
whether or not we're letting in new connections in those states.
This also allows having just one copy of the logic for killing child
processes in smart and fast shutdown modes. I moved that logic into
PostmasterStateMachine() by inventing a new state PM_STOP_BACKENDS.
Back-patch to 9.6 where parallel query was added. In principle
this'd be a good idea in 9.5 as well, but the risk/reward ratio
is not as good there, since lack of autovacuum is not a problem
during typical uses of smart shutdown.
Per report from Bharath Rupireddy.
Patch by me, reviewed by Thomas Munro
Discussion: https://postgr.es/m/CALj2ACXAZ5vKxT9P7P89D87i3MDO9bfS+_bjMHgnWJs8uwUOOw@mail.gmail.com
M doc/src/sgml/ref/pg_ctl-ref.sgml
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/libpq-be.h
Handle new HOT chains in index-build table scans
commit : e5902117dd83a330caf68ec2a479c4410cef5518
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Aug 2020 17:33:49 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Aug 2020 17:33:49 -0400
When a table is scanned by heapam_index_build_range_scan (née
IndexBuildHeapScan) and the table lock being held allows concurrent data
changes, it is possible for new HOT chains to sprout in a page that were
unknown when the scan of a page happened. This leads to an error such
as
ERROR: failed to find parent tuple for heap-only tuple at (X,Y) in table "tbl"
because the root tuple was not present when we first obtained the list
of the page's root tuples. This can be fixed by re-obtaining the list
of root tuples, if we see that a heap-only tuple appears to point to a
non-existing root.
This was reported by Anastasia as occurring for BRIN summarization
(which exists since 9.5), but I think it could theoretically also happen
with CREATE INDEX CONCURRENTLY (much older) or REINDEX CONCURRENTLY
(very recent). It seems a happy coincidence that BRIN forces us to
backpatch this all the way to 9.5.
Reported-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Diagnosed-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Co-authored-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/602d8487-f0b2-5486-0088-0f372b2549fa@postgrespro.ru
Backpatch: 9.5 - master
M src/backend/access/heap/pruneheap.c
M src/backend/catalog/index.c
BRIN: Handle concurrent desummarization properly
commit : 7a3c261fbbb427ffa2ee9223728e811556284a6e
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 12 Aug 2020 15:33:36 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 12 Aug 2020 15:33:36 -0400
If a page range is desummarized at just the right time concurrently with
an index walk, BRIN would raise an error indicating index corruption.
This is scary and unhelpful; silently returning that the page range is
not summarized is sufficient reaction.
This bug was introduced by commit 975ad4e602ff as additional protection
against a bug whose actual fix was elsewhere. Backpatch equally.
Reported-By: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Diagnosed-By: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/2588667e-d07d-7e10-74e2-7e1e46194491@postgrespro.ru
Backpatch: 9.5 - master
M src/backend/access/brin/brin_revmap.c