Stamp 10.17.
commit : cffc8fd1ecb09fcd4d8aa98134f4ba8613a01a3a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 16:47:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 16:47:56 -0400
M configure
M configure.in
M doc/bug.template
M src/include/pg_config.h.win32
M src/interfaces/libpq/libpq.rc.in
M src/port/win32ver.rc
Last-minute updates for release notes.
commit : 25387cc56d344eaa9545b8233496296cf6087477
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 13:10:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 13:10:29 -0400
Security: CVE-2021-32027, CVE-2021-32028, CVE-2021-32029
M doc/src/sgml/release-10.sgml
Fix mishandling of resjunk columns in ON CONFLICT ... UPDATE tlists.
commit : 52a4413627319980843bb8f375f28c7f01c45e18
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 11:02:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 11:02:30 -0400
It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE
list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present.
If it happens, the ON CONFLICT UPDATE code path would end up storing
tuples that include the values of the extra resjunk columns. That's
fairly harmless in the short run, but if new columns are added to
the table then the values would become accessible, possibly leading
to malfunctions if they don't match the datatypes of the new columns.
This had escaped notice through a confluence of missing sanity checks,
including
* There's no cross-check that a tuple presented to heap_insert or
heap_update matches the table rowtype. While it's difficult to
check that fully at reasonable cost, we can easily add assertions
that there aren't too many columns.
* The output-column-assignment cases in execExprInterp.c lacked
any sanity checks on the output column numbers, which seems like
an oversight considering there are plenty of assertion checks on
input column numbers. Add assertions there too.
* We failed to apply nodeModifyTable's ExecCheckPlanOutput() to
the ON CONFLICT UPDATE tlist. That wouldn't have caught this
specific error, since that function is chartered to ignore resjunk
columns; but it sure seems like a bad omission now that we've seen
this bug.
In HEAD, the right way to fix this is to make the processing of
ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists
now do, that is don't add "SET x = x" entries, and use
ExecBuildUpdateProjection to evaluate the tlist and combine it with
old values of the not-set columns. This adds a little complication
to ExecBuildUpdateProjection, but allows removal of a comparable
amount of now-dead code from the planner.
In the back branches, the most expedient solution seems to be to
(a) use an output slot for the ON CONFLICT UPDATE projection that
actually matches the target table, and then (b) invent a variant of
ExecBuildProjectionInfo that can be told to not store values resulting
from resjunk columns, so it doesn't try to store into nonexistent
columns of the output slot. (We can't simply ignore the resjunk columns
altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.)
This works back to v10. In 9.6, projections work much differently and
we can't cheaply give them such an option. The 9.6 version of this
patch works by inserting a JunkFilter when it's necessary to get rid
of resjunk columns.
In addition, v11 and up have the reverse problem when trying to
perform ON CONFLICT UPDATE on a partitioned table. Through a
further oversight, adjust_partition_tlist() discarded resjunk columns
when re-ordering the ON CONFLICT UPDATE tlist to match a partition.
This accidentally prevented the storing-bogus-tuples problem, but
at the cost that MULTIEXPR_SUBLINK cases didn't work, typically
crashing if more than one row has to be updated. Fix by preserving
resjunk columns in that routine. (I failed to resist the temptation
to add more assertions there too, and to do some minor code
beautification.)
Per report from Andres Freund. Back-patch to all supported branches.
Security: CVE-2021-32028
M src/backend/access/heap/heapam.c
M src/backend/executor/execExpr.c
M src/backend/executor/execExprInterp.c
M src/backend/executor/nodeModifyTable.c
M src/include/executor/executor.h
M src/test/regress/expected/update.out
M src/test/regress/sql/update.sql
Prevent integer overflows in array subscripting calculations.
commit : 2fb809d3e1927c0885ad80e18dd3a3aacd612b8b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 10:44:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 May 2021 10:44:38 -0400
While we were (mostly) careful about ensuring that the dimensions of
arrays aren't large enough to cause integer overflow, the lower bound
values were generally not checked. This allows situations where
lower_bound + dimension overflows an integer. It seems that that's
harmless so far as array reading is concerned, except that array
elements with subscripts notionally exceeding INT_MAX are inaccessible.
However, it confuses various array-assignment logic, resulting in a
potential for memory stomps.
Fix by adding checks that array lower bounds aren't large enough to
cause lower_bound + dimension to overflow. (Note: this results in
disallowing cases where the last subscript position would be exactly
INT_MAX. In principle we could probably allow that, but there's a lot
of code that computes lower_bound + dimension and would need adjustment.
It seems doubtful that it's worth the trouble/risk to allow it.)
Somewhat independently of that, array_set_element() was careless
about possible overflow when checking the subscript of a fixed-length
array, creating a different route to memory stomps. Fix that too.
Security: CVE-2021-32027
M src/backend/executor/execExprInterp.c
M src/backend/utils/adt/array_userfuncs.c
M src/backend/utils/adt/arrayfuncs.c
M src/backend/utils/adt/arrayutils.c
M src/include/utils/array.h
Translation updates
commit : d215576e2ac3f9c020f7b13110d234dd87caffba
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 14:27:15 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 May 2021 14:27:15 +0200
Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: f89e869abf8730f74d0a75c59436fd02ab7840b1
M src/backend/po/de.po
M src/backend/po/fr.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/fr.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/fr.po
Release notes for 13.3, 12.7, 11.12, 10.17, 9.6.22.
commit : 47f0bc511df52740bf2f3e7cfc594937f1b5a216
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 May 2021 13:31:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 May 2021 13:31:40 -0400
M doc/src/sgml/release-10.sgml
AlterSubscription_refresh: avoid stomping on global variable
commit : 1803c76260006fb8b866bdc8112b9b51f0a32fef
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 7 May 2021 11:46:37 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 7 May 2021 11:46:37 -0400
This patch replaces use of the global "wrconn" variable in
AlterSubscription_refresh with a local variable of the same name, making
it consistent with other functions in subscriptioncmds.c (e.g.
DropSubscription).
The global wrconn is only meant to be used for logical apply/tablesync worker.
Abusing it this way is known to cause trouble if an apply worker
manages to do a subscription refresh, such as reported by Jeremy Finzel
and diagnosed by Andres Freund back in November 2020, at
https://www.postgresql.org/message-id/20201111215820.qihhrz7fayu6myfi@alap3.anarazel.de
Backpatch to 10. In branch master, also move the connection establishment
to occur outside the PG_TRY block; this way we can remove a test for NULL in
PG_FINALLY, and it also makes the code more consistent with similar code in
the same file.
Author: Peter Smith <peter.b.smith@fujitsu.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com
M src/backend/commands/subscriptioncmds.c
Document lock level used by ALTER TABLE VALIDATE CONSTRAINT
commit : f414bd8e4efed6d7fd2a56789cdf56116539b595
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 17:17:56 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 6 May 2021 17:17:56 -0400
Backpatch all the way back to 9.6.
Author: Simon Riggs <simon.riggs@enterprisedb.com>
Discussion: https://postgr.es/m/CANbhV-EwxvdhHuOLdfG2ciYrHOHXV=mm6=fD5aMhqcH09Li3Tg@mail.gmail.com
M doc/src/sgml/ref/alter_table.sgml
Doc: add an example of a self-referential foreign key to ddl.sgml.
commit : 7dcec3ed45d5a128fcc8bce23b9a2879fc520aa0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:37:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:37:57 -0400
While we've always allowed such cases, the documentation didn't
say you could do it.
Discussion: https://postgr.es/m/161969805833.690.13680986983883602407@wrigleys.postgresql.org
M doc/src/sgml/ddl.sgml
Doc: update libpq's documentation for PQfn().
commit : 49172d88f9d03d063e680e45fc51dffea218add1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:10:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 15:10:06 -0400
Mention specifically that you can't call aggregates, window functions,
or procedures this way (the inability to call SRFs was already
mentioned).
Also, the claim that PQfn doesn't support NULL arguments or results
has been a lie since we invented protocol 3.0. Not sure why this
text was never updated for that, but do it now.
Discussion: https://postgr.es/m/2039442.1615317309@sss.pgh.pa.us
M doc/src/sgml/libpq.sgml
Disallow calling anything but plain functions via the fastpath API.
commit : 0627f3630ce780e39aee0dc5881076a6148fcc18
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 14:10:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 30 Apr 2021 14:10:26 -0400
Reject aggregates, window functions, and procedures. Aggregates
failed anyway, though with a somewhat obscure error message.
Window functions would hit an Assert or null-pointer dereference.
Procedures seemed to work as long as you didn't try to do
transaction control, but (a) transaction control is sort of the
point of a procedure, and (b) it's not entirely clear that no
bugs lurk in that path. Given the lack of testing of this area,
it seems safest to be conservative in what we support.
Also reject proretset functions, as the fastpath protocol can't
support returning a set.
Also remove an easily-triggered assertion that the given OID
isn't 0; the subsequent lookups can handle that case themselves.
Per report from Theodor-Arsenij Larionov-Trichkin.
Back-patch to all supported branches. (The procedure angle
only applies in v11+, of course.)
Discussion: https://postgr.es/m/2039442.1615317309@sss.pgh.pa.us
M src/backend/tcop/fastpath.c
Fix some more omissions in pg_upgrade's tests for non-upgradable types.
commit : d5722c92795d6300be731bd7af4b41567e402c79
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Apr 2021 15:24:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Apr 2021 15:24:37 -0400
Commits 29aeda6e4 et al closed up some oversights involving not checking
for non-upgradable types within container types, such as arrays and
ranges. However, I only looked at version.c, failing to notice that
there were substantially-equivalent tests in check.c. (The division
of responsibility between those files is less than clear...)
In addition, because genbki.pl does not guarantee that auto-generated
rowtype OIDs will hold still across versions, we need to consider that
the composite type associated with a system catalog or view is
non-upgradable. It seems unlikely that someone would have a user
column declared that way, but if they did, trying to read it in another
PG version would likely draw "no such pg_type OID" failures, thanks
to the type OID embedded in composite Datums.
To support the composite and reg*-type cases, extend the recursive
query that does the search to allow any base query that returns
a column of pg_type OIDs, rather than limiting it to exactly one
starting type.
As before, back-patch to all supported branches.
Discussion: https://postgr.es/m/2798740.1619622555@sss.pgh.pa.us
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/pg_upgrade.h
M src/bin/pg_upgrade/version.c
Doc: fix discussion of how to get real Julian Dates.
commit : 56e234b6aff9c2e3b97fe647156012b5b175680f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Apr 2021 10:03:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Apr 2021 10:03:28 -0400
Somehow I'd convinced myself that rotating to UTC-12 was the way
to do this, but upon further review, it's definitely UTC+12.
Discussion: https://postgr.es/m/1197050.1619123213@sss.pgh.pa.us
M doc/src/sgml/datetime.sgml
Fix use-after-release issue with pg_identify_object_as_address()
commit : b797918d27cc532b4835d87c676de34d4b9ee9bd
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:58:55 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:58:55 +0900
Spotted by buildfarm member prion, with -DRELCACHE_FORCE_RELEASE.
Introduced in f7aab36.
Discussion: https://postgr.es/m/2759018.1619577848@sss.pgh.pa.us
Backpatch-through: 9.6
M src/backend/catalog/objectaddress.c
Fix pg_identify_object_as_address() with event triggers
commit : 90c9bad307bfabc3a397826349af76b4ca140aca
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:18:28 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 28 Apr 2021 11:18:28 +0900
Attempting to use this function with event triggers failed, as, since
its introduction in a676201, this code has never associated an object
name with event triggers. This addresses the failure by adding the
event trigger name to the set defining its object address.
Note that regression tests are added within event_trigger and not
object_address to avoid issues with concurrent connections in parallel
schedules.
Author: Joel Jacobson
Discussion: https://postgr.es/m/3c905e77-a026-46ae-8835-c3f6cd1d24c8@www.fastmail.com
Backpatch-through: 9.6
M src/backend/catalog/objectaddress.c
M src/test/regress/expected/event_trigger.out
M src/test/regress/sql/event_trigger.sql
Doc: document EXTRACT(JULIAN ...), improve Julian Date explanation.
commit : 64d617de3c595226804e850b8c8a45517f794932
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Apr 2021 11:50:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 26 Apr 2021 11:50:35 -0400
For some reason, the "julian" option for extract()/date_part() has
never gotten listed in the manual. Also, while Appendix B mentioned
in passing that we don't conform to the usual astronomical definition
that a Julian date starts at noon UTC, it was kind of vague about what
we do instead. Clarify that, and add an example showing how to get
the astronomical definition if you want it.
It's been like this for ages, so back-patch to all supported branches.
Discussion: https://postgr.es/m/1197050.1619123213@sss.pgh.pa.us
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
doc: Fix obsolete description about pg_basebackup.
commit : d59964999d15be29fcd97544e66780fc5c9f17a9
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 23 Apr 2021 15:45:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 23 Apr 2021 15:45:46 +0900
Previously it was documented that if using "-X none" option there was
no guarantee that all required WAL files were archived at the end of
pg_basebackup when taking a backup from the standby. But this limitation
was removed by commit 52f8a59dd9. Now, even when taking a backup
from the standby, pg_basebackup can wait for all required WAL files
to be archived. Therefore this commit removes such obsolete
description from the docs.
Also this commit adds new description about the limitation when
taking a backup from the standby, into the docs. The limitation is that
pg_basebackup cannot force the standbfy to switch to a new WAL file
at the end of backup, which may cause pg_basebackup to wait a long
time for the last required WAL file to be switched and archived,
especially when write activity on the primary is low.
Back-patch to v10 where the issue was introduced.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi, Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/20210420.133235.1342729068750553399.horikyota.ntt@gmail.com
M doc/src/sgml/ref/pg_basebackup.sgml
fix silly perl error in commit d064afc720
commit : 246e9f0341ef6215c09f398ea90ae09abdf5d714
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 11:12:04 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 11:12:04 -0400
M src/test/perl/PostgresNode.pm
Only ever test for non-127.0.0.1 addresses on Windows in PostgresNode
commit : a2eb086042ba7b2c50f0708ba302ce83a844529b
author : Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 10:21:22 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Wed, 21 Apr 2021 10:21:22 -0400
This has been found to cause hangs where tcp usage is forced.
Alexey Kodratov
Discussion: https://postgr.es/m/82e271a9a11928337fcb5b5e57b423c0@postgrespro.ru
Backpatch to all live branches
M src/test/perl/PostgresNode.pm
Allow TestLib::slurp_file to skip contents, and use as needed
commit : f8ba416a98ceded2f97bd3aaa06b081c216ac4d7
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 16 Apr 2021 16:54:04 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 16 Apr 2021 16:54:04 -0400
In order to avoid getting old logfile contents certain functions in
PostgresNode were doing one of two things. On Windows it rotated the
logfile and restarted the server, while elsewhere it truncated the log
file. Both of these are unnecessary. We borrow from the buildfarm which
does this instead: note the size of the logfile before we start, and
then when fetching the logfile skip to that position before accumulating
contents. This is spelled differently on Windows but the effect is the
same. This is largely centralized in TestLib's slurp_file function,
which has a new optional parameter, the offset to skip to before
starting to reading the file. Code in the client becomes much neater.
Backpatch to all live branches.
Michael Paquier, slightly modified by me.
Discussion: https://postgr.es/m/YHajnhcMAI3++pJL@paquier.xyz
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix some inappropriately-disallowed uses of ALTER ROLE/DATABASE SET.
commit : 46b6635b7742cea22b981e8f38f651b7f66c5f46
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 15:10:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Apr 2021 15:10:18 -0400
Most GUC check hooks that inspect database state have special checks
that prevent them from throwing hard errors for state-dependent issues
when source == PGC_S_TEST. This allows, for example,
"ALTER DATABASE d SET default_text_search_config = foo" when the "foo"
configuration hasn't been created yet. Without this, we have problems
during dump/reload or pg_upgrade, because pg_dump has no idea about
possible dependencies of GUC values and can't ensure a safe restore
ordering.
However, check_role() and check_session_authorization() hadn't gotten
the memo about that, and would throw hard errors anyway. It's not
entirely clear what is the use-case for "ALTER ROLE x SET role = y",
but we've now heard two independent complaints about that bollixing
an upgrade, so apparently some people are doing it.
Hence, fix these two functions to act more like other check hooks
with similar needs. (But I did not change their insistence on
being inside a transaction, as it's still not apparent that setting
either GUC from the configuration file would be wise.)
Also fix check_temp_buffers, which had a different form of the disease
of making state-dependent checks without any exception for PGC_S_TEST.
A cursory survey of other GUC check hooks did not find any more issues
of this ilk. (There are a lot of interdependencies among
PGC_POSTMASTER and PGC_SIGHUP GUCs, which may be a bad idea, but
they're not relevant to the immediate concern because they can't be
set via ALTER ROLE/DATABASE.)
Per reports from Charlie Hornsby and Nathan Bossart. Back-patch
to all supported branches.
Discussion: https://postgr.es/m/HE1P189MB0523B31598B0C772C908088DB7709@HE1P189MB0523.EURP189.PROD.OUTLOOK.COM
Discussion: https://postgr.es/m/20160711223641.1426.86096@wrigleys.postgresql.org
M src/backend/commands/variable.c
M src/backend/utils/misc/guc.c
Use "-I." in directories holding Bison parsers, for Oracle compilers.
commit : fb9812b6cbdb07313ae8e01c764f8f00d7b36689
author : Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:41 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:41 -0700
With the Oracle Developer Studio 12.6 compiler, #line directives alter
the current source file location for purposes of #include "..."
directives. Hence, a VPATH build failed with 'cannot find include file:
"specscanner.c"'. With two exceptions, parser-containing directories
already add "-I. -I$(srcdir)"; eliminate the exceptions. Back-patch to
9.6 (all supported versions).
M src/test/isolation/Makefile
Port regress-python3-mangle.mk to Solaris "sed".
commit : 72a9bd04780449fcabebce1d09a5225151076b7c
author : Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:21 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 12 Apr 2021 19:24:21 -0700
It doesn't support "\(foo\)*" like a POSIX "sed" implementation does;
see the Autoconf manual. Back-patch to 9.6 (all supported versions).
M src/pl/plpython/regress-python3-mangle.mk
Fix old bug with coercing the result of a COLLATE expression.
commit : 4b0aecee8a7d4ca63075a8bee2d96c457147ad2a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Apr 2021 14:37:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Apr 2021 14:37:22 -0400
There are hacks in parse_coerce.c to push down a requested coercion
to below any CollateExpr that may appear. However, we did that even
if the requested data type is non-collatable, leading to an invalid
expression tree in which CollateExpr is applied to a non-collatable
type. The fix is just to drop the CollateExpr altogether, reasoning
that it's useless.
This bug is ten years old, dating to the original addition of
COLLATE support. The lack of field complaints suggests that there
aren't a lot of user-visible consequences. We noticed the problem
because it would trigger an assertion in DefineVirtualRelation if
the invalid structure appears as an output column of a view; however,
in a non-assert build, you don't see a crash just a (subtly incorrect)
complaint about applying collation to a non-collatable type. I found
that by putting the incorrect structure further down in a view, I could
make a view definition that would fail dump/reload, per the added
regression test case. But CollateExpr doesn't do anything at run-time,
so this likely doesn't lead to any really exciting consequences.
Per report from Yulin Pei. Back-patch to all supported branches.
Discussion: https://postgr.es/m/HK0PR01MB22744393C474D503E16C8509F4709@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/parser/parse_coerce.c
M src/test/regress/expected/collate.out
M src/test/regress/sql/collate.sql
Fix out-of-bound memory access for interval -> char conversion
commit : 1cc110f68dcca496c72292af2dba565f68932ae3
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Apr 2021 11:31:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 12 Apr 2021 11:31:40 +0900
Using Roman numbers (via "RM" or "rm") for a conversion to calculate a
number of months has never considered the case of negative numbers,
where a conversion could easily cause out-of-bound memory accesses. The
conversions in themselves were not completely consistent either, as
specifying 12 would result in NULL, but it should mean XII.
This commit reworks the conversion calculation to have a more
consistent behavior:
- If the number of months and years is 0, return NULL.
- If the number of months is positive, return the exact month number.
- If the number of months is negative, do a backward calculation, with
-1 meaning December, -2 November, etc.
Reported-by: Theodor Arsenij Larionov-Trichkin
Author: Julien Rouhaud
Discussion: https://postgr.es/m/16953-f255a18f8c51f1d5@postgresql.org
backpatch-through: 9.6
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/timestamp.out
M src/test/regress/sql/timestamp.sql
Fix typo
commit : 91a3a7dc63e8fded60b41cd842eba9a8e31f2b5d
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 9 Apr 2021 12:40:14 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 9 Apr 2021 12:40:14 +0200
Author: Daniel Westermann
Backpatch-through: 9.6
Discussion: https://postgr.es/m/GV0P278MB0483A7AA85BAFCC06D90F453D2739@GV0P278MB0483.CHEP278.PROD.OUTLOOK.COM
M src/backend/utils/misc/guc.c
Fix typos and grammar in documentation and code comments
commit : a2b35b3117eba20fc9d0410e0b737180d835595d
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 9 Apr 2021 13:53:33 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 9 Apr 2021 13:53:33 +0900
Comment fixes are applied on HEAD, and documentation improvements are
applied on back-branches where needed.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210408164008.GJ6592@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/ref/createuser.sgml
Don't add non-existent pages to bitmap from BRIN
commit : e4f251be70ac792df79ed22a01bb1f05911984ef
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 7 Apr 2021 15:58:35 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 7 Apr 2021 15:58:35 +0200
The code in bringetbitmap() simply added the whole matching page range
to the TID bitmap, as determined by pages_per_range, even if some of the
pages were beyond the end of the heap. The query then might fail with
an error like this:
ERROR: could not open file "base/20176/20228.2" (target block
262144): previous segment is only 131021 blocks
In this case, the relation has 262093 pages (131072 and 131021 pages),
but we're trying to acess block 262144, i.e. first block of the 3rd
segment. At that point _mdfd_getseg() notices the preceding segment is
incomplete, and fails.
Hitting this in practice is rather unlikely, because:
* Most indexes use power-of-two ranges, so segments and page ranges
align perfectly (segment end is also a page range end).
* The table size has to be just right, with the last segment being
almost full - less than one page range from full segment, so that the
last page range actually crosses the segment boundary.
* Prefetch has to be enabled. The regular page access checks that
pages are not beyond heap end, but prefetch does not. On older
releases (before 12) the execution stops after hitting the first
non-existent page, so the prefetch distance has to be sufficient
to reach the first page in the next segment to trigger the issue.
Since 12 it's enough to just have prefetch enabled, the prefetch
distance does not matter.
Fixed by not adding non-existent pages to the TID bitmap. Backpatch
all the way back to 9.6 (BRIN indexes were introduced in 9.5, but that
release is EOL).
Backpatch-through: 9.6
M src/backend/access/brin/brin.c
Shut down transaction tracking at startup process exit.
commit : b9cf9d7d3bede9e86e75a5b093d76e1dddcbd084
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 02:25:37 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 6 Apr 2021 02:25:37 +0900
Maxim Orlov reported that the shutdown of standby server could result in
the following assertion failure. The cause of this issue was that,
when the shutdown caused the startup process to exit, recovery-time
transaction tracking was not shut down even if it's already initialized,
and some locks the tracked transactions were holding could not be released.
At this situation, if other process was invoked and the PGPROC entry that
the startup process used was assigned to it, it found such unreleased locks
and caused the assertion failure, during the initialization of it.
TRAP: FailedAssertion("SHMQueueEmpty(&(MyProc->myProcLocks[i]))"
This commit fixes this issue by making the startup process shut down
transaction tracking and release all locks, at the exit of it.
Back-patch to all supported branches.
Reported-by: Maxim Orlov
Author: Fujii Masao
Reviewed-by: Maxim Orlov
Discussion: https://postgr.es/m/ad4ce692cc1d89a093b471ab1d969b0b@postgrespro.ru
M src/backend/postmaster/startup.c
M src/backend/storage/ipc/standby.c
Use macro MONTHS_PER_YEAR instead of '12' in /ecpg/pgtypeslib
commit : b4c3a9f32058694e63f053043e11caaae577c9a9
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Apr 2021 16:42:29 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Apr 2021 16:42:29 -0400
All other places already use MONTHS_PER_YEAR appropriately.
Backpatch-through: 9.6
M src/interfaces/ecpg/pgtypeslib/interval.c
Clarify documentation of RESET ROLE
commit : 5ef79d4062b81c69204c58eacfc81bd38641235a
author : Joe Conway <mail@joeconway.com>
date : Fri, 2 Apr 2021 13:48:53 -0400
committer: Joe Conway <mail@joeconway.com>
date : Fri, 2 Apr 2021 13:48:53 -0400
Command-line options, or previous "ALTER (ROLE|DATABASE) ...
SET ROLE ..." commands, can change the value of the default role
for a session. In the presence of one of these, RESET ROLE will
change the current user identifier to the default role rather
than the session user identifier. Fix the documentation to
reflect this reality. Backpatch to all supported versions.
Author: Nathan Bossart
Reviewed-By: Laurenz Albe, David G. Johnston, Joe Conway
Reported by: Nathan Bossart
Discussion: https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com
Backpatch-through: 9.6
M doc/src/sgml/ref/set_role.sgml
doc: Clarify how to generate backup files with non-exclusive backups
commit : 89bd8e8b332841247649551e95c8d22c19a97529
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 2 Apr 2021 16:37:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 2 Apr 2021 16:37:21 +0900
The current instructions describing how to write the backup_label and
tablespace_map files are confusing. For example, opening a file in text
mode on Windows and copy-pasting the file's contents would result in a
failure at recovery because of the extra CRLF characters generated. The
documentation was not stating that clearly, and per discussion this is
not considered as a supported scenario.
This commit extends a bit the documentation to mention that it may be
required to open the file in binary mode before writing its data.
Reported-by: Wang Shenhao
Author: David Steele
Reviewed-by: Andrew Dunstan, Magnus Hagander
Discussion: https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local
Backpatch-through: 9.6
M doc/src/sgml/backup.sgml
doc: mention that intervening major releases can be skipped
commit : 0489953f17dfb7e5b9650b04ba8c25d0d2fe77a9
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 1 Apr 2021 21:17:24 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 1 Apr 2021 21:17:24 -0400
Also mention that you should read the intervening major releases notes.
This change was also applied to the website.
Discussion: https://postgr.es/m/20210330144949.GA8259@momjian.us
Backpatch-through: 9.6
M doc/src/sgml/runtime.sgml
Fix pg_restore's misdesigned code for detecting archive file format.
commit : 1b6961c8fec880525e5a9ca18dc8b27c5c311548
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Apr 2021 13:34:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Apr 2021 13:34:16 -0400
Despite the clear comments pointing out that the duplicative code
segments in ReadHead() and _discoverArchiveFormat() needed to be
in sync, they were not: the latter did not bother to apply any of
the sanity checks in the former. We'd missed noticing this partly
because none of those checks would fail in scenarios we customarily
test, and partly because the oversight would be masked if both
segments execute, which they would in cases other than needing to
autodetect the format of a non-seekable stdin source. However,
in a case meeting all these requirements --- for example, trying
to read a newer-than-supported archive format from non-seekable
stdin --- pg_restore missed applying the version check and would
likely dump core or otherwise misbehave.
The whole thing is silly anyway, because there seems little reason
to duplicate the logic beyond the one-line verification that the
file starts with "PGDMP". There seems to have been an undocumented
assumption that multiple major formats (major enough to require
separate reader modules) would nonetheless share the first half-dozen
fields of the custom-format header. This seems unlikely, so let's
fix it by just nuking the duplicate logic in _discoverArchiveFormat().
Also get rid of the pointless attempt to seek back to the start of
the file after successful autodetection. That wastes cycles and
it means we have four behaviors to verify not two.
Per bug #16951 from Sergey Koposov. This has been broken for
decades, so back-patch to all supported versions.
Discussion: https://postgr.es/m/16951-a4dd68cf0de23048@postgresql.org
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_tar.c
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections
commit : dce3d2610dd627f07ded5a289702c43a6d997495
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 1 Apr 2021 15:29:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 1 Apr 2021 15:29:06 +0900
Some sections of the documentation used "exclusive lock" to describe
that an ACCESS EXCLUSIVE lock is taken during a given operation. This
can be confusing to the reader as ACCESS SHARE is allowed with an
EXCLUSIVE lock is used, but that would not be the case with what is
described on those parts of the documentation.
Author: Greg Rychlewski
Discussion: https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com
Backpatch-through: 9.6
M doc/src/sgml/ddl.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/pgrowlocks.sgml
M doc/src/sgml/ref/drop_index.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/vacuum.sgml
Add a docs section for obsoleted and renamed functions and settings
commit : 58960fca4ef1953220e626aa2bce2446da04bfc7
author : Stephen Frost <sfrost@snowman.net>
date : Wed, 31 Mar 2021 16:23:03 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Wed, 31 Mar 2021 16:23:03 -0400
The new appendix groups information on renamed or removed settings,
commands, etc into an out-of-the-way part of the docs.
The original id elements are retained in each subsection to ensure that
the same filenames are produced for HTML docs. This prevents /current/
links on the web from breaking, and allows users of the web docs
to follow links from old version pages to info on the changes in the
new version. Prior to this change, a link to /current/ for renamed
sections like the recovery.conf docs would just 404. Similarly if
someone searched for recovery.conf they would find the pg11 docs,
but there would be no /12/ or /current/ link, so they couldn't easily
find out that it was removed in pg12 or how to adapt.
Index entries are also added so that there's a breadcrumb trail for
users to follow when they know the old name, but not what we changed it
to. So a user who is trying to find out how to set standby_mode in
PostgreSQL 12+, or where pg_resetxlog went, now has more chance of
finding that information.
Craig Ringer and Stephen Frost
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com
Backpatch-through: 10
A doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml
A doc/src/sgml/appendix-obsolete-pgresetxlog.sgml
A doc/src/sgml/appendix-obsolete-pgxlogdump.sgml
A doc/src/sgml/appendix-obsolete.sgml
M doc/src/sgml/filelist.sgml
M doc/src/sgml/postgres.sgml
Update obsolete comment.
commit : c8d7ea5241308682488bd09d7b15fbe6ff587210
author : Etsuro Fujita <efujita@postgresql.org>
date : Tue, 30 Mar 2021 13:00:06 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Tue, 30 Mar 2021 13:00:06 +0900
Back-patch to all supported branches.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com
M contrib/postgres_fdw/postgres_fdw.c
doc: Define TLS as an acronym
commit : b8be9690735c85d0b91769f4002e494156c60276
author : Stephen Frost <sfrost@snowman.net>
date : Sun, 28 Mar 2021 11:28:20 -0400
committer: Stephen Frost <sfrost@snowman.net>
date : Sun, 28 Mar 2021 11:28:20 -0400
Commit c6763156589 added an acronym reference for "TLS" but the definition
was never added.
Author: Daniel Gustafsson
Reviewed-by: Michael Paquier
Backpatch-through: 9.6
Discussion: https://postgr.es/m/27109504-82DB-41A8-8E63-C0498314F5B0@yesql.se
M doc/src/sgml/acronyms.sgml
Fix ndistinct estimates with system attributes
commit : e5eb40eed470ef98e50787ccefe6d32674437be4
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 22:34:53 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 26 Mar 2021 22:34:53 +0100
When estimating the number of groups using extended statistics, the code
was discarding information about system attributes. This led to strange
situation that
SELECT 1 FROM t GROUP BY ctid;
could have produced higher estimate (equal to pg_class.reltuples) than
SELECT 1 FROM t GROUP BY a, b, ctid;
with extended statistics on (a,b). Fixed by retaining information about
the system attribute.
Backpatch all the way to 10, where extended statistics were introduced.
Author: Tomas Vondra
Backpatch-through: 10
M src/backend/utils/adt/selfuncs.c
Fix bug in WAL replay of COMMIT_TS_SETTS record.
commit : d544671f1572a0f9138f0e4b3aff38bb683483b2
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Mar 2021 11:23:30 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Mar 2021 11:23:30 +0900
Previously the WAL replay of COMMIT_TS_SETTS record called
TransactionTreeSetCommitTsData() with the argument write_xlog=true,
which generated and wrote new COMMIT_TS_SETTS record.
This should not be acceptable because it's during recovery.
This commit fixes the WAL replay of COMMIT_TS_SETTS record
so that it calls TransactionTreeSetCommitTsData() with write_xlog=false
and doesn't generate new WAL during recovery.
Back-patch to all supported branches.
Reported-by: lx zou <zoulx1982@163.com>
Author: Fujii Masao
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/16931-620d0f2fdc6108f1@postgresql.org
M src/backend/access/transam/commit_ts.c
Fix psql's \connect command some more.
commit : d5a905ed5e7b277392db6c07efd40ec2ba9e6341
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Mar 2021 14:27:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 23 Mar 2021 14:27:50 -0400
Jasen Betts reported yet another unintended side effect of commit
85c54287a: reconnecting with "\c service=whatever" did not have the
expected results. The reason is that starting from the output of
PQconndefaults() effectively allows environment variables (such
as PGPORT) to override entries in the service file, whereas the
normal priority is the other way around.
Not using PQconndefaults at all would require yet a third main code
path in do_connect's parameter setup, so I don't really want to fix
it that way. But we can have the logic effectively ignore all the
default values for just a couple more lines of code.
This patch doesn't change the behavior for "\c -reuse-previous=on
service=whatever". That remains significantly different from before
85c54287a, because many more parameters will be re-used, and thus
not be possible for service entries to replace. But I think this
is (mostly?) intentional. In any case, since libpq does not report
where it got parameter values from, it's hard to do differently.
Per bug #16936 from Jasen Betts. As with the previous patches,
back-patch to all supported branches. (9.5 is unfortunately now
out of support, so this won't get fixed there.)
Discussion: https://postgr.es/m/16936-3f524322a53a29f0@postgresql.org
M src/bin/psql/command.c
Use correct spelling of statistics kind
commit : fb742ce2687a9fdb103479950603cfbecbed285e
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 04:54:34 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 23 Mar 2021 04:54:34 +0100
A couple error messages and comments used 'statistic kind', not the
correct 'statistics kind'. Fix and backpack all the way back to 10,
where extended statistics were introduced.
Backpatch-through: 10
M doc/src/sgml/catalogs.sgml
M src/backend/statistics/dependencies.c
M src/backend/statistics/extended_stats.c
M src/backend/statistics/mvdistinct.c
M src/include/nodes/relation.h
pg_waldump: Fix bug in per-record statistics.
commit : 5386a8506d7da3cf1af3fbdb04d7c4f811674574
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 23 Mar 2021 09:53:08 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 23 Mar 2021 09:53:08 +0900
pg_waldump --stats=record identifies a record by a combination
of the RmgrId and the four bits of the xl_info field of the record.
But XACT records use the first bit of those four bits for an optional
flag variable, and the following three bits for the opcode to
identify a record. So previously the same type of XACT record
could have different four bits (three bits are the same but the
first one bit is different), and which could cause
pg_waldump --stats=record to show two lines of per-record statistics
for the same XACT record. This is a bug.
This commit changes pg_waldump --stats=record so that it processes
only XACT record differently, i.e., filters the opcode out of xl_info
and uses a combination of the RmgrId and those three bits as
the identifier of a record, only for XACT record. For other records,
the four bits of the xl_info field are still used.
Back-patch to all supported branches.
Author: Kyotaro Horiguchi
Reviewed-by: Shinya Kato, Fujii Masao
Discussion: https://postgr.es/m/2020100913412132258847@highgo.ca
M src/bin/pg_waldump/pg_waldump.c
Fix new TAP test for 2PC transactions and PITRs on Windows
commit : 6864adc70ed28b097cb1752a6de215b0bc5e03d9
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 09:51:27 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 09:51:27 +0900
The test added by 595b9cb forgot that on Windows it is necessary to set
up pg_hba.conf (see PostgresNode::set_replication_conf) with a specific
entry or base backups fail. Any node that requires to support
replication just needs to pass down allows_streaming at initialization.
This updates the test to do so. Simplify things a bit while on it.
Per buildfarm member fairywren. Any Windows hosts running this test
would have failed, and I have reproduced the problem as well.
Backpatch-through: 10
M src/test/recovery/t/023_pitr_prepared_xact.pl
Fix timeline assignment in checkpoints with 2PC transactions
commit : 1ec7162a8d3494bce97bf52250943143cf6638ad
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 08:31:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 22 Mar 2021 08:31:14 +0900
Any transactions found as still prepared by a checkpoint have their
state data read from the WAL records generated by PREPARE TRANSACTION
before being moved into their new location within pg_twophase/. While
reading such records, the WAL reader uses the callback
read_local_xlog_page() to read a page, that is shared across various
parts of the system. This callback, since 1148e22a, has introduced an
update of ThisTimeLineID when reading a record while in recovery, which
is potentially helpful in the context of cascading WAL senders.
This update of ThisTimeLineID interacts badly with the checkpointer if a
promotion happens while some 2PC data is read from its record, as, by
changing ThisTimeLineID, any follow-up WAL records would be written to
an timeline older than the promoted one. This results in consistency
issues. For instance, a subsequent server restart would cause a failure
in finding a valid checkpoint record, resulting in a PANIC, for
instance.
This commit changes the code reading the 2PC data to reset the timeline
once the 2PC record has been read, to prevent messing up with the static
state of the checkpointer. It would be tempting to do the same thing
directly in read_local_xlog_page(). However, based on the discussion
that has led to 1148e22a, users may rely on the updates of
ThisTimeLineID when a WAL record page is read in recovery, so changing
this callback could break some cases that are working currently.
A TAP test reproducing the issue is added, relying on a PITR to
precisely trigger a promotion with a prepared transaction still
tracked.
Per discussion with Heikki Linnakangas, Kyotaro Horiguchi, Fujii Masao
and myself.
Author: Soumyadeep Chakraborty, Jimmy Yih, Kevin Yeap
Discussion: https://postgr.es/m/CAE-ML+_EjH_fzfq1F3RJ1=XaaNG=-Jz-i3JqkNhXiLAsM3z-Ew@mail.gmail.com
Backpatch-through: 10
M src/backend/access/transam/twophase.c
A src/test/recovery/t/023_pitr_prepared_xact.pl
Fix memory leak when rejecting bogus DH parameters.
commit : ad6c19066d931c66612531d83b15dfd2464e2a29
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 12:47:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 12:47:21 -0400
While back-patching e0e569e1d, I noted that there were some other
places where we ought to be applying DH_free(); namely, where we
load some DH parameters from a file and then reject them as not
being sufficiently secure. While it seems really unlikely that
anybody would hit these code paths in production, let alone do
so repeatedly, let's fix it for consistency.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/16160-18367e56e9a28264@postgresql.org
M src/backend/libpq/be-secure-openssl.c
Fix memory leak when initializing DH parameters in backend
commit : 7d9629ed2f69cd123514e0338cb20afca9abcd59
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 12:38:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 20 Mar 2021 12:38:22 -0400
When loading DH parameters used for the generation of ephemeral DH keys
in the backend, the code has never bothered releasing the memory used
for the DH information loaded from a file or from libpq's default. This
commit makes sure that the information is properly free()'d.
Back-patch of e0e569e1d. We originally thought the leak was minor and
not worth back-patching, but Jelte Fennema pointed out that repeated
SIGHUP's can result in very serious bloat of the postmaster, which is
then multiplied by being duplicated into eadh forked child.
Back-patch to v10; the code looked different before c0a15e07c,
and didn't have a leak in the actually-live code paths.
Michael Paquier
Discussion: https://postgr.es/m/16160-18367e56e9a28264@postgresql.org
M src/backend/libpq/be-secure-openssl.c
Don't leak malloc'd error string in libpqrcv_check_conninfo().
commit : ba986b7bc5cf2cc9f586e553f78ab9b2b199fac2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:21:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:21:58 -0400
We leaked the error report from PQconninfoParse, when there was
one. It seems unlikely that real usage patterns would repeat
the failure often enough to create serious bloat, but let's
back-patch anyway to keep the code similar in all branches.
Found via valgrind testing.
Back-patch to v10 where this code was added.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
Don't leak malloc'd strings when a GUC setting is rejected.
commit : 5058e95a6ef99e03dd5e41c9cda531cbed721d53
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:09:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 22:09:41 -0400
Because guc.c prefers to keep all its string values in malloc'd
not palloc'd storage, it has to be more careful than usual to
avoid leaks. Error exits out of string GUC hook checks failed
to clear the proposed value string, and error exits out of
ProcessGUCArray() failed to clear the malloc'd results of
ParseLongOption().
Found via valgrind testing.
This problem is ancient, so back-patch to all supported branches.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/utils/misc/guc.c
Don't leak compiled regex(es) when an ispell cache entry is dropped.
commit : 0b618ddf8bb2fba5492445b56d21647722b8350e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 21:44:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 21:44:43 -0400
The text search cache mechanisms assume that we can clean up
an invalidated dictionary cache entry simply by resetting the
associated long-lived memory context. However, that does not work
for ispell affixes that make use of regular expressions, because
the regex library deals in plain old malloc. Hence, we leaked
compiled regex(es) any time we dropped such a cache entry. That
could quickly add up, since even a fairly trivial regex can use up
tens of kB, and a large one can eat megabytes. Add a memory context
callback to ensure that a regex gets freed when its owning cache
entry is cleared.
Found via valgrind testing.
This problem is ancient, so back-patch to all supported branches.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/tsearch/spell.c
M src/include/tsearch/dicts/spell.h
Don't leak rd_statlist when a relcache entry is dropped.
commit : 2bed650c4841cb63a8cede0db9452664f39bc314
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 20:37:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Mar 2021 20:37:09 -0400
Although these lists are usually NIL, and even when not empty
are unlikely to be large, constant relcache update traffic could
eventually result in visible bloat of CacheMemoryContext.
Found via valgrind testing.
Back-patch to v10 where this field was added.
Discussion: https://postgr.es/m/3816764.1616104288@sss.pgh.pa.us
M src/backend/utils/cache/relcache.c
Prevent buffer overrun in read_tablespace_map().
commit : 2a4c9fd9c77041fc7207c268eaf0155d11f100a2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:10:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Mar 2021 16:10:38 -0400
Robert Foggia of Trustwave reported that read_tablespace_map()
fails to prevent an overrun of its on-stack input buffer.
Since the tablespace map file is presumed trustworthy, this does
not seem like an interesting security vulnerability, but still
we should fix it just in the name of robustness.
While here, document that pg_basebackup's --tablespace-mapping option
doesn't work with tar-format output, because it doesn't. To make it
work, we'd have to modify the tablespace_map file within the tarball
sent by the server, which might be possible but I'm not volunteering.
(Less-painful solutions would require changing the basebackup protocol
so that the source server could adjust the map. That's not very
appetizing either.)
M doc/src/sgml/ref/pg_basebackup.sgml
M src/backend/access/transam/xlog.c
Avoid corner-case memory leak in SSL parameter processing.
commit : 7ce7f2b79890ea1465c654fd0ee5271bcd01b716
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Mar 2021 16:02:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Mar 2021 16:02:50 -0400
After reading the root cert list from the ssl_ca_file, immediately
install it as client CA list of the new SSL context. That gives the
SSL context ownership of the list, so that SSL_CTX_free will free it.
This avoids a permanent memory leak if we fail further down in
be_tls_init(), which could happen if bogus CRL data is offered.
The leak could only amount to something if the CRL parameters get
broken after server start (else we'd just quit) and then the server
is SIGHUP'd many times without fixing the CRL data. That's rather
unlikely perhaps, but it seems worth fixing, if only because the
code is clearer this way.
While we're here, add some comments about the memory management
aspects of this logic.
Noted by Jelte Fennema and independently by Andres Freund.
Back-patch to v10; before commit de41869b6 it doesn't matter,
since we'd not re-execute this code during SIGHUP.
Discussion: https://postgr.es/m/16160-18367e56e9a28264@postgresql.org
M src/backend/libpq/be-secure-openssl.c
Fix race condition in psql \e's detection of file modification.
commit : 8915e79067d666793bcf9e8f5383d41e9ee50f3e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 12:20:15 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 12:20:15 -0500
psql's editing commands decide whether the user has edited the file
by checking for change of modification timestamp. This is probably
fine for a pre-existing file, but with a temporary file that is
created within the command, it's possible for a fast typist to
save-and-exit in less than the one-second granularity of stat(2)
timestamps. On Windows FAT filesystems the granularity is even
worse, 2 seconds, making the race a bit easier to hit.
To fix, try to set the temp file's mod time to be two seconds ago.
It's unlikely this would fail, but then again the race condition
itself is unlikely, so just ignore any error.
Also, we might as well check the file size as well as its mod time.
While this is a difficult bug to hit, it still seems worth
back-patching, to ensure that users' edits aren't lost.
Laurenz Albe, per gripe from Jacob Champion; based on fix suggestions
from Jacob and myself
Discussion: https://postgr.es/m/0ba3f2a658bac6546d9934ab6ba63a805d46a49b.camel@cybertec.at
M src/bin/psql/command.c
Forbid marking an identity column as nullable.
commit : e5794cd593940e48710693c0cdd97c6d7b91f500
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 11:08:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Mar 2021 11:08:42 -0500
GENERATED ALWAYS AS IDENTITY implies NOT NULL, but the code failed
to complain if you overrode that with "GENERATED ALWAYS AS IDENTITY
NULL". One might think the old behavior was a feature, but it was
inconsistent because the outcome varied depending on the order of
the clauses, so it seems to have been just an oversight.
Per bug #16913 from Pavel Boev. Back-patch to v10 where identity
columns were introduced.
Vik Fearing (minor tweaks by me)
Discussion: https://postgr.es/m/16913-3b5198410f67d8c6@postgresql.org
M doc/src/sgml/ref/create_table.sgml
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/identity.out
M src/test/regress/sql/identity.sql
Re-simplify management of inStart in pqParseInput3's subroutines.
commit : d2be6cdc55ad0eb0fa019a453491e1410c5315e8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Mar 2021 14:43:45 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Mar 2021 14:43:45 -0500
Commit 92785dac2 copied some logic related to advancement of inStart
from pqParseInput3 into getRowDescriptions and getAnotherTuple,
because it wanted to allow user-defined row processor callbacks to
potentially longjmp out of the library, and inStart would have to be
updated before that happened to avoid an infinite loop. We later
decided that that API was impossibly fragile and reverted it, but
we didn't undo all of the related code changes, and this bit of
messiness survived. Undo it now so that there's just one place in
pqParseInput3's processing where inStart is advanced; this will
simplify addition of better tracing support.
getParamDescriptions had grown similar processing somewhere along
the way (not in 92785dac2; I didn't track down just when), but it's
actually buggy because its handling of corrupt-message cases seems to
have been copied from the v2 logic where we lacked a known message
length. The cases where we "goto not_enough_data" should not simply
return EOF, because then we won't consume the message, potentially
creating an infinite loop. That situation now represents a
definitively corrupt message, and we should report it as such.
Although no field reports of getParamDescriptions getting stuck in
a loop have been seen, it seems appropriate to back-patch that fix.
I chose to back-patch all of this to keep the logic looking more alike
in supported branches.
Discussion: https://postgr.es/m/2217283.1615411989@sss.pgh.pa.us
M src/interfaces/libpq/fe-protocol3.c
tutorial: land height is "elevation", not "altitude"
commit : 338b9a0d4c999a6dda7797edf0eba764399d3866
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 10 Mar 2021 20:25:18 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 10 Mar 2021 20:25:18 -0500
This is a follow-on patch to 92c12e46d5. In that patch, we renamed
"altitude" to "elevation" in the docs, based on these details:
https://mapscaping.com/blogs/geo-candy/what-is-the-difference-between-elevation-relief-and-altitude
This renames the tutorial SQL files to match the documentation.
Reported-by: max1@inbox.ru
Discussion: https://postgr.es/m/161512392887.1046.3137472627109459518@wrigleys.postgresql.org
Backpatch-through: 9.6
M src/tutorial/advanced.source
Validate the OID argument of pg_import_system_collations().
commit : 37228ecde1f061ce47a2f60fcaa0f64a93af4ea8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:21:51 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 8 Mar 2021 18:21:51 -0500
"SELECT pg_import_system_collations(0)" caused an assertion failure.
With a random nonzero argument --- or indeed with zero, in non-assert
builds --- it would happily make pg_collation entries with garbage
values of collnamespace. These are harmless as far as I can tell
(unless maybe the OID happens to become used for a schema, later on?).
In any case this isn't a security issue, since the function is
superuser-only. But it seems like a gotcha for unwary DBAs, so let's
add a check that the given OID belongs to some schema.
Back-patch to v10 where this function was introduced.
M src/backend/commands/collationcmds.c
Clarify the usage of max_replication_slots on the subscriber side.
commit : 90c737669fd2ab8e02ef7e8200adbce6fccf5c65
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Mar 2021 11:49:12 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 3 Mar 2021 11:49:12 +0530
It was not clear in the docs that the max_replication_slots is also used
to track replication origins on the subscriber side.
Author: Paul Martinez
Reviewed-by: Amit Kapila
Backpatch-through: 10 where logical replication was introduced
Discussion: https://postgr.es/m/CACqFVBZgwCN_pHnW6dMNCrOS7tiHCw6Retf_=U2Vvj3aUSeATw@mail.gmail.com
M doc/src/sgml/config.sgml
M doc/src/sgml/logical-replication.sgml
Use native path separators to pg_ctl in initdb
commit : 926139dd04bb77ad3c18c9c69544104d15f69672
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 2 Mar 2021 15:39:34 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 2 Mar 2021 15:39:34 -0300
On Windows, CMD.EXE allegedly does not run a command that uses forward slashes,
so let's convert the path to use backslashes instead.
Backpatch to 10.
Author: Nitin Jadhav <nitinjadhavpostgres@gmail.com>
Reviewed-by: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
Discussion: https://postgr.es/m/CAMm1aWaNDuaPYFYMAqDeJrZmPtNvLcJRS++CcZWY8LT6KcoBZw@mail.gmail.com
M src/bin/initdb/initdb.c
Doc: further clarify libpq's description of connection string URIs.
commit : 8386f5840ac08a70a649abc80679ad7c2127b7d7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Feb 2021 15:24:01 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Feb 2021 15:24:01 -0500
Break the synopsis into named parts to make it less confusing.
Make more than zero effort at applying SGML markup. Do a bit
of copy-editing of nearby text.
The synopsis revision is by Alvaro Herrera and Paul Förster,
the rest is my fault. Back-patch to v10 where multi-host
connection strings appeared.
Discussion: https://postgr.es/m/6E752D6B-487C-463E-B6E2-C32E7FB007EA@gmail.com
M doc/src/sgml/libpq.sgml
Fix some typos, grammar and style in docs and comments
commit : a8f26f6c8882170d1fa7b86f2d83a314ee746e7e
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Feb 2021 16:14:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 24 Feb 2021 16:14:08 +0900
The portions fixing the documentation are backpatched where needed.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20210210235557.GQ20012@telsasoft.com
backpatch-through: 9.6
M doc/src/sgml/charset.sgml
M doc/src/sgml/pageinspect.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/create_type.sgml
M doc/src/sgml/ref/drop_index.sgml
M doc/src/sgml/rules.sgml
Reinstate HEAP_XMAX_LOCK_ONLY|HEAP_KEYS_UPDATED as allowed
commit : bf518fefacc74a75d67a492ebee8d7a585989a34
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 17:30:21 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 23 Feb 2021 17:30:21 -0300
Commit 866e24d47db1 added an assert that HEAP_XMAX_LOCK_ONLY and
HEAP_KEYS_UPDATED cannot appear together, on the faulty assumption that
the latter necessarily referred to an update and not a tuple lock; but
that's wrong, because SELECT FOR UPDATE can use precisely that
combination, as evidenced by the amcheck test case added here.
Remove the Assert(), and also patch amcheck's verify_heapam.c to not
complain if the combination is found. Also, out of overabundance of
caution, update (across all branches) README.tuplock to be more explicit
about this.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Mahendra Singh Thalor <mahi6run@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Discussion: https://postgr.es/m/20210124061758.GA11756@nol
M src/backend/access/heap/README.tuplock
Fix another ancient bug in parsing of BRE-mode regular expressions.
commit : b0645097962575eee3546d7a65420f7c78725cec
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Feb 2021 22:38:55 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Feb 2021 22:38:55 -0500
While poking at the regex code, I happened to notice that the bug
squashed in commit afcc8772e had a sibling: next() failed to return
a specific value associated with the '}' token for a "\{m,n\}"
quantifier when parsing in basic RE mode. Again, this could result
in treating the quantifier as non-greedy, which it never should be in
basic mode. For that to happen, the last character before "\}" that
sets "nextvalue" would have to set it to zero, or it'd have to have
accidentally been zero from the start. The failure can be provoked
repeatably with, for example, a bound ending in digit "0".
Like the previous patch, back-patch all the way.
M src/backend/regex/regc_lex.c
Fix compiler warning in back branches (9.6, 10).
commit : 26812bcaa664b17843b3dbc8803551d720109b6e
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 16 Feb 2021 13:10:37 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 16 Feb 2021 13:10:37 +1300
Back-patch a tiny bit of commit fbb2e9a0 into 9.6 and 10, to silence an
uninitialized variable warning from GCC 10.2. Seen on buildfarm member
handfish, and my own development workflow where I like to use -Werror.
Discussion: https://postgr.es/m/CA%2BhUKGJRcwvK86Uf5t-FrTekZjqHtpv3u%3D3MuBg8Zw8R933Mqg%40mail.gmail.com
M src/backend/utils/misc/guc.c
Default to wal_sync_method=fdatasync on FreeBSD.
commit : 800131df74c4b870b6a459bcee0acc0bb89f75ff
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 15:43:39 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 15:43:39 +1300
FreeBSD 13 gained O_DSYNC, which would normally cause wal_sync_method to
choose open_datasync as its default value. That may not be a good
choice for all systems, and performs worse than fdatasync in some
scenarios. Let's preserve the existing default behavior for now.
Like commit 576477e73c4, which did the same for Linux, back-patch to all
supported releases.
Discussion: https://postgr.es/m/CA%2BhUKGLsAMXBQrCxCXoW-JsUYmdOL8ALYvaX%3DCrHqWxm-nWbGA%40mail.gmail.com
M doc/src/sgml/config.sgml
M src/backend/utils/misc/postgresql.conf.sample
M src/include/port/freebsd.h
Hold interrupts while running dsm_detach() callbacks.
commit : 4b426f77c3cf7fab24115ddb99174d1efa311aee
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 13:32:58 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Feb 2021 13:32:58 +1300
While cleaning up after a parallel query or parallel index creation that
created temporary files, we could be interrupted by a statement timeout.
The error handling path would then fail to clean up the files when it
ran dsm_detach() again, because the callback was already popped off the
list. Prevent this hazard by holding interrupts while the cleanup code
runs.
Thanks to Heikki Linnakangas for this suggestion, and also to Kyotaro
Horiguchi, Masahiko Sawada, Justin Pryzby and Tom Lane for discussion of
this and earlier ideas on how to fix the problem.
Back-patch to all supported releases.
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20191212180506.GR2082@telsasoft.com
M src/backend/storage/ipc/dsm.c
pg_attribute_no_sanitize_alignment() macro
commit : 02e7da01a4362ca241e814d5bf9793e849f1c90c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Feb 2021 17:49:08 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Feb 2021 17:49:08 -0500
Modern gcc and clang compilers offer alignment sanitizers, which help to detect
pointer misalignment. However, our codebase already contains x86-specific
crc32 computation code, which uses unalignment access. Thankfully, those
compilers also support the attribute, which disables alignment sanitizers at
the function level. This commit adds pg_attribute_no_sanitize_alignment(),
which wraps this attribute, and applies it to pg_comp_crc32c_sse42() function.
Back-patch of commits 993bdb9f9 and ad2ad698a, to enable doing
alignment testing in all supported branches.
Discussion: https://postgr.es/m/CAPpHfdsne3%3DT%3DfMNU45PtxdhSL_J2PjLTeS8rwKnJzUR4YNd4w%40mail.gmail.com
Discussion: https://postgr.es/m/475514.1612745257%40sss.pgh.pa.us
Author: Alexander Korotkov, revised by Tom Lane
Reviewed-by: Tom Lane
M src/include/c.h
M src/port/pg_crc32c_sse42.c
Avoid divide-by-zero in regex_selectivity() with long fixed prefix.
commit : 374f1cefe56c2f2a6f54f3d8ad7f2454b420418f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Feb 2021 16:26:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 12 Feb 2021 16:26:47 -0500
Given a regex pattern with a very long fixed prefix (approaching 500
characters), the result of pow(FIXED_CHAR_SEL, fixed_prefix_len) can
underflow to zero. Typically the preceding selectivity calculation
would have underflowed as well, so that we compute 0/0 and get NaN.
In released branches this leads to an assertion failure later on.
That doesn't happen in HEAD, for reasons I've not explored yet,
but it's surely still a bug.
To fix, just skip the division when the pow() result is zero, so
that we'll (most likely) return a zero selectivity estimate. In
the edge cases where "sel" didn't yet underflow, perhaps this
isn't desirable, but I'm not sure that the case is worth spending
a lot of effort on. The results of regex_selectivity_sub() are
barely worth the electrons they're written on anyway :-(
Per report from Alexander Lakhin. Back-patch to all supported versions.
Discussion: https://postgr.es/m/6de0a0c3-ada9-cd0c-3e4e-2fa9964b41e3@gmail.com
M src/backend/utils/adt/selfuncs.c