Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
commit : 73fc2e5bab1e4bad433d00b2b645cfdd234657db
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Jan 2021 15:37:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 21 Jan 2021 15:37:23 -0500
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
M contrib/postgres_fdw/postgres_fdw.c
M src/backend/optimizer/path/clausesel.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/path/pathkeys.c
M src/backend/optimizer/path/tidpath.c
M src/backend/optimizer/plan/analyzejoins.c
M src/backend/optimizer/plan/initsplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepjointree.c
M src/backend/optimizer/util/clauses.c
M src/backend/optimizer/util/inherit.c
M src/backend/optimizer/util/orclauses.c
M src/backend/optimizer/util/placeholder.c
M src/backend/optimizer/util/restrictinfo.c
M src/backend/optimizer/util/var.c
M src/backend/utils/adt/selfuncs.c
M src/include/optimizer/clauses.h
M src/include/optimizer/optimizer.h
M src/include/optimizer/paths.h
M src/include/optimizer/planmain.h
M src/include/optimizer/restrictinfo.h
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Further tweaking of PG_SYSROOT heuristics for macOS.
commit : 6671e81946266fae44eb812a1c76e21845f2990c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 12:07:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 12:07:23 -0500
It emerges that in some phases of the moon (perhaps to do with
directory entry order?), xcrun will report that the SDK path is
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
which is normally a symlink to a version-numbered sibling directory.
Our heuristic to skip non-version-numbered pathnames was rejecting
that, which is the wrong thing to do. We'd still like to end up
with a version-numbered PG_SYSROOT value, but we can have that by
dereferencing the symlink.
Like the previous fix, back-patch to all supported versions.
Discussion: https://postgr.es/m/522433.1611089678@sss.pgh.pa.us
M src/template/darwin
Disable vacuum page skipping in selected test cases.
commit : a57dbfcda5535da31aedb16c76bc532a72a6e7a5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 11:49:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 20 Jan 2021 11:49:29 -0500
By default VACUUM will skip pages that it can't immediately get
exclusive access to, which means that even activities as harmless
and unpredictable as checkpoint buffer writes might prevent a page
from being processed. Ordinarily this is no big deal, but we have
a small number of test cases that examine the results of VACUUM's
processing and therefore will fail if the page of interest is skipped.
This seems to be the explanation for some rare buildfarm failures.
To fix, add the DISABLE_PAGE_SKIPPING option to the VACUUM commands
in tests where this could be an issue.
In passing, remove a duplicated query in pageinspect/sql/page.sql.
Back-patch as necessary (some of these cases are as old as v10).
Discussion: https://postgr.es/m/413923.1611006484@sss.pgh.pa.us
M contrib/pageinspect/expected/page.out
M contrib/pageinspect/sql/page.sql
M contrib/pg_visibility/expected/pg_visibility.out
M contrib/pg_visibility/sql/pg_visibility.sql
Fix bug in detecting concurrent page splits in GiST insert
commit : b8403d140f4e7d753d21116c0fa79dc4ca5ca5cb
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 20 Jan 2021 11:58:03 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Wed, 20 Jan 2021 11:58:03 +0200
In commit 9eb5607e699, I got the condition on checking for split or
deleted page wrong: I used && instead of ||. The comment correctly said
"concurrent split _or_ deletion".
As a result, GiST insertion could miss a concurrent split, and insert to
wrong page. Duncan Sands demonstrated this with a test script that did a
lot of concurrent inserts.
Backpatch to v12, where this was introduced. REINDEX is required to fix
indexes that were affected by this bug.
Backpatch-through: 12
Reported-by: Duncan Sands
Discussion: https://www.postgresql.org/message-id/a9690483-6c6c-3c82-c8ba-dc1a40848f11%40deepbluecap.com
M src/backend/access/gist/gist.c
Fix ALTER DEFAULT PRIVILEGES with duplicated objects
commit : 31e0f9d76bb0196b92f6870a8e1e3e2a4e5e2b28
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Jan 2021 11:39:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 20 Jan 2021 11:39:14 +0900
Specifying duplicated objects in this command would lead to unique
constraint violations in pg_default_acl or "tuple already updated by
self" errors. Similarly to GRANT/REVOKE, increment the command ID after
each subcommand processing to allow this case to work transparently.
A regression test is added by tweaking one of the existing queries of
privileges.sql to stress this case.
Reported-by: Andrus
Author: Michael Paquier
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/ae2a7dc1-9d71-8cba-3bb9-e4cb7eb1f44e@hot.ee
Backpatch-through: 9.5
M src/backend/catalog/aclchk.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Remove faulty support for MergeAppend plan with WHERE CURRENT OF.
commit : 188cd4f440ed6bb2b3120ade9a2277c91d79215c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Jan 2021 13:25:33 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 19 Jan 2021 13:25:33 -0500
Somebody extended search_plan_tree() to treat MergeAppend exactly
like Append, which is 100% wrong, because unlike Append we can't
assume that only one input node is actively returning tuples.
Hence a cursor using a MergeAppend across a UNION ALL or inheritance
tree could falsely match a WHERE CURRENT OF query at a row that
isn't actually the cursor's current output row, but coincidentally
has the same TID (in a different table) as the current output row.
Delete the faulty code; this means that such a case will now return
an error like 'cursor "foo" is not a simply updatable scan of table
"bar"', instead of silently misbehaving. Users should not find that
surprising though, as the same cursor query could have failed that way
already depending on the chosen plan. (It would fail like that if the
sort were done with an explicit Sort node instead of MergeAppend.)
Expand the clearly-inadequate commentary to be more explicit about
what this code is doing, in hopes of forestalling future mistakes.
It's been like this for awhile, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/482865.1611075182@sss.pgh.pa.us
M src/backend/executor/execCurrent.c
doc: adjust alignment of doc file list for "pg_waldump.sgml"
commit : 6c183aff1817d86397aa9c54cd06c286e1876bc7
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Jan 2021 18:48:25 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 18 Jan 2021 18:48:25 -0500
Backpatch-through: 10
M doc/src/sgml/ref/allfiles.sgml
Avoid crash with WHERE CURRENT OF and a custom scan plan.
commit : f0f53195b51a10e5093e0070bb24ef1f574ee725
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 18:32:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 18 Jan 2021 18:32:30 -0500
execCurrent.c's search_plan_tree() assumed that ForeignScanStates
and CustomScanStates necessarily have a valid ss_currentRelation.
This is demonstrably untrue for postgres_fdw's remote join and
remote aggregation plans, and non-leaf custom scans might not have
an identifiable scan relation either. Avoid crashing by ignoring
such nodes when the field is null.
This solution will lead to errors like 'cursor "foo" is not a
simply updatable scan of table "bar"' in cases where maybe we
could have allowed WHERE CURRENT OF to work. That's not an issue
for postgres_fdw's usages, since joins or aggregations would render
WHERE CURRENT OF invalid anyway. But an otherwise-transparent
upper level custom scan node might find this annoying. When and if
someone cares to expend work on such a scenario, we could invent a
custom-scan-provider callback to determine what's safe.
Report and patch by David Geier, commentary by me. It's been like
this for awhile, so back-patch to all supported branches.
Discussion: https://postgr.es/m/0253344d-9bdd-11c4-7f0d-d88c02cd7991@swarm64.com
M src/backend/executor/execCurrent.c
Fix pg_dump for GRANT OPTION among initial privileges.
commit : b8daf894f0d3440dd79131e12221c30b114e4b3e
author : Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
The context is an object that no longer bears some aclitem that it bore
initially. (A user issued REVOKE or GRANT statements upon the object.)
pg_dump is forming SQL to reproduce the object ACL. Since initdb
creates no ACL bearing GRANT OPTION, reaching this bug requires an
extension where the creation script establishes such an ACL. No PGXN
extension does that. If an installation did reach the bug, pg_dump
would have omitted a semicolon, causing a REVOKE and the next SQL
statement to fail. Separately, since the affected code exists to
eliminate an entire aclitem, it wants plain REVOKE, not REVOKE GRANT
OPTION FOR. Back-patch to 9.6, where commit
23f34fa4ba358671adab16773e79c17c92cbc870 first appeared.
Discussion: https://postgr.es/m/20210109102423.GA160022@rfd.leadboat.com
M src/bin/pg_dump/dumputils.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
Prevent excess SimpleLruTruncate() deletion.
commit : 6eb3fc7fcd89258c2f4bb3dde03e41e6ede2be5f
author : Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 16 Jan 2021 12:21:35 -0800
Every core SLRU wraps around. With the exception of pg_notify, the wrap
point can fall in the middle of a page. Account for this in the
PagePrecedes callback specification and in SimpleLruTruncate()'s use of
said callback. Update each callback implementation to fit the new
specification. This changes SerialPagePrecedesLogically() from the
style of asyncQueuePagePrecedes() to the style of CLOGPagePrecedes().
(Whereas pg_clog and pg_serial share a key space, pg_serial is nothing
like pg_notify.) The bug fixed here has the same symptoms and user
followup steps as 592a589a04bd456410b853d86bd05faa9432cbbb. Back-patch
to 9.5 (all supported versions).
Reviewed by Andrey Borodin and (in earlier versions) by Tom Lane.
Discussion: https://postgr.es/m/20190202083822.GC32531@gust.leadboat.com
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/multixact.c
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/storage/lmgr/predicate.c
M src/include/access/slru.h
Disallow CREATE STATISTICS on system catalogs
commit : d26d4c717dbfb24cc9dfd83044b5c9a377dc954a
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 15 Jan 2021 23:24:19 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Fri, 15 Jan 2021 23:24:19 +0100
Add a check that CREATE STATISTICS does not add extended statistics on
system catalogs, similarly to indexes etc. It can be overriden using
the allow_system_table_mods GUC.
This bug exists since 7b504eb282c, adding the extended statistics, so
backpatch all the way back to PostgreSQL 10.
Author: Tomas Vondra
Reported-by: Dean Rasheed
Backpatch-through: 10
Discussion: https://postgr.es/m/CAEZATCXAPrrOKwEsyZKQ4uzzJQWBCt6QAvOcgqRGdWwT1zb%2BrQ%40mail.gmail.com
M src/backend/commands/statscmds.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Improve our heuristic for selecting PG_SYSROOT on macOS.
commit : f44ae4db5feccb8012c1b2df169bf87576ce760e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Jan 2021 11:28:51 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 15 Jan 2021 11:28:51 -0500
In cases where Xcode is newer than the underlying macOS version,
asking xcodebuild for the SDK path will produce a pointer to the
SDK shipped with Xcode, which may end up building code that does
not work on the underlying macOS version. It appears that in
such cases, xcodebuild's answer also fails to match the default
behavior of Apple's compiler: assuming one has installed Xcode's
"command line tools", there will be an SDK for the OS's own version
in /Library/Developer/CommandLineTools, and the compiler will
default to using that. This is all pretty poorly documented,
but experimentation suggests that "xcrun --show-sdk-path" gives
the sysroot path that the compiler is actually using, at least
in some cases. Hence, try that first, but revert to xcodebuild
if xcrun fails (in very old Xcode, it is missing or lacks the
--show-sdk-path switch).
Also, "xcrun --show-sdk-path" may give a path that is valid but lacks
any OS version identifier. We don't really want that, since most
of the motivation for wiring -isysroot into the build flags at all
is to ensure that all parts of a PG installation are built against
the same SDK, even when considering extensions built later and/or on
a different machine. Insist on finding "N.N" in the directory name
before accepting the result. (Adding "--sdk macosx" to the xcrun
call seems to produce the same answer as xcodebuild, but usually
more quickly because it's cached, so we also try that as a fallback.)
The core reason why we don't want to use Xcode's default SDK in cases
like this is that Apple's technology for introducing new syscalls
does not play nice with Autoconf: for example, configure will think
that preadv/pwritev exist when using a Big Sur SDK, even when building
on an older macOS version where they don't exist. It'd be nice to
have a better solution to that problem, but this patch doesn't attempt
to fix that.
Per report from Sergey Shinderuk. Back-patch to all supported versions.
Discussion: https://postgr.es/m/ed3b8e5d-0da8-6ebd-fd1c-e0ac80a4b204@postgrespro.ru
M src/template/darwin
Fix calculation of how much shared memory is required to store a TOC.
commit : 60369db86f6cc9432626df5a5ccdd9eb3338798e
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 15 Jan 2021 12:44:17 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 15 Jan 2021 12:44:17 +0900
Commit ac883ac453 refactored shm_toc_estimate() but changed its calculation
of shared memory size for TOC incorrectly. Previously this could cause too
large memory to be allocated.
Back-patch to v11 where the bug was introduced.
Author: Takayuki Tsunakawa
Discussion: https://postgr.es/m/TYAPR01MB2990BFB73170E2C4921E2C4DFEA80@TYAPR01MB2990.jpnprd01.prod.outlook.com
M src/backend/storage/ipc/shm_toc.c
pg_dump: label PUBLICATION TABLE ArchiveEntries with an owner.
commit : 79d3ac72f690b45e2b278be746f858a1f6311310
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 14 Jan 2021 16:19:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 14 Jan 2021 16:19:38 -0500
This is the same fix as commit 9eabfe300 applied to INDEX ATTACH
entries, but for table-to-publication attachments. As in that
case, even though the backend doesn't record "ownership" of the
attachment, we still ought to label it in the dump archive with
the role name that should run the ALTER PUBLICATION command.
The existing behavior causes the ALTER to be done by the original
role that started the restore; that will usually work fine, but
there may be corner cases where it fails.
The bulk of the patch is concerned with changing struct
PublicationRelInfo to include a pointer to the associated
PublicationInfo object, so that we can get the owner's name
out of that when the time comes. While at it, I rewrote
getPublicationTables() to do just one query of pg_publication_rel,
not one per table.
Back-patch to v10 where this code was introduced.
Discussion: https://postgr.es/m/1165710.1610473242@sss.pgh.pa.us
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dump.h
Prevent drop of tablespaces used by partitioned relations
commit : 5b01a6f13ff7669f37339a9e2416baa02b7429b2
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 14 Jan 2021 15:32:14 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 14 Jan 2021 15:32:14 -0300
When a tablespace is used in a partitioned relation (per commits
ca4103025dfe in pg12 for tables and 33e6c34c3267 in pg11 for indexes),
it is possible to drop the tablespace, potentially causing various
problems. One such was reported in bug #16577, where a rewriting ALTER
TABLE causes a server crash.
Protect against this by using pg_shdepend to keep track of tablespaces
when used for relations that don't keep physical files; we now abort a
tablespace if we see that the tablespace is referenced from any
partitioned relations.
Backpatch this to 11, where this problem has been latent all along. We
don't try to create pg_shdepend entries for existing partitioned
indexes/tables, but any ones that are modified going forward will be
protected.
Note slight behavior change: when trying to drop a tablespace that
contains both regular tables as well as partitioned ones, you'd
previously get ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE and now you'll
get ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST. Arguably, the latter is more
correct.
It is possible to add protecting pg_shdepend entries for existing
tables/indexes, by doing
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE pg_default;
ALTER TABLE ONLY some_partitioned_table SET TABLESPACE original_tablespace;
for each partitioned table/index that is not in the database default
tablespace. Because these partitioned objects do not have storage, no
file needs to be actually moved, so it shouldn't take more time than
what's required to acquire locks.
This query can be used to search for such relations:
SELECT ... FROM pg_class WHERE relkind IN ('p', 'I') AND reltablespace <> 0
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/16577-881633a9f9894fd5@postgresql.org
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/heap.c
M src/backend/catalog/pg_shdepend.c
M src/backend/commands/tablecmds.c
M src/backend/commands/tablespace.c
M src/include/catalog/dependency.h
M src/test/regress/input/tablespace.source
M src/test/regress/output/tablespace.source
Stabilize timeline switch regression test.
commit : 8523a0971ba6490919c8e04bc7f7229aa38c789b
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 14:37:01 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 14:37:01 +0900
Commit fef5b47f6b added the regression test to check whether a standby is
able to follow a primary on a newer timeline when WAL archiving is enabled.
But the buildfarm member florican reported that this test failed because
the requested WAL segment was removed and replication failed. This is a
timing issue. Since neither replication slot is used nor wal_keep_size is set
in the test, checkpoint could remove the WAL segment that's still necessary
for replication.
This commit stabilizes the test by setting wal_keep_size.
Back-patch to v13 where the regression test that this commit stabilizes
was added.
Author: Fujii Masao
Discussion: https://postgr.es/m/X//PsenxcC50jDzX@paquier.xyz
M src/test/recovery/t/004_timeline_switch.pl
Ensure that a standby is able to follow a primary on a newer timeline.
commit : 94f52929a0c4e92c271c5a03bae782ddb0b086bd
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 12:28:47 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 14 Jan 2021 12:28:47 +0900
Commit 709d003fbd refactored WAL-reading code, but accidentally caused
WalSndSegmentOpen() to fail to follow a timeline switch while reading from
a historic timeline. This issue caused a standby to fail to follow a primary
on a newer timeline when WAL archiving is enabled.
If there is a timeline switch within the segment, WalSndSegmentOpen() should
read from the WAL segment belonging to the new timeline. But previously
since it failed to follow a timeline switch, it tried to read the WAL segment
with old timeline. When WAL archiving is enabled, that WAL segment with
old timeline doesn't exist because it's renamed to .partial. This leads
a primary to have tried to read non-existent WAL segment, and which caused
replication to faill with the error "ERROR: requested WAL segment ... has
already been removed".
This commit fixes WalSndSegmentOpen() so that it's able to follow a timeline
switch, to ensure that a standby is able to follow a primary on a newer
timeline even when WAL archiving is enabled.
This commit also adds the regression test to check whether a standby is
able to follow a primary on a newer timeline when WAL archiving is enabled.
Back-patch to v13 where the bug was introduced.
Reported-by: Kyotaro Horiguchi
Author: Kyotaro Horiguchi, tweaked by Fujii Masao
Reviewed-by: Alvaro Herrera, Fujii Masao
Discussion: https://postgr.es/m/20201209.174314.282492377848029776.horikyota.ntt@gmail.com
M src/backend/replication/walsender.c
M src/test/recovery/t/004_timeline_switch.pl
Call out vacuum considerations in create index docs
commit : c285a244f6d36073c6a8c9852e17492568664211
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Jan 2021 17:55:41 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 13 Jan 2021 17:55:41 -0300
Backpatch to pg12, which is as far as it goes without conflicts.
Author: James Coleman <jtc331@gmail.com>
Reviewed-by: "David G. Johnston" <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAaqYe9oEfbz7AxXq7OX+FFVi5w5p1e_Of8ON8ZnKO9QqBfmjg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
M doc/src/sgml/ref/reindex.sgml
Disallow a digit as the first character of a variable name in pgbench.
commit : 6b045ca6cce01f1512af3438a8d4db4dc83b2d07
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 14:52:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 14:52:49 -0500
The point of this restriction is to avoid trying to substitute variables
into timestamp literal values, which may contain strings like '12:34'.
There is a good deal more that should be done to reduce pgbench's
tendency to substitute where it shouldn't. But this is sufficient to
solve the case complained of by Jaime Soler, and it's simple enough
to back-patch.
Back-patch to v11; before commit 9d36a3866, pgbench had a slightly
different definition of what a variable name is, and anyway it seems
unwise to change long-stable branches for this.
Fabien Coelho
Discussion: https://postgr.es/m/alpine.DEB.2.22.394.2006291740420.805678@pseudo
M doc/src/sgml/ref/pgbench.sgml
M src/bin/pgbench/pgbench.c
Doc: clarify behavior of back-half options in pg_dump.
commit : c77f31171c8349d9ae4d6328be1922f06c5d590f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 13:30:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 13 Jan 2021 13:30:04 -0500
Options that change how the archive data is converted to SQL text
are ignored when dumping to archive formats. The documentation
previously said "not meaningful", which is not helpful.
Discussion: https://postgr.es/m/161052021249.12228.9598689907884726185@wrigleys.postgresql.org
M doc/src/sgml/ref/pg_dump.sgml
Remove incorrect markup
commit : bff8d0fe3bff0ce52c0343d81afa8f1745d0209e
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 13 Jan 2021 11:07:37 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 13 Jan 2021 11:07:37 +0100
Seems 737d69ffc3c made a copy/paste or automation error resulting in two
extra right-parenthesis.
Reported-By: Michael Vastola
Backpatch-through: 13
Discussion: https://postgr.es/m/161051035421.12224.1741822783166533529@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
Fix memory leak in SnapBuildSerialize.
commit : 0685c5c1b9225352bbaf4fe81c550f09508379ce
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 13 Jan 2021 08:31:45 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 13 Jan 2021 08:31:45 +0530
The memory for the snapshot was leaked while serializing it to disk during
logical decoding. This memory will be freed only once walsender stops
streaming the changes. This can lead to a huge memory increase when master
logs Standby Snapshot too frequently say when the user is trying to create
many replication slots.
Reported-by: funnyxj.fxj@alibaba-inc.com
Diagnosed-by: funnyxj.fxj@alibaba-inc.com
Author: Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/033ab54c-6393-42ee-8ec9-2b399b5d8cde.funnyxj.fxj@alibaba-inc.com
M src/backend/replication/logical/snapbuild.c
pg_dump: label INDEX ATTACH ArchiveEntries with an owner.
commit : 0011c5a0fdacc5991b996e0081c218fbea4461a8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 13:37:38 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 13:37:38 -0500
Although a partitioned index's attachment to its parent doesn't
have separate ownership, the ArchiveEntry for it needs to be
marked with an owner anyway, to ensure that the ALTER command
is run by the appropriate role when restoring with
--use-set-session-authorization. Without this, the ALTER will
be run by the role that started the restore session, which will
usually work but it's formally the wrong thing.
Back-patch to v11 where this type of ArchiveEntry was added.
In HEAD, add equivalent commentary to the just-added TABLE ATTACH
case, which I'd made do the right thing already.
Discussion: https://postgr.es/m/1094034.1610418498@sss.pgh.pa.us
M src/bin/pg_dump/pg_dump.c
Doc: fix description of privileges needed for ALTER PUBLICATION.
commit : 0725bf3aac643b077b031139a61d2a9b298cc0fe
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 12:52:14 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 12 Jan 2021 12:52:14 -0500
Adding a table to a publication requires ownership of the table
(in addition to ownership of the publication). This was mentioned
nowhere.
M doc/src/sgml/ref/alter_publication.sgml
Fix thinko in comment
commit : ee69833e6e5667b5396c4b843ef88a688a27bd1f
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Jan 2021 11:48:45 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 12 Jan 2021 11:48:45 -0300
This comment has been wrong since its introduction in commit
2c03216d8311.
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAzz6qipFJBbGEaHmyWxvvNDp8httbwLR9tUQWaTjUs2Q@mail.gmail.com
M src/backend/access/transam/xlogreader.c
Fix relation descriptor leak.
commit : decf3fe61ca2b707e8ac7ef996b16ace8df1d165
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Jan 2021 08:30:16 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 12 Jan 2021 08:30:16 +0530
We missed closing the relation descriptor while sending changes via the
root of partitioned relations during logical replication.
Author: Amit Langote and Mark Zhao
Reviewed-by: Amit Kapila and Ashutosh Bapat
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/tencent_41FEA657C206F19AB4F406BE9252A0F69C06@qq.com
Discussion: https://postgr.es/m/tencent_6E296D2F7D70AFC90D83353B69187C3AA507@qq.com
M src/backend/replication/pgoutput/pgoutput.c
doc: expand description of how non-SELECT queries are processed
commit : 14a608aef41b35ed4c2599493aaafe496fec3b3c
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 9 Jan 2021 12:11:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 9 Jan 2021 12:11:16 -0500
The previous description of how the executor processes non-SELECT
queries was very dense, causing lack of clarity. This expanded text
spells it out more simply.
Reported-by: fotis.koutoupas@gmail.com
Discussion: https://postgr.es/m/160912275508.676.17469511338925622905@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/arch-dev.sgml
Fix ancient bug in parsing of BRE-mode regular expressions.
commit : 49c928c0c067a8ec0882eeea5c03ccbd1b1b1a62
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Jan 2021 12:16:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 8 Jan 2021 12:16:00 -0500
brenext(), when parsing a '*' quantifier, forgot to return any "value"
for the token; per the equivalent case in next(), it should return
value 1 to indicate that greedy rather than non-greedy behavior is
wanted. The result is that the compiled regexp could behave like 'x*?'
rather than the intended 'x*', if we were unlucky enough to have
a zero in v->nextvalue at this point. That seems to happen with some
reliability if we have '.*' at the beginning of a BRE-mode regexp,
although that depends on the initial contents of a stack-allocated
struct, so it's not guaranteed to fail.
Found by Alexander Lakhin using valgrind testing. This bug seems
to be aboriginal in Spencer's code, so back-patch all the way.
Discussion: https://postgr.es/m/16814-6c5e3edd2bdf0d50@postgresql.org
M src/backend/regex/regc_lex.c
Adjust createdb TAP tests to work on recent OpenBSD.
commit : 5ba046948ed49c326d124261ae354bd9fae96489
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 20:36:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 20:36:09 -0500
We found last February that the error-case tests added by commit
008cf0409 failed on OpenBSD, because that platform doesn't really
check locale names. At the time it seemed that that was only an issue
for LC_CTYPE, but testing on a more recent version of OpenBSD shows
that it's now equally lax about LC_COLLATE.
Rather than dropping the LC_COLLATE test too, put back LC_CTYPE
(reverting c4b0edb07), and adjust these tests to accept the different
error message that we get if setlocale() doesn't reject a bogus locale
name. The point of these tests is not really what the backend does
with the locale name, but to show that createdb quotes funny locale
names safely; so we're not losing test reliability this way.
Back-patch as appropriate.
Discussion: https://postgr.es/m/231373.1610058324@sss.pgh.pa.us
M src/bin/scripts/t/020_createdb.pl
Further second thoughts about idle_session_timeout patch.
commit : 5db4fdc22472919f97ce83d276fb34b47c794d1f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 11:45:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 7 Jan 2021 11:45:09 -0500
On reflection, the order of operations in PostgresMain() is wrong.
These timeouts ought to be shut down before, not after, we do the
post-command-read CHECK_FOR_INTERRUPTS, to guarantee that any
timeout error will be detected there rather than at some ill-defined
later point (possibly after having wasted a lot of work).
This is really an error in the original idle_in_transaction_timeout
patch, so back-patch to 9.6 where that was introduced.
M src/backend/tcop/postgres.c
Detect the deadlocks between backends and the startup process.
commit : 0f8977b3f2536c91a0a97e2395c297d3acf1f491
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 12:29:43 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 12:29:43 +0900
The deadlocks that the recovery conflict on lock is involved in can
happen between hot-standby backends and the startup process.
If a backend takes an access exclusive lock on the table and which
finally triggers the deadlock, that deadlock can be detected
as expected. On the other hand, previously, if the startup process
took an access exclusive lock and which finally triggered the deadlock,
that deadlock could not be detected and could remain even after
deadlock_timeout passed. This is a bug.
The cause of this bug was that the code for handling the recovery
conflict on lock didn't take care of deadlock case at all. It assumed
that deadlocks involving the startup process and backends were able
to be detected by the deadlock detector invoked within backends.
But this assumption was incorrect. The startup process also should
have invoked the deadlock detector if necessary.
To fix this bug, this commit makes the startup process invoke
the deadlock detector if deadlock_timeout is reached while handling
the recovery conflict on lock. Specifically, in that case, the startup
process requests all the backends holding the conflicting locks to
check themselves for deadlocks.
Back-patch to v9.6. v9.5 has also this bug, but per discussion we decided
not to back-patch the fix to v9.5. Because v9.5 doesn't have some
infrastructure codes (e.g., 37c54863cf) that this bug fix patch depends on.
We can apply those codes for the back-patch, but since the next minor
version release is the final one for v9.5, it's risky to do that. If we
unexpectedly introduce new bug to v9.5 by the back-patch, there is no
chance to fix that. We determined that the back-patch to v9.5 would give
more risk than gain.
Author: Fujii Masao
Reviewed-by: Bertrand Drouvot, Masahiko Sawada, Kyotaro Horiguchi
Discussion: https://postgr.es/m/4041d6b6-cf24-a120-36fa-1294220f8243@oss.nttdata.com
M src/backend/storage/ipc/procarray.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/proc.c
M src/backend/tcop/postgres.c
M src/include/storage/procarray.h
doc: Fix description about default behavior of recovery_target_timeline.
commit : b1ebec2d800d676ffd3374945efc18eefe9ac4a8
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 11:58:23 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 6 Jan 2021 11:58:23 +0900
The default value of recovery_target_timeline was changed in v12,
but the description about the default behavior of that was not updated.
Back-patch to v12 where the default behavior of recovery_target_timeline
was changed.
Author: Benoit Lobréau
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAPE8EZ7c3aruEmM24GYkj8y8WmHKD1m9TtPtgCF0nQ3zw4LCkQ@mail.gmail.com
M doc/src/sgml/backup.sgml
doc: improve NLS instruction wording
commit : b266a406877b46ab0197d3c7da8c457c305f29be
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 5 Jan 2021 14:26:37 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 5 Jan 2021 14:26:37 -0500
Reported-by: "Tang, Haiying"
Discussion: https://postgr.es/m/bbbccf7a3c2d436e85d45869d612fd6b@G08CNEXMBPEKD05.g08.fujitsu.local
Author: "Tang, Haiying"
Backpatch-through: 9.5
M doc/src/sgml/nls.sgml
Add an explicit cast to double when using fabs().
commit : 5777b6ea29a581f073c80ae48931adadcbc268d4
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:51:21 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:51:21 +0000
Commit bc43b7c2c0 used fabs() directly on an int variable, which
apparently requires an explicit cast on some platforms.
Per buildfarm.
M src/backend/utils/adt/numeric.c
Fix numeric_power() when the exponent is INT_MIN.
commit : e15c384d7acaa2d7d967f2d8feb6bb0d3b793b3d
author : Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:08:59 +0000
committer: Dean Rasheed <dean.a.rasheed@gmail.com>
date : Tue, 5 Jan 2021 11:08:59 +0000
In power_var_int(), the computation of the number of significant
digits to use in the computation used log(Abs(exp)), which isn't safe
because Abs(exp) returns INT_MIN when exp is INT_MIN. Use fabs()
instead of Abs(), so that the exponent is cast to a double before the
absolute value is taken.
Back-patch to 9.6, where this was introduced (by 7d9a4737c2).
Discussion: https://postgr.es/m/CAEZATCVd6pMkz=BrZEgBKyqqJrt2xghr=fNc8+Z=5xC6cgWrWA@mail.gmail.com
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/numeric.out
M src/test/regress/sql/numeric.sql
Fix integer-overflow corner cases in substring() functions.
commit : 9e7d87ca84b91b0c6d8cb052bb6193881f6861fb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 18:32:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 4 Jan 2021 18:32:40 -0500
If the substring start index and length overflow when added together,
substring() misbehaved, either throwing a bogus "negative substring
length" error on a case that should succeed, or failing to complain that
a negative length is negative (and instead returning the whole string,
in most cases). Unsurprisingly, the text, bytea, and bit variants of
the function all had this issue. Rearrange the logic to ensure that
negative lengths are always rejected, and add an overflow check to
handle the other case.
Also install similar guards into detoast_attr_slice() (nee
heap_tuple_untoast_attr_slice()), since it's far from clear that
no other code paths leading to that function could pass it values
that would overflow.
Patch by myself and Pavel Stehule, per bug #16804 from Rafi Shamim.
Back-patch to v11. While these bugs are old, the common/int.h
infrastructure for overflow-detecting arithmetic didn't exist before
commit 4d6ad3125, and it doesn't seem like these misbehaviors are bad
enough to justify developing a standalone fix for the older branches.
Discussion: https://postgr.es/m/16804-f4eeeb6c11ba71d4@postgresql.org
M src/backend/access/common/detoast.c
M src/backend/utils/adt/varbit.c
M src/backend/utils/adt/varlena.c
M src/test/regress/expected/bit.out
M src/test/regress/expected/strings.out
M src/test/regress/sql/bit.sql
M src/test/regress/sql/strings.sql
Update copyright for 2021
commit : c09f6882d6f78bde26fcc1e1a3da11c274de596a
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 2 Jan 2021 13:06:24 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 2 Jan 2021 13:06:24 -0500
Backpatch-through: 9.5
M COPYRIGHT
M doc/src/sgml/legal.sgml
Doc: improve explanation of EXTRACT(EPOCH) for timestamp without tz.
commit : 4750d92ce82fa70dfee890161576743c151c422a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Jan 2021 15:51:09 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 1 Jan 2021 15:51:09 -0500
Try to be clearer about what computation is actually happening here.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/16797-f264b0b980b53b8b@postgresql.org
M doc/src/sgml/func.sgml
Get heap page max offset with buffer lock held.
commit : 55e5352266b1edc943a2a57a5d349aac73bac1a2
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 17:21:41 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 17:21:41 -0800
On further reflection it seems better to call PageGetMaxOffsetNumber()
after acquiring a buffer lock on the page. This shouldn't really
matter, but doing it this way is cleaner.
Follow-up to commit 42288174.
Backpatch: 12-, just like commit 42288174
M src/backend/access/heap/heapam.c
Fix index deletion latestRemovedXid bug.
commit : 7a57960ba6a7ae74d0830d0c766c275c288ed51a
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 16:29:03 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 30 Dec 2020 16:29:03 -0800
The logic for determining the latest removed XID for the purposes of
generating recovery conflicts in REDO routines was subtly broken. It
failed to follow links from HOT chains, and so failed to consider all
relevant heap tuple headers in some cases.
To fix, expand the loop that deals with LP_REDIRECT line pointers to
also deal with HOT chains. The new version of the loop is loosely based
on a similar loop from heap_prune_chain().
The impact of this bug is probably quite limited, since the horizon code
necessarily deals with heap tuples that are pointed to by LP_DEAD-set
index tuples. The process of setting LP_DEAD index tuples (e.g. within
the kill_prior_tuple mechanism) is highly correlated with opportunistic
pruning of pointed-to heap tuples. Plus the question of generating a
recovery conflict usually comes up some time after index tuple LP_DEAD
bits were initially set, unlike heap pruning, where a latestRemovedXid
is generated at the point of the pruning operation (heap pruning has no
deferred "would-be page split" style processing that produces conflicts
lazily).
Only backpatch to Postgres 12, the first version where this logic runs
during original execution (following commit 558a9165e08). The index
latestRemovedXid mechanism has had the same bug since it first appeared
over 10 years ago (in commit a760893d), but backpatching to all
supported versions now seems like a bad idea on balance. Running the
new improved code during recovery seems risky, especially given the lack
of complaints from the field.
Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-Wz=Eib393+HHcERK_9MtgNS7Ew1HY=RDC_g6GL46zM5C6Q@mail.gmail.com
Backpatch: 12-
M src/backend/access/heap/heapam.c
M src/backend/storage/ipc/standby.c
Doc: spell out comparison behaviors for the date/time types.
commit : 624fd9e56b455d89d2d6faca73be16442c784190
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 17:48:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 17:48:43 -0500
The behavior of cross-type comparisons among date/time data types was
not really explained anywhere. You could probably infer it if you
recognized the applicability of comments elsewhere about datatype
conversions, but it seems worthy of explicit documentation.
Per bug #16797 from Dana Burd.
Discussion: https://postgr.es/m/16797-f264b0b980b53b8b@postgresql.org
M doc/src/sgml/func.sgml
Fix up usage of krb_server_keyfile GUC parameter.
commit : 861e967176e99da9122bb19bc2312c2ecdf6673c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 11:38:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Dec 2020 11:38:42 -0500
secure_open_gssapi() installed the krb_server_keyfile setting as
KRB5_KTNAME unconditionally, so long as it's not empty. However,
pg_GSS_recvauth() only installed it if KRB5_KTNAME wasn't set already,
leading to a troubling inconsistency: in theory, clients could see
different sets of server principal names depending on whether they
use GSSAPI encryption. Always using krb_server_keyfile seems like
the right thing, so make both places do that. Also fix up
secure_open_gssapi()'s lack of a check for setenv() failure ---
it's unlikely, surely, but security-critical actions are no place
to be sloppy.
Also improve the associated documentation.
This patch does nothing about secure_open_gssapi()'s use of setenv(),
and indeed causes pg_GSS_recvauth() to use it too. That's nominally
against project portability rules, but since this code is only built
with --with-gssapi, I do not feel a need to do something about this
in the back branches. A fix will be forthcoming for HEAD though.
Back-patch to v12 where GSSAPI encryption was introduced. The
dubious behavior in pg_GSS_recvauth() goes back further, but it
didn't have anything to be inconsistent with, so let it be.
Discussion: https://postgr.es/m/2187460.1609263156@sss.pgh.pa.us
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/config.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/utils/misc/postgresql.conf.sample
In pg_upgrade cross-version test, handle lack of oldstyle_length().
commit : 239213684d01a64f82dfa6263cccc8bf68aeddd3
author : Noah Misch <noah@leadboat.com>
date : Wed, 30 Dec 2020 01:43:43 -0800
committer: Noah Misch <noah@leadboat.com>
date : Wed, 30 Dec 2020 01:43:43 -0800
This suffices for testing v12 -> v13; some other version pairs need more
changes. Back-patch to v10, which removed the function.
M src/bin/pg_upgrade/test.sh
doc: Improve some grammar and sentences
commit : 5253906fac5a2f3669f7867bcb5507f6f0ea891c
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Dec 2020 18:18:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Dec 2020 18:18:59 +0900
90fbf7c has taken care of that for HEAD. This includes the portion of
the fixes that applies to the documentation, where needed depending on
the branch.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20201227202604.GC26311@telsasoft.com
Backpatch-through: 9.5
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/pgstatstatements.sgml
M doc/src/sgml/ref/explain.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/wal.sgml
Improve log messages related to pg_hba.conf not matching a connection.
commit : d05e14d786acacfdf0bd7f3202c543fffaf832ca
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:58:58 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:58:58 -0500
Include details on whether GSS encryption has been activated;
since we added "hostgssenc" type HBA entries, that's relevant info.
Kyotaro Horiguchi and Tom Lane. Back-patch to v12 where
GSS encryption was introduced.
Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se
M src/backend/libpq/auth.c
Fix assorted issues in backend's GSSAPI encryption support.
commit : c1c88bf03e1eb85d5ca04bc7cfe2630154ec70d3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:44:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 17:44:17 -0500
Unrecoverable errors detected by GSSAPI encryption can't just be
reported with elog(ERROR) or elog(FATAL), because attempting to
send the error report to the client is likely to lead to infinite
recursion or loss of protocol sync. Instead make this code do what
the SSL encryption code has long done, which is to just report any
such failure to the server log (with elevel COMMERROR), then pretend
we've lost the connection by returning errno = ECONNRESET.
Along the way, fix confusion about whether message translation is done
by pg_GSS_error() or its callers (the latter should do it), and make
the backend version of that function work more like the frontend
version.
Avoid allocating the port->gss struct until it's needed; we surely
don't need to allocate it in the postmaster.
Improve logging of "connection authorized" messages with GSS enabled.
(As part of this, I back-patched the code changes from dc11f31a1.)
Make BackendStatusShmemSize() account for the GSS-related space that
will be allocated by CreateSharedBackendStatus(). This omission
could possibly cause out-of-shared-memory problems with very high
max_connections settings.
Remove arbitrary, pointless restriction that only GSS authentication
can be used on a GSS-encrypted connection.
Improve documentation; notably, document the fact that libpq now
prefers GSS encryption over SSL encryption if both are possible.
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se
M doc/src/sgml/client-auth.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/runtime.sgml
M src/backend/libpq/auth.c
M src/backend/libpq/be-gssapi-common.c
M src/backend/libpq/be-secure-gssapi.c
M src/backend/libpq/be-secure.c
M src/backend/libpq/hba.c
M src/backend/libpq/pqcomm.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/be-gssapi-common.h
M src/include/libpq/libpq-be.h
Fix bugs in libpq's GSSAPI encryption support.
commit : 06b844c2b8d3e7743b9ff7734893815df1fb68f0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 15:43:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 15:43:44 -0500
The critical issue fixed here is that if a GSSAPI-encrypted connection
is successfully made, pqsecure_open_gss() cleared conn->allow_ssl_try,
as an admittedly-hacky way of preventing us from then trying to tunnel
SSL encryption over the already-encrypted connection. The problem
with that is that if we abandon the GSSAPI connection because of a
failure during authentication, we would not attempt SSL encryption
in the next try with the same server. This can lead to unexpected
connection failure, or silently getting a non-encrypted connection
where an encrypted one is expected.
Fortunately, we'd only manage to make a GSSAPI-encrypted connection
if both client and server hold valid tickets in the same Kerberos
infrastructure, which is a relatively uncommon environment.
Nonetheless this is a very nasty bug with potential security
consequences. To fix, don't reset the flag, instead adding a
check for conn->gssenc being already true when deciding whether
to try to initiate SSL.
While here, fix some lesser issues in libpq's GSSAPI code:
* Use the need_new_connection stanza when dropping an attempted
GSSAPI connection, instead of partially duplicating that code.
The consequences of this are pretty minor: AFAICS it could only
lead to auth_req_received or password_needed remaining set when
they shouldn't, which is not too harmful.
* Fix pg_GSS_error() to not repeat the "mprefix" it's given multiple
times, and to notice any failure return from gss_display_status().
* Avoid gratuitous dependency on NI_MAXHOST in
pg_GSS_load_servicename().
Per report from Mikael Gustavsson. Back-patch to v12 where
this code was introduced.
Discussion: https://postgr.es/m/e5b0b6ed05764324a2f3fe7acfc766d5@smhi.se
M src/interfaces/libpq/fe-connect.c
M src/interfaces/libpq/fe-gssapi-common.c
M src/interfaces/libpq/fe-secure-gssapi.c
Expose the default for channel_binding in PQconndefaults().
commit : 31d2b11b94416ba94624ab07bcc1cb4a47c58c2e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 12:13:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 12:13:40 -0500
If there's a static default value for a connection option,
it should be shown in the PQconninfoOptions array.
Daniele Varrazzo
Discussion: https://postgr.es/m/CA+mi_8Zo8Rgn7p+6ZRY7QdDu+23ukT9AvoHNyPbgKACxwgGhZA@mail.gmail.com
M src/interfaces/libpq/fe-connect.c
Further fix thinko in plpgsql memory leak fix.
commit : 63d78d106d2f69768f9b4c66ff849d95a83205f7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:55:23 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:55:23 -0500
There's a second call of get_eval_mcontext() that should also be
get_stmt_mcontext(). This is actually dead code, since no
interesting allocations happen before switching back to the
original context, but we should keep it in sync with the other
call to forestall possible future bugs.
Discussion: https://postgr.es/m/f075f7be-c654-9aa8-3ffc-e9214622f02a@enterprisedb.com
M src/pl/plpgsql/src/pl_exec.c
Fix thinko in plpgsql memory leak fix.
commit : 0ea25ed108d8344ac17012e62790e7e9ef7f1a7a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:41:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Dec 2020 11:41:25 -0500
Commit a6b1f5365 intended to place the transient "target" list of
a CALL statement in the function's statement-lifespan context,
but I fat-fingered that and used get_eval_mcontext() instead of
get_stmt_mcontext(). The eval_mcontext belongs to the "simple
expression" infrastructure, which is destroyed at transaction end.
The net effect is that a CALL in a procedure to another procedure
that has OUT or INOUT parameters would fail if the called procedure
did a COMMIT.
Per report from Peter Eisentraut. Back-patch to v11, like the
prior patch.
Discussion: https://postgr.es/m/f075f7be-c654-9aa8-3ffc-e9214622f02a@enterprisedb.com
M src/pl/plpgsql/src/expected/plpgsql_call.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/sql/plpgsql_call.sql
Fix inconsistent code with shared invalidations of snapshots
commit : 30803bd1cd6c4c9a0dc3362b02b6aa549b876d77
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Dec 2020 22:16:57 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 28 Dec 2020 22:16:57 +0900
The code in charge of processing a single invalidation message has been
using since 568d413 the structure for relation mapping messages. This
had fortunately no consequence as both locate the database ID at the
same location, but it could become a problem in the future if this area
of the code changes.
Author: Konstantin Knizhnik
Discussion: https://postgr.es/m/8044c223-4d3a-2cdb-42bf-29940840ce94@postgrespro.ru
Backpatch-through: 9.5
M src/backend/utils/cache/inval.c
postgres_fdw: Fix connection leak.
commit : 546f143740a07c4d9798f5870af8dad73ae057b5
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 28 Dec 2020 19:57:51 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 28 Dec 2020 19:57:51 +0900
In postgres_fdw, the cached connections to foreign servers will not be
closed until the local session exits if the user mappings or foreign servers
that those connections depend on are dropped. Those connections can be
leaked.
To fix that connection leak issue, after a change to a pg_foreign_server
or pg_user_mapping catalog entry, this commit makes postgres_fdw close
the connections depending on that entry immediately if current
transaction has not used those connections yet. Otherwise, mark those
connections as invalid and then close them at the end of current transaction,
since they cannot be closed in the midst of the transaction using them.
Closed connections will be remade at the next opportunity if necessary.
Back-patch to all supported branches.
Author: Bharath Rupireddy
Reviewed-by: Zhihong Yu, Zhijie Hou, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVNcGH_6qLY-4_tXz8JLvA+4yeBThRfxMz7Oxbk1aHcpQ@mail.gmail.com
M contrib/postgres_fdw/connection.c
M contrib/postgres_fdw/expected/postgres_fdw.out
M contrib/postgres_fdw/sql/postgres_fdw.sql
Second attempt to stabilize 05c02589.
commit : cd7d8cde75063a85ee8c5fd27713061e56a8684d
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 12:09:00 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 12:09:00 -0800
Removing the EXPLAIN test to stabilize the buildfarm. The execution
test should still be effective to catch the bug even if the plan is
slightly different on different platforms.
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Stabilize test introduced in 05c02589, per buildfarm.
commit : 6669cc769c44b691510c14be12acd9148c74d4d1
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 09:47:23 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 27 Dec 2020 09:47:23 -0800
In passing, make the capitalization match the rest of the file.
Reported-by: Tom Lane
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix bug #16784 in Disk-based Hash Aggregation.
commit : 7b8692eaf113a56933c77cf4c3993716ab37e763
author : Jeff Davis <jdavis@postgresql.org>
date : Sat, 26 Dec 2020 17:25:30 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Sat, 26 Dec 2020 17:25:30 -0800
Before processing tuples, agg_refill_hash_table() was setting all
pergroup pointers to NULL to signal to advance_aggregates() that it
should not attempt to advance groups that had spilled.
The problem was that it also set the pergroups for sorted grouping
sets to NULL, which caused rescanning to fail.
Instead, change agg_refill_hash_table() to only set the pergroups for
hashed grouping sets to NULL; and when compiling the expression, pass
doSort=false.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16784-7ff169bf2c3d1588%40postgresql.org
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Invalidate acl.c caches when pg_authid changes.
commit : 9f8a48bb2c0e5d6557d78d7cce34444b249fbead
author : Noah Misch <noah@leadboat.com>
date : Fri, 25 Dec 2020 10:41:59 -0800
committer: Noah Misch <noah@leadboat.com>
date : Fri, 25 Dec 2020 10:41:59 -0800
This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as
quickly as they have been reflecting "GRANT role_name". Back-patch to
9.5 (all supported versions).
Reviewed by Nathan Bossart.
Discussion: https://postgr.es/m/20201221095028.GB3777719@rfd.leadboat.com
M src/backend/utils/adt/acl.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Avoid time-of-day-dependent failure in log rotation test.
commit : 6f7e972e2f44912b15d4a8884534745b1d5f492b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 21:37:46 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 21:37:46 -0500
Buildfarm members pogona and petalura have shown a failure when
pg_ctl/t/004_logrotate.pl starts just before local midnight.
The default rotate-at-midnight behavior occurs just before the
Perl script examines current_logfiles, so it figures that the
rotation it's already requested has occurred ... but in reality,
that rotation happens just after it looks, so the expected new
log data goes into a different file than the one it's examining.
In HEAD, src/test/kerberos/t/001_auth.pl has acquired similar code
that evidently has a related failure mode. Besides being quite new,
few buildfarm critters run that test, so it's unsurprising that
we've not yet seen a failure there.
Fix both cases by setting log_rotation_age = 0 so that no time-based
rotation can occur. Also absorb 004_logrotate.pl's decision to
set lc_messages = 'C' into the kerberos test, in hopes that it will
work in non-English prevailing locales.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pogona&dt=2020-12-24%2022%3A10%3A04
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=petalura&dt=2020-02-01%2022%3A20%3A04
M src/bin/pg_ctl/t/004_logrotate.pl
Fix race condition between shutdown and unstarted background workers.
commit : 0217ad806637fed6b3bce759169724f31b66256d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 17:00:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Dec 2020 17:00:43 -0500
If a database shutdown (smart or fast) is commanded between the time
some process decides to request a new background worker and the time
that the postmaster can launch that worker, then nothing happens
because the postmaster won't launch any bgworkers once it's exited
PM_RUN state. This is fine ... unless the requesting process is
waiting for that worker to finish (or even for it to start); in that
case the requestor is stuck, and only manual intervention will get us
to the point of being able to shut down.
To fix, cancel pending requests for workers when the postmaster sends
shutdown (SIGTERM) signals, and similarly cancel any new requests that
arrive after that point. (We can optimize things slightly by only
doing the cancellation for workers that have waiters.) To fit within
the existing bgworker APIs, the "cancel" is made to look like the
worker was started and immediately stopped, causing deregistration of
the bgworker entry. Waiting processes would have to deal with
premature worker exit anyway, so this should introduce no bugs that
weren't there before. We do have a side effect that registration
records for restartable bgworkers might disappear when theoretically
they should have remained in place; but since we're shutting down,
that shouldn't matter.
Back-patch to v10. There might be value in putting this into 9.6
as well, but the management of bgworkers is a bit different there
(notably see 8ff518699) and I'm not convinced it's worth the effort
to validate the patch for that branch.
Discussion: https://postgr.es/m/661570.1608673226@sss.pgh.pa.us
M contrib/pg_prewarm/autoprewarm.c
M src/backend/postmaster/bgworker.c
M src/backend/postmaster/postmaster.c
M src/include/postmaster/bgworker_internals.h
docs: document which server-side languages can create procs
commit : d420ae74a7b97ab7a44f381f8c0ef401f74c9d38
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 23 Dec 2020 09:37:38 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 23 Dec 2020 09:37:38 -0500
This was missed when the feature was added.
Reported-by: Daniel Westermann
Discussion: https://postgr.es/m/160624532969.25818.4767632047905006142@wrigleys.postgresql.org
Backpatch-through: 11
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/spi.sgml
Fix portability issues with parsing of recovery_target_xid
commit : 1e54664eee4b3be0591841f50a9cac5b421c3401
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Dec 2020 12:51:35 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 23 Dec 2020 12:51:35 +0900
The parsing of this parameter has been using strtoul(), which is not
portable across platforms. On most Unix platforms, unsigned long has a
size of 64 bits, while on Windows it is 32 bits. It is common in
recovery scenarios to rely on the output of txid_current() or even the
newer pg_current_xact_id() to get a transaction ID for setting up
recovery_target_xid. The value returned by those functions includes the
epoch in the computed result, which would cause strtoul() to fail where
unsigned long has a size of 32 bits once the epoch is incremented.
WAL records and 2PC data include only information about 32-bit XIDs and
it is not possible to have XIDs across more than one epoch, so
discarding the high bits from the transaction ID set has no impact on
recovery. On the contrary, the use of strtoul() prevents a consistent
behavior across platforms depending on the size of unsigned long.
This commit changes the parsing of recovery_target_xid to use
pg_strtouint64() instead, available down to 9.6. There is one TAP test
stressing recovery with recovery_target_xid, where a tweak based on
pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets
tested, as per an idea from Alexander Lakhin.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16780-107fd0c0385b1035@postgresql.org
Backpatch-through: 9.6
M src/backend/utils/misc/guc.c
M src/test/recovery/t/003_recovery_targets.pl
Improve autoprewarm's handling of early-shutdown scenarios.
commit : 4b0292253cfca558b76c7a869ba0930a5e6d82fe
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Dec 2020 13:23:49 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 22 Dec 2020 13:23:49 -0500
Bad things happen if the DBA issues "pg_ctl stop -m fast" before
autoprewarm finishes loading its list of blocks to prewarm.
The current worker process successfully terminates early, but
(if this wasn't the last database with blocks to prewarm) the
leader process will just try to launch another worker for the
next database. Since the postmaster is now in PM_WAIT_BACKENDS
state, it ignores the launch request, and the leader just sits
until it's killed manually.
This is mostly the fault of our half-baked design for launching
background workers, but a proper fix for that is likely to be
too invasive to be back-patchable. To ameliorate the situation,
fix apw_load_buffers() to check whether SIGTERM has arrived
just before trying to launch another worker. That leaves us with
only a very narrow window in each worker launch where SIGTERM
could occur between the launch request and successful worker start.
Another issue is that if the leader process does manage to exit,
it unconditionally rewrites autoprewarm.blocks with only the
blocks currently in shared buffers, thus forgetting any blocks
that we hadn't reached yet while prewarming. This seems quite
unhelpful, since the next database start will then not have the
expected prewarming benefit. Fix it to not modify the file if
we shut down before the initial load attempt is complete.
Per bug #16785 from John Thompson. Back-patch to v11 where
the autoprewarm code was introduced.
Discussion: https://postgr.es/m/16785-c0207d8c67fb5f25@postgresql.org
M contrib/pg_prewarm/autoprewarm.c
Improve find_em_expr_usable_for_sorting_rel comment
commit : 336879f5557e6bb1f6c8d7837fd8b87158441078
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 22 Dec 2020 02:00:39 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 22 Dec 2020 02:00:39 +0100
Clarify the relationship between find_em_expr_usable_for_sorting_rel and
prepare_sort_from_pathkeys, i.e. what restrictions need to be shared
between those two places.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/plan/createplan.c
Don't search for volatile expr in find_em_expr_usable_for_sorting_rel
commit : aa97890b6ec2ad07700c6e4825022ae3979ece7f
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 19:37:14 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 19:37:14 +0100
While prepare_sort_from_pathkeys has to be concerned about matching up
a volatile expression to the proper tlist entry, we don't need to do
that in find_em_expr_usable_for_sorting_rel becausee such a sort will
have to be postponed anyway.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
M src/backend/optimizer/path/equivclass.c
Disallow SRFs when considering sorts below Gather Merge
commit : d0167631e8b7388b78203c6798621f98beed93d5
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:58:32 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:58:32 +0100
While we do allow SRFs in ORDER BY, scan/join processing should not
consider such cases - such sorts should only happen via final Sort atop
a ProjectSet. So make sure we don't try adding such sorts below Gather
Merge, just like we do for expressions that are volatile and/or not
parallel safe.
Backpatch to PostgreSQL 13, where this code was introduced as part of
the Incremental Sort patch.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
Discussion: https://postgr.es/m/295524.1606246314%40sss.pgh.pa.us
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/util/tlist.c
M src/include/optimizer/optimizer.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Remove "invalid concatenation of jsonb objects" error case.
commit : 38d30a14b05e0cc2996fd311d94d7ae4fe2122aa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Dec 2020 13:11:29 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Dec 2020 13:11:29 -0500
The jsonb || jsonb operator arbitrarily rejected certain combinations
of scalar and non-scalar inputs, while being willing to concatenate
other combinations. This was of course quite undocumented. Rather
than trying to document it, let's just remove the restriction,
creating a uniform rule that unless we are handling an object-to-object
concatenation, non-array inputs are converted to one-element arrays,
resulting in an array-to-array concatenation. (This does not change
the behavior for any case that didn't throw an error before.)
Per complaint from Joel Jacobson. Back-patch to all supported branches.
Discussion: https://postgr.es/m/163099.1608312033@sss.pgh.pa.us
M doc/src/sgml/func.sgml
M src/backend/utils/adt/jsonfuncs.c
M src/test/regress/expected/jsonb.out
M src/test/regress/sql/jsonb.sql
Check parallel safety in generate_useful_gather_paths
commit : be9c3cd186ba86b9bc3df7ecc64b81ce4726810d
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:29:46 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:29:46 +0100
Commit ebb7ae839d ensured we ignore pathkeys with volatile expressions
when considering adding a sort below a Gather Merge. Turns out we need
to care about parallel safety of the pathkeys too, otherwise we might
try sorting e.g. on results of a correlated subquery (as demonstrated
by a report from Luis Roberto).
Initial investigation by Tom Lane, patch by James Coleman. Backpatch
to 13, where the code was instroduced (as part of Incremental Sort).
Reported-by: Luis Roberto
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/622580997.37108180.1604080457319.JavaMail.zimbra%40siscobra.com.br
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Consider unsorted paths in generate_useful_gather_paths
commit : ea190ed14b4b75b38a490707d5d08231dcacfb8c
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:16:16 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Mon, 21 Dec 2020 18:16:16 +0100
generate_useful_gather_paths used to skip unsorted paths (without any
pathkeys), but that is unnecessary - the later code actually can handle
such paths just fine by adding a Sort node. This is clearly a thinko,
preventing construction of useful plans.
Backpatch to 13, where Incremental Sort was introduced.
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13
Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Doc: fix description of how to use src/tutorial files.
commit : bd6939a4e22ff5cc4ed77eec2c3c2d4c58ea2143
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 15:28:22 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 15:28:22 -0500
The separate "cd" command before invoking psql made sense (or at least
I thought so) when it was added in commit ed1939332. But 4e3a61635
removed the supporting text that explained when to use it, making it
just confusing. So drop it.
Also switch from four-dot to three-dot filler for the unsupplied
part of the path, since at least one person has read the four-dot
filler as a typo for "../..". And fix these/those inconsistency.
Discussion: https://postgr.es/m/160837647714.673.5195186835607800484@wrigleys.postgresql.org
M doc/src/sgml/query.sgml
Doc: improve description of pgbench script weights.
commit : 22d1569af95d6b409363a6c58ac347a3eb5878dc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 13:37:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 20 Dec 2020 13:37:25 -0500
Point out the workaround to be used if you want to write a script
file name that includes "@". Clean up the text a little.
Fabien Coelho, additional wordsmithing by me
Discussion: https://postgr.es/m/1c4e81550d214741827a03292222db8d@G08CNEXMBPEKD06.g08.fujitsu.local
M doc/src/sgml/ref/pgbench.sgml
Avoid memcpy() with same source and destination during relmapper init.
commit : 722eb325b9511e9d0a8669112c636edebe2cb01b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Dec 2020 15:46:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Dec 2020 15:46:44 -0500
A narrow reading of the C standard says that memcpy(x,x,n) is undefined,
although it's hard to envision an implementation that would really
misbehave. However, analysis tools such as valgrind might whine about
this; accordingly, let's band-aid relmapper.c to not do it.
See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes.
Apparently, none of those folk tried valgrinding initdb? This has been
like this for long enough that I'm surprised it hasn't been reported
before.
Back-patch, just in case anybody wants to use a back branch on a platform
that complains about this; we back-patched those earlier fixes too.
Discussion: https://postgr.es/m/161790.1608310142@sss.pgh.pa.us
M src/backend/utils/cache/relmapper.c
doc: Fix explanation related to pg_shmem_allocations
commit : d28a14d2d4009cec91f9d0f0ceaa60f06c6e289c
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Dec 2020 10:39:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 16 Dec 2020 10:39:29 +0900
Offsets are shown as NULL only for anonymous allocations.
Author: Benoit Lobréau
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CAPE8EZ5Lnoyqoz7aZpvQM0E8sW+hw+k6G2NULe+m4arFRrA1aA@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/catalogs.sgml
doc: clarify COPY TO for partitioning/inheritance
commit : de7b034dafd3847cddcd5f96cca1efce6f7ace8a
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 15 Dec 2020 19:20:15 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 15 Dec 2020 19:20:15 -0500
It was not clear how COPY TO behaved with partitioning/inheritance
because the paragraphs were so far apart. Also reword to simplify.
Discussion: https://postgr.es/m/20201203211723.GR24052@telsasoft.com
Author: Justin Pryzby
Backpatch-through: 10
M doc/src/sgml/ref/copy.sgml
Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE."
commit : fde5f130c9c30e75c7a8ee33095a6a7e79d5b626
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Dec 2020 23:48:44 -0800
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 14 Dec 2020 23:48:44 -0800
This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327.
Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked
around.
This enables proper pipelining of commands after terminating
replication, eliminating an undocumented limitation.
Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi
Backpatch-through: 9.5
M src/backend/replication/walsender.c
initdb: complete getopt_long alphabetization
commit : 787d06a206b24138c06393b4737c6f116aaf30e6
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:59:09 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:59:09 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
initdb: properly alphabetize getopt_long options in C string
commit : 38bcd4340f5a7d618961d185f79776f5802336e0
author : Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:51:16 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Sat, 12 Dec 2020 12:51:16 -0500
Backpatch-through: 9.5
M src/bin/initdb/initdb.c
Teach contain_leaked_vars that assignment SubscriptingRefs are leaky.
commit : c0549cee07ea3b6b0260a3c08c5f44807999a983
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 17:50:54 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 17:50:54 -0500
array_get_element and array_get_slice qualify as leakproof, since
they will silently return NULL for bogus subscripts. But
array_set_element and array_set_slice throw errors for such cases,
making them clearly not leakproof. contain_leaked_vars was evidently
written with only the former case in mind, as it gave the wrong answer
for assignment SubscriptingRefs (nee ArrayRefs).
This would be a live security bug, were it not that assignment
SubscriptingRefs can only occur in INSERT and UPDATE target lists,
while we only care about leakproofness for qual expressions; so the
wrong answer can't occur in practice. Still, that's a rather shaky
answer for a security-related question; and maybe in future somebody
will want to ask about leakproofness of a tlist. So it seems wise to
fix and even back-patch this correction.
(We would need some change here anyway for the upcoming
generic-subscripting patch, since extensions might make different
tradeoffs about whether to throw errors. Commit 558d77f20 attempted
to lay groundwork for that by asking check_functions_in_node whether a
SubscriptingRef contains leaky functions; but that idea fails now that
the implementation methods of a SubscriptingRef are not SQL-visible
functions that could be marked leakproof or not.)
Back-patch to 9.6. While 9.5 has the same issue, the code's a bit
different. It seems quite unlikely that we'd introduce any actual bug
in the short time 9.5 has left to live, so the work/risk/reward balance
isn't attractive for changing 9.5.
Discussion: https://postgr.es/m/3143742.1607368115@sss.pgh.pa.us
M src/backend/optimizer/util/clauses.c
Doc: clarify that CREATE TABLE discards redundant unique constraints.
commit : c6f8d17d04d1bf1ddcbe0f2293d8f1462a1379f4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 13:09:47 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 13:09:47 -0500
The SQL standard says that redundant unique constraints are disallowed,
but we long ago decided that throwing an error would be too
user-unfriendly, so we just drop redundant ones. The docs weren't very
clear about that though, as this behavior was only explained for PRIMARY
KEY vs UNIQUE, not UNIQUE vs UNIQUE.
While here, I couldn't resist doing some copy-editing and markup-fixing
on the adjacent text about INCLUDE options.
Per bug #16767 from Matthias vd Meent.
Discussion: https://postgr.es/m/16767-1714a2056ca516d0@postgresql.org
M doc/src/sgml/ref/create_table.sgml
Doc: explain that the string types can't store \0 (ASCII NUL).
commit : c5ba66077054e05f07f4e1c80d588f3f3c374b1c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 12:06:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 8 Dec 2020 12:06:19 -0500
This restriction was mentioned in connection with string literals,
but it wasn't made clear that it's a general restriction not just
a syntactic limitation in query strings.
Per unsigned documentation comment.
Discussion: https://postgr.es/m/160720552914.710.16625261471128631268@wrigleys.postgresql.org
M doc/src/sgml/datatype.sgml
pgcrypto: Detect errors with EVP calls from OpenSSL
commit : dfd8bf2b9255f361d5541260a83ce634216c40f3
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Dec 2020 15:22:38 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 8 Dec 2020 15:22:38 +0900
The following routines are called within pgcrypto when handling digests
but there were no checks for failures:
- EVP_MD_CTX_size (can fail with -1 as of 3.0.0)
- EVP_MD_CTX_block_size (can fail with -1 as of 3.0.0)
- EVP_DigestInit_ex
- EVP_DigestUpdate
- EVP_DigestFinal_ex
A set of elog(ERROR) is added by this commit to detect such failures,
that should never happen except in the event of a processing failure
internal to OpenSSL.
Note that it would be possible to use ERR_reason_error_string() to get
more context about such errors, but these refer mainly to the internals
of OpenSSL, so it is not really obvious how useful that would be. This
is left out for simplicity.
Per report from Coverity. Thanks to Tom Lane for the discussion.
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
jit: Correct parameter type for generated expression evaluation functions.
commit : 01c6370a32e5875a63400c6e465de775a51ef1b8
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:21:06 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:21:06 -0800
clang only uses the 'i1' type for scalar booleans, not for pointers to
booleans (as the pointer might be pointing into a larger memory
allocation). Therefore a pointer-to-bool needs to the "storage" boolean.
There's no known case of wrong code generation due to this, but it seems quite
possible that it could cause problems (see e.g. 72559438f92).
Author: Andres Freund
Discussion: https://postgr.es/m/20201207212142.wz5tnbk2jsaqzogb@alap3.anarazel.de
Backpatch: 11-, where jit support was added
M src/backend/jit/llvm/llvmjit_expr.c
jit: configure: Explicitly reference 'native' component.
commit : 4f64daf73af76cbf32a01c7cba1c3a6fccf3062a
author : Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:12:23 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 7 Dec 2020 18:12:23 -0800
Until recently 'native' was implicitly included via 'orcjit', but a change
included in LLVM 11 (not yet released) removed a number of such indirect
component references.
Reported-By: Fabien COELHO <coelho@cri.ensmp.fr>
Reported-By: Andres Freund <andres@anarazel.de>
Reported-By: Thomas Munro <thomas.munro@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20201201064949.mex6kvi2kygby3ni@alap3.anarazel.de
Backpatch: 11-, where jit support was added
M config/llvm.m4
M configure
backpatch "jit: Add support for LLVM 12."
commit : 6a192c77d21b5eafc7f431cbf5e7ecdd6c8b5462
author : Andres Freund <andres@anarazel.de>
date : Mon, 9 Nov 2020 20:01:33 -0800
committer: Andres Freund <andres@anarazel.de>
date : Mon, 9 Nov 2020 20:01:33 -0800
As there haven't been problem on the buildfarm due to this change,
backpatch 6c57f2ed16e now.
Author: Andres Freund
Discussion: https://postgr.es/m/20201016011244.pmyvr3ee2gbzplq4@alap3.anarazel.de
Backpatch: 11-, where jit support was added
M src/backend/jit/llvm/llvmjit.c
M src/tools/pgindent/typedefs.list
Fix more race conditions in the newly-added pg_rewind test.
commit : e6dc04d436f1c5f173fc8b6e2f722f2d53719a92
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 7 Dec 2020 14:44:34 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 7 Dec 2020 14:44:34 +0200
pg_rewind looks at the control file to check what timeline a server is on.
But promotion doesn't immediately write a checkpoint, it merely writes
an end-of-recovery WAL record. If pg_rewind runs immediately after
promotion, before the checkpoint has completed, it will think think that
the server is still on the earlier timeline. We ran into this issue a long
time ago already, see commit 484a848a73f.
It's a bit bogus that pg_rewind doesn't determine the timeline correctly
until the end-of-recovery checkpoint has completed. We probably should
fix that. But for now work around it by waiting for the checkpoint
to complete before running pg_rewind, like we did in commit 484a848a73f.
In the passing, tidy up the new test a little bit. Rerder the INSERTs so
that the comments make more sense, remove a spurious CHECKPOINT call after
pg_rewind has already run, and add --debug option, so that if this fails
again, we'll have more data.
Per buildfarm failure at https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2020-12-06%2018%3A32%3A19&stg=pg_rewind-check.
Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/1713707e-e318-761c-d287-5b6a4aa807e8@iki.fi
M src/bin/pg_rewind/t/008_min_recovery_point.pl
Fix missed step in removal of useless RESULT RTEs in the planner.
commit : 7d43b76f6ed97b3b27f30114636f7b116009ef61
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Dec 2020 16:16:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 5 Dec 2020 16:16:13 -0500
Commit 4be058fe9 forgot that the append_rel_list would already be
populated at the time we remove useless result RTEs, and it might contain
PlaceHolderVars that need to be adjusted like the ones in the main parse
tree. This could lead to "no relation entry for relid N" failures later
on, when the planner tries to do something with an unadjusted PHV.
Per report from Tom Ellis. Back-patch to v12 where the bug came in.
Discussion: https://postgr.es/m/20201205173056.GF30712@cloudinit-builder
M src/backend/optimizer/prep/prepjointree.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Fix race conditions in newly-added test.
commit : e41a2efbca10438fa0a506d4158edd1a1964aacf
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Dec 2020 18:20:18 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 4 Dec 2020 18:20:18 +0200
Buildfarm has been failing sporadically on the new test. I was able to
reproduce this by adding a random 0-10 s delay in the walreceiver, just
before it connects to the primary. There's a race condition where node_3
is promoted before it has fully caught up with node_1, leading to diverged
timelines. When node_1 is later reconfigured as standby following node_3,
it fails to catch up:
LOG: primary server contains no more WAL on requested timeline 1
LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0
That's the situation where you'd need to use pg_rewind, but in this case
it happens already when we are just setting up the actual pg_rewind
scenario we want to test, so change the test so that it waits until
node_3 is connected and fully caught up before promoting it, so that you
get a clean, controlled failover.
Also rewrite some of the comments, for clarity. The existing comments
detailed what each step in the test did, but didn't give a good overview
of the situation the steps were trying to create.
For reasons I don't understand, the test setup had to be written slightly
differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version
needed node 1 to be reinitialized from backup, whereas in later versions
it could be shut down and reconfigured to be a standby. But even 9.5 should
support "clean switchover", where primary makes sure that pending WAL is
replicated to standby on shutdown. It would be nice to figure out what's
going on there, but that's independent of pg_rewind and the scenario that
this test tests.
Discussion: https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi
M src/bin/pg_rewind/t/008_min_recovery_point.pl
doc: remove unnecessary blank before command option text
commit : 7b0bd08a325fd4f21269dbc9a6dd82809342644d
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 11:33:24 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 11:33:24 -0500
Backpatch-through: 11
M doc/src/sgml/ref/pg_checksums.sgml
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
docs: list single-letter options first in command-line summary
commit : 610e9f5b361e19f160fa27747e47d5495eb0f2ba
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 10:28:58 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 3 Dec 2020 10:28:58 -0500
In a few places, the long-version options were listed before the
single-letter ones in the command summary of a few commands. This
didn't match other commands, and didn't match the option ordering later
in the same reference page.
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_controldata.sgml
M doc/src/sgml/ref/pg_resetwal.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
Fix pg_rewind bugs when rewinding a standby server.
commit : abd0abfb749d39f46682fe84a1c6f973d2d8ddc2
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Dec 2020 15:57:48 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 3 Dec 2020 15:57:48 +0200
If the target is a standby server, its WAL doesn't end at the last
checkpoint record, but at minRecoveryPoint. We must scan all the
WAL from the last common checkpoint all the way up to minRecoveryPoint
for modified pages, and also consider that portion when determining
whether the server needs rewinding.
Backpatch to all supported versions.
Author: Ian Barwick and me
Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com
M src/bin/pg_rewind/parsexlog.c
M src/bin/pg_rewind/pg_rewind.c
A src/bin/pg_rewind/t/008_min_recovery_point.pl
pg_checksums: data_checksum_version is unsigned so use %u not %d
commit : eec90ffbf86f77102e0238b39591a26667cab0db
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Dec 2020 20:27:06 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Dec 2020 20:27:06 -0500
While the previous behavior didn't generate a warning, we might as well
use an accurate *printf specification.
Backpatch-through: 12
M src/bin/pg_checksums/pg_checksums.c
Ensure that expandTableLikeClause() re-examines the same table.
commit : dffc82a5b9d48bded63e1beed718b24bbf58c6a4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Dec 2020 14:02:27 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Dec 2020 14:02:27 -0500
As it stood, expandTableLikeClause() re-did the same relation_openrv
call that transformTableLikeClause() had done. However there are
scenarios where this would not find the same table as expected.
We hold lock on the LIKE source table, so it can't be renamed or
dropped, but another table could appear before it in the search path.
This explains the odd behavior reported in bug #16758 when cloning a
table as a temp table of the same name. This case worked as expected
before commit 502898192 introduced the need to open the source table
twice, so we should fix it.
To make really sure we get the same table, let's re-open it by OID not
name. That requires adding an OID field to struct TableLikeClause,
which is a little nervous-making from an ABI standpoint, but as long
as it's at the end I don't think there's any serious risk.
Per bug #16758 from Marc Boeren. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/16758-840e84a6cfab276d@postgresql.org
M src/backend/nodes/copyfuncs.c
M src/backend/nodes/equalfuncs.c
M src/backend/nodes/outfuncs.c
M src/backend/parser/gram.y
M src/backend/parser/parse_utilcmd.c
M src/include/nodes/parsenodes.h
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Avoid memcpy() with a NULL source pointer and count == 0
commit : 5a1d1b9540a4f5bf4ee6761a17b21ad7c0012b49
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Dec 2020 11:46:56 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Dec 2020 11:46:56 -0300
When memcpy() is called on a pointer, the compiler is entitled to assume
that the pointer is not null, which can lead to optimizing nearby code
in potentially undesirable ways. We still want such optimizations
(gcc's -fdelete-null-pointer-checks) in cases where they're valid.
Related: commit 13bba02271dc.
Backpatch to pg11, where this particular instance appeared.
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Reported-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CAEudQApUndmQkr5fLrCKXQ7+ib44i7S+Kk93pyVThS85PnG3bQ@mail.gmail.com
Discussion: https://postgr.es/m/CALNJ-vSdhwSM5f4tnNn1cdLHvXMVe_S+V3nR5GwNrmCPNB2VtQ@mail.gmail.com
M src/backend/commands/indexcmds.c
Free disk space for dropped relations on commit.
commit : fd3a75d820a4fee3e25b699f1ccc043469afc55c
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Dec 2020 13:21:03 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 1 Dec 2020 13:21:03 +1300
When committing a transaction that dropped a relation, we previously
truncated only the first segment file to free up disk space (the one
that won't be unlinked until the next checkpoint).
Truncate higher numbered segments too, even though we unlink them on
commit. This frees the disk space immediately, even if other backends
have open file descriptors and might take a long time to get around to
handling shared invalidation events and closing them. Also extend the
same behavior to the first segment, in recovery.
Back-patch to all supported releases.
Bug: #16663
Reported-by: Denis Patron <denis.patron@previnet.it>
Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com>
Reviewed-by: Neil Chen <carpenter.nail.cz@gmail.com>
Reviewed-by: David Zhang <david.zhang@highgo.ca>
Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org
M src/backend/storage/smgr/md.c
Fix missing outfuncs.c support for IncrementalSortPath.
commit : a095e04f63a47ef02ec98577cc1fc4e4542e5ddd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 16:32:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 16:32:56 -0500
For debugging purposes, Path nodes are supposed to have outfuncs
support, but this was overlooked in the original incremental sort patch.
While at it, clean up a couple other minor oversights, as well as
bizarre choice of return type for create_incremental_sort_path().
(All the existing callers just cast it to "Path *" immediately, so
they don't care, but some future caller might care.)
outfuncs.c fix by Zhijie Hou, the rest by me
Discussion: https://postgr.es/m/324c4d81d8134117972a5b1f6cdf9560@G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/nodes/outfuncs.c
M src/backend/optimizer/README
M src/backend/optimizer/util/pathnode.c
M src/include/nodes/pathnodes.h
M src/include/optimizer/pathnode.h
Document concurrent indexes waiting on each other
commit : 3fe0e7c3fa27de80419de9ce66be2767d2ddae57
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Nov 2020 18:24:55 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 30 Nov 2020 18:24:55 -0300
Because regular CREATE INDEX commands are independent, and there's no
logical data dependency, it's not immediately obvious that transactions
held by concurrent index builds on one table will block the second phase
of concurrent index creation on an unrelated table, so document this
caveat.
Backpatch this all the way back. In branch master, mention that only
some indexes are involved.
Author: James Coleman <jtc331@gmail.com>
Reviewed-by: David Johnston <david.g.johnston@gmail.com>
Discussion: https://postgr.es/m/CAAaqYe994=PUrn8CJZ4UEo_S-FfRr_3ogERyhtdgHAb2WG_Ufg@mail.gmail.com
M doc/src/sgml/ref/create_index.sgml
Remove configure-time probe for DocBook DTD.
commit : d34916fee5c29098af17c2ef3fe29444d7f0f112
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 15:24:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 15:24:13 -0500
Checking for DocBook being installed was valuable when we were on the
OpenSP docs toolchain, because that was rather hard to get installed
fully. Nowadays, as long as you have xmllint and xsltproc installed,
you're good, because those programs will fetch the DocBook files off
the net at need. Moreover, testing this at configure time means that
a network access may well occur whether or not you have any interest
in building the docs later. That can be slow (typically 2 or 3
seconds, though much higher delays have been reported), and it seems
not very nice to be doing an off-machine access without warning, too.
Hence, drop the PGAC_CHECK_DOCBOOK probe, and adjust related
documentation. Without that macro, there's not much left of
config/docbook.m4 at all, so I just removed it.
Back-patch to v11, where we started to use xmllint in the
PGAC_CHECK_DOCBOOK probe.
Discussion: https://postgr.es/m/E2EE6B76-2D96-408A-B961-CAE47D1A86F0@yesql.se
Discussion: https://postgr.es/m/A55A7FC9-FA60-47FE-98B5-139CDC57CE6E@gmail.com
M aclocal.m4
D config/docbook.m4
M configure
M configure.in
M doc/src/sgml/docguide.sgml
Prevent parallel index build in a standalone backend.
commit : fac31b2cd4470124d7d68a7eebdb13cfff8b3d3d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 14:38:00 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 14:38:00 -0500
This can't work if there's no postmaster, and indeed the code got an
assertion failure trying. There should be a check on IsUnderPostmaster
gating the use of parallelism, as the planner has for ordinary
parallel queries.
Commit 40d964ec9 got this right, so follow its model of checking
IsUnderPostmaster at the same place where we check for
max_parallel_maintenance_workers == 0. In general, new code
implementing parallel utility operations should do the same.
Report and patch by Yulin Pei, cosmetically adjusted by me.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/HK0PR01MB22747D839F77142D7E76A45DF4F50@HK0PR01MB2274.apcprd01.prod.exchangelabs.com
M src/backend/optimizer/plan/planner.c
Fix miscomputation of direct_lateral_relids for join relations.
commit : 666a4de939bf78df7295c35899a5f2e89eaea382
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 12:22:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 30 Nov 2020 12:22:43 -0500
If a PlaceHolderVar is to be evaluated at a join relation, but
its value is only needed there and not at higher levels, we neglected
to update the joinrel's direct_lateral_relids to include the PHV's
source rel. This causes problems because join_is_legal() then won't
allow joining the joinrel to the PHV's source rel at all, leading
to "failed to build any N-way joins" planner failures.
Per report from Andreas Seltenreich. Back-patch to 9.5
where the problem originated.
Discussion: https://postgr.es/m/87blfgqa4t.fsf@aurora.ydns.eu
M src/backend/optimizer/util/placeholder.c
M src/test/regress/expected/join.out
M src/test/regress/sql/join.sql
Remove leftover comments, left behind by removal of WITH OIDS.
commit : 74d6fb0a037a7453693418e1069718a7f08ba31e
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 30 Nov 2020 10:26:43 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 30 Nov 2020 10:26:43 +0200
Author: Amit Langote
Discussion: https://www.postgresql.org/message-id/CA%2BHiwqGaRoF3XrhPW-Y7P%2BG7bKo84Z_h%3DkQHvMh-80%3Dav3wmOw%40mail.gmail.com
M contrib/file_fdw/file_fdw.c
M src/backend/commands/copy.c
Fix recently-introduced breakage in psql's \connect command.
commit : 72b930f5045ee5a44e222cee1ceb76d4c341d4b3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 15:22:04 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 15:22:04 -0500
Through my misreading of what the existing code actually did,
commits 85c54287a et al. broke psql's behavior for the case where
"\c connstring" provides a password in the connstring. We should
use that password in such a case, but as of 85c54287a we ignored it
(and instead, prompted for a password).
Commit 94929f1cf fixed that in HEAD, but since I thought it was
cleaning up a longstanding misbehavior and not one I'd just created,
I didn't back-patch it.
Hence, back-patch the portions of 94929f1cf having to do with
password management. In addition to fixing the introduced bug,
this means that "\c -reuse-previous=on connstring" will allow
re-use of an existing connection's password if the connstring
doesn't change user/host/port. That didn't happen before, but
it seems like a bug fix, and anyway I'm loath to have significant
differences in this code across versions.
Also fix an error with the same root cause about whether or not to
override a connstring's setting of client_encoding. As of 85c54287a
we always did so; restore the previous behavior of overriding only
when stdin/stdout are a terminal and there's no environment setting
of PGCLIENTENCODING. (I find that definition a bit surprising, but
right now doesn't seem like the time to revisit it.)
Per bug #16746 from Krzysztof Gradek. As with the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/16746-44b30e2edf4335d4@postgresql.org
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Doc: clarify behavior of PQconnectdbParams().
commit : 1eb499a8a5e6aa06f5bd8d53f2119e74efdd3db7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 13:58:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 29 Nov 2020 13:58:30 -0500
The documentation omitted the critical tidbit that a keyword-array entry
is simply ignored if its corresponding value-array entry is NULL or an
empty string; it will *not* override any previously-obtained value for
the parameter. (See conninfo_array_parse().) I'd supposed that would
force the setting back to default, which is what led me into bug #16746;
but it doesn't.
While here, I couldn't resist the temptation to do some copy-editing,
both in the description of PQconnectdbParams() and in the section
about connection URI syntax.
Discussion: https://postgr.es/m/931505.1606618746@sss.pgh.pa.us
M doc/src/sgml/libpq.sgml
Retry initial slurp_file("current_logfiles"), in test 004_logrotate.pl.
commit : 9a02dbdd03e8eadbe5331311c122548fac156f56
author : Noah Misch <noah@leadboat.com>
date : Sat, 28 Nov 2020 21:52:27 -0800
committer: Noah Misch <noah@leadboat.com>
date : Sat, 28 Nov 2020 21:52:27 -0800
Buildfarm member topminnow failed when the test script attempted this
before the syslogger would have created the file. Back-patch to v12,
which introduced the test.
M src/bin/pg_ctl/t/004_logrotate.pl
Fix a recently-introduced race condition in LISTEN/NOTIFY handling.
commit : f5de090cc175072b9ececcf48f55ffc502c06add
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Nov 2020 14:03:40 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 28 Nov 2020 14:03:40 -0500
Commit 566372b3d fixed some race conditions involving concurrent
SimpleLruTruncate calls, but it introduced new ones in async.c.
A newly-listening backend could attempt to read Notify SLRU pages that
were in process of being truncated, possibly causing an error. Also,
the QUEUE_TAIL pointer could become set to a value that's not equal to
the queue position of any backend. While that's fairly harmless in
v13 and up (thanks to commit 51004c717), in older branches it resulted
in near-permanent disabling of the queue truncation logic, so that
continued use of NOTIFY led to queue-fill warnings and eventual
inability to send any more notifies. (A server restart is enough to
make that go away, but it's still pretty unpleasant.)
The core of the problem is confusion about whether QUEUE_TAIL
represents the "logical" tail of the queue (i.e., the oldest
still-interesting data) or the "physical" tail (the oldest data we've
not yet truncated away). To fix, split that into two variables.
QUEUE_TAIL regains its definition as the logical tail, and we
introduce a new variable to track the oldest un-truncated page.
Per report from Mikael Gustavsson. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/1b8561412e8a4f038d7a491c8b922788@smhi.se
M src/backend/commands/async.c
Fix CLUSTER progress reporting of number of blocks scanned.
commit : dcc20946a8fc192a71cedcaae25c996973747ff5
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 20:16:44 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 27 Nov 2020 20:16:44 +0900
Previously pg_stat_progress_cluster view reported the current block
number in heap scan as the number of heap blocks scanned (i.e.,
heap_blks_scanned). This reported number could be incorrect when
synchronize_seqscans is enabled, because it allowed the heap scan to
start at block in middle. This could result in wraparounds in the
heap_blks_scanned column when the heap scan wrapped around.
This commit fixes the bug by calculating the number of blocks from
the block that the heap scan starts at to the current block in scan,
and reporting that number in the heap_blks_scanned column.
Also, in pg_stat_progress_cluster view, previously heap_blks_scanned
could not reach heap_blks_total at the end of heap scan phase
if the last pages scanned were empty. This commit fixes the bug by
manually updating heap_blks_scanned to the same value as
heap_blks_total when the heap scan phase finishes.
Back-patch to v12 where pg_stat_progress_cluster view was introduced.
Reported-by: Matthias van de Meent
Author: Matthias van de Meent
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAEze2WjCBWSGkVfYag001Rc4+-nNLDpWM7QbyD6yPvuhKs-gYQ@mail.gmail.com
M src/backend/access/heap/heapam_handler.c
In psql's \d commands, don't truncate attribute default values.
commit : a0ef0817204dbbcb326b14071e827bf181cfa2cb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 16:19:25 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 25 Nov 2020 16:19:25 -0500
Historically, psql has truncated the text of a column's default
expression at 128 characters. This is unlike any other behavior
in describe.c, and it's become particularly confusing now that
the limit is only applied to the expression proper and not to
the "generated always as (...) stored" text that may get wrapped
around it.
Excavation in our git history suggests that the original motivation
for this limit was not really to limit the display width (as I'd long
supposed), but to make it safe to use a fixed-width output buffer to
store the result. That implementation restriction is long gone of
course, but the limit remained. Let's just get rid of it.
While here, rearrange the logic about when to free the output string
so that it's not so dependent on unstated assumptions about the
possible values of attidentity and attgenerated.
Per bug #16743 from David Turon. Back-patch to v12 where GENERATED
came in. (Arguably we could take it back further, but I'm hesitant
to change the behavior of long-stable branches for this.)
Discussion: https://postgr.es/m/16743-7b1bacc4af76e7ad@postgresql.org
M src/bin/psql/describe.c
doc: Fix typos
commit : 7ef52b5d5de42cdd7d29f6bce7d7a07a9d4c6345
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 09:49:00 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 25 Nov 2020 09:49:00 +0100
Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/20201121194105.GO24784@telsasoft.com
M doc/src/sgml/contrib.sgml
Remove obsolete comment atop ri_PlanCheck.
commit : 5f734acd70dc202a58b9de5992dd5ef401120e88
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 25 Nov 2020 09:21:33 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 25 Nov 2020 09:21:33 +0530
Commit 5b7ba75f7f removed the unused parameter but forgot to update the
nearby comments.
Author: Li Japin
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/0E2F62A2-B2F1-4052-83AE-F0BEC8A75789@hotmail.com
M src/backend/utils/adt/ri_triggers.c
Properly check index mark/restore in ExecSupportsMarkRestore.
commit : 6dda057043df4a56683e8fa31a74737b63da4c47
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 24 Nov 2020 20:58:32 +0000
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Tue, 24 Nov 2020 20:58:32 +0000
Previously this code assumed that all IndexScan nodes supported
mark/restore, which is not true since it depends on optional index AM
support functions. This could lead to errors about missing support
functions in rare edge cases of mergejoins with no sort keys, where an
unordered non-btree index scan was placed on the inner path without a
protecting Materialize node. (Normally, the fact that merge join
requires ordered input would avoid this error.)
Backpatch all the way since this bug is ancient.
Per report from Eugen Konkov on irc.
Discussion: https://postgr.es/m/87o8jn50be.fsf@news-spur.riddles.org.uk
M src/backend/executor/execAmi.c
M src/backend/optimizer/util/plancat.c
M src/include/nodes/pathnodes.h
Skip allocating hash table in EXPLAIN-only mode.
commit : 340ae3cfb8f38a49664c0da3ff033346e92f0450
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 20 Nov 2020 14:41:14 +0200
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 20 Nov 2020 14:41:14 +0200
This is a backpatch of commit 2cccb627f1, backpatched due to popular
demand. Backpatch to all supported versions.
Author: Alexey Bashtanov
Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc
M src/backend/executor/nodeAgg.c
On macOS, use -isysroot in link steps as well as compile steps.
commit : 9e9a31bd009663bcdd5e676d6012ee691d8944b6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Nov 2020 00:58:26 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 20 Nov 2020 00:58:26 -0500
We previously put the -isysroot switch only into CPPFLAGS, theorizing
that it was only needed to find the right copies of include files.
However, it seems that we also need to use it while linking programs,
to find the right stub ".tbd" files for libraries. We got away
without that up to now, but apparently that was mostly luck. It may
also be that failures are only observed when the Xcode version is
noticeably out of sync with the host macOS version; the case that's
prompting action right now is that builds fail when using latest Xcode
(12.2) on macOS Catalina, even though it's fine on Big Sur.
Hence, add -isysroot to LDFLAGS as well. (It seems that the more
common practice is to put it in CFLAGS, whence it'd be included at
both compile and link steps. However, we can't mess with CFLAGS in
the platform template file without confusing configure's logic for
choosing default CFLAGS.)
Back-patch of 49407dc32 into all supported branches.
Report and patch by James Hilliard (some cosmetic mods by me)
Discussion: https://postgr.es/m/20201120003314.20560-1-james.hilliard1@gmail.com
M configure
M configure.in
M src/template/darwin
Further fixes for CREATE TABLE LIKE: cope with self-referential FKs.
commit : 98f3f1d5c1bff02a25a0af7dcf3a4676dddf58e1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Nov 2020 15:03:17 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 19 Nov 2020 15:03:17 -0500
Commit 502898192 was too careless about the order of execution of the
additional ALTER TABLE operations generated by expandTableLikeClause.
It just stuck them all at the end, which seems okay for most purposes.
But it falls down in the case where LIKE is importing a primary key
or unique index and the outer CREATE TABLE includes a FOREIGN KEY
constraint that needs to depend on that index. Weird as that is,
it used to work, so we ought to keep it working.
To fix, make parse_utilcmd.c insert LIKE clauses between index-creation
and FK-creation commands in the transformed list of commands, and change
utility.c so that the commands generated by expandTableLikeClause are
executed immediately not at the end. One could imagine scenarios where
this wouldn't work either; but currently expandTableLikeClause only
makes column default expressions, CHECK constraints, and indexes, and
this ordering seems fine for those.
Per bug #16730 from Sofoklis Papasofokli. Like the previous patch,
back-patch to all supported branches.
Discussion: https://postgr.es/m/16730-b902f7e6e0276b30@postgresql.org
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Don't Insert() a VFD entry until it's fully built.
commit : 95d39547d83999aff728806e36fd740ee97bd86a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 20:32:35 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 20:32:35 -0500
Otherwise, if FDDEBUG is enabled, the debugging output fails because
it tries to read the fileName, which isn't set up yet (and should in
fact always be NULL).
AFAICT, this has been wrong since Berkeley. Before 96bf88d52,
it would accidentally fail to crash on platforms where snprintf()
is forgiving about being passed a NULL pointer for %s; but the
file name intended to be included in the debug output wouldn't
ever have shown up.
Report and fix by Greg Nancarrow. Although this is only visibly
broken in custom-made builds, it still seems worth back-patching
to all supported branches, as the FDDEBUG code is pretty useless
as it stands.
Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com
M src/backend/storage/file/fd.c
Do not return NULL for error cases in satisfies_hash_partition().
commit : fea5960fafb0002ea2a80bed1dc03e3a4f85fa1d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 16:39:59 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 16:39:59 -0500
Since this function is used as a CHECK constraint condition,
returning NULL is tantamount to returning TRUE, which would have the
effect of letting in a row that doesn't satisfy the hash condition.
Admittedly, the cases for which this is done should be unreachable
in practice, but that doesn't make it any less a bad idea. It also
seems like a dartboard was used to decide which error cases should
throw errors as opposed to returning NULL.
For the checks for NULL input values, I just switched it to returning
false. There's some argument that an error would be better; but the
case really should be can't-happen in a generated hash constraint,
so it's likely not worth more code for.
For the parent-relation-open-failure case, it seems like we might
as well let relation_open throw an error, instead of having an
impossible-to-diagnose constraint failure.
Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/24067.1605134819@sss.pgh.pa.us
M src/backend/partitioning/partbounds.c
M src/test/regress/expected/hash_part.out
Use "true" not "TRUE" in one ICU function call.
commit : 53c7b4f6221be2800ba49840ce29cb1b5c0b1ab7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 15:16:39 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 16 Nov 2020 15:16:39 -0500
This was evidently missed in commit 6337865f3, which generally did
s/TRUE/true/ everywhere. It escaped notice up to now because ICU
versions before ICU 68 provided definitions of "TRUE" and "FALSE"
regardless. With ICU 68, it fails to compile.
Per report from Condor. Back-patch to v11 where 6337865f3 came in.
(I've not tested v10, where this call originated, but I imagine
it's fine since we defined TRUE in c.h back then.)
Discussion: https://postgr.es/m/7a6f3336165bfe3ca66abcda7966f9d0@stz-bg.com
M src/backend/commands/collationcmds.c
doc: update bgwriter description
commit : b7fc2593233f5bd1e651852c6d7780cef561a797
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 13:13:43 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 13:13:43 -0500
This clarifies exactly what the bgwriter does, which should help with
tuning.
Reported-by: Chris Wilson
Discussion: https://postgr.es/m/160399562040.7809.7335281028960123489@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
doc: clarify how to find pg_type_d.h in the install tree
commit : f75a7bb6c2c99d6759a823785f379691403ec9b2
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 12:36:17 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 12:36:17 -0500
Followup to patch 152ed04799.
Reported-by: Alvaro Herrera
Discussion: https://postgr.es/m/20201112202900.GA28098@alvherre.pgsql
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
doc: improve wording of the need for analyze of exp. indexes
commit : d8395970ea9b46354f009b384ac9b9ff66c4410c
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 10:26:17 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 16 Nov 2020 10:26:17 -0500
This is a followup commit on 3370207986.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20201112211143.GL30691@telsasoft.com
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
Fix typo
commit : 89e0ee9a7bada0101e1dff175cd002d16a364f94
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Nov 2020 10:54:11 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 16 Nov 2020 10:54:11 -0300
Introduced in 90fdc259866e; backpatch to 12.
Author: Erik Rijkers <er@xs4all.nl>
Discussion: https://postgr.es/m/e92b3fba98a0c0f7afc0a2a37e765954@xs4all.nl
M doc/src/sgml/ref/create_index.sgml
Fix fuzzy thinking about amcanmulticol versus amcaninclude.
commit : 7c89246d0bb233be7d6670f0a8f024e99423e8cc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Nov 2020 16:10:48 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 15 Nov 2020 16:10:48 -0500
These flags should be independent: in particular an index AM should
be able to say that it supports include columns without necessarily
supporting multiple key columns. The included-columns patch got
this wrong, possibly aided by the fact that it didn't bother to
update the documentation.
While here, clarify some text about amcanreturn, which was a little
vague about what should happen when amcanreturn reports that only
some of the index columns are returnable.
Noted while reviewing the SP-GiST included-columns patch, which
quite incorrectly (and unsafely) changed SP-GiST to claim
amcanmulticol = true as a workaround for this bug.
Backpatch to v11 where included columns were introduced.
M doc/src/sgml/indexam.sgml
M src/backend/commands/indexcmds.c
Doc: improve partitioning discussion in ddl.sgml.
commit : 0e0e71abdcaeb8a3887094de078b77cb35bd03ba
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Nov 2020 13:09:53 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 14 Nov 2020 13:09:53 -0500
This started with the intent to explain that range upper bounds
are exclusive, which previously you could only find out by reading
the CREATE TABLE man page. But I soon found that section 5.11
really could stand a fair amount of editorial attention. It's
apparently been revised several times without much concern for
overall flow, nor careful copy-editing.
Back-patch to v11, which is as far as the patch goes easily.
Per gripe from Edson Richter. Thanks to David Johnston for review.
Discussion: https://postgr.es/m/DM6PR13MB3988736CF8F5DC5720440231CFE60@DM6PR13MB3988.namprd13.prod.outlook.com
M doc/src/sgml/ddl.sgml
doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h
commit : 3f93b3431f8b9539eb2147f29029de50d3d43ec7
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:13:01 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:13:01 -0500
These files are in compiled directories and install directories.
Reported-by: e.indrupskaya@postgrespro.ru
Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
docs: mention that expression indexes need analyze
commit : e4b5e5f7fdedda5092065c546a8c92bf1355464e
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:00:44 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 15:00:44 -0500
Expression indexes can't benefit from pre-computed statistics on
columns.
Reported-by: Nikolay Samokhvalov
Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com
Author: Nikolay Samokhvalov, modified
Backpatch-through: 9.5
M doc/src/sgml/ref/create_index.sgml
doc: wire protocol data type for history file content is bytea
commit : 52003bf3c4b4f7e6d6a3aeb4fff0f6aca210b3df
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 14:33:28 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 12 Nov 2020 14:33:28 -0500
Document that though the history file content is marked as bytea, it is
the same a text, and neither is btyea-escaped or encoding converted.
Reported-by: Brar Piening
Discussion: https://postgr.es/m/6a1b9cd9-17e3-df67-be55-86102af6bdf5@gmx.de
Backpatch-through: 13 - 9.5 (not master)
M doc/src/sgml/protocol.sgml
M src/backend/replication/walsender.c
pg_trgm: fix crash in 2-item picksplit
commit : 48ab1fa304fe51d563bea11f1334572a7f2832b1
author : Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 12 Nov 2020 14:34:37 +0000
committer: Andrew Gierth <rhodiumtoad@postgresql.org>
date : Thu, 12 Nov 2020 14:34:37 +0000
Whether from size overflow in gistSplit or from secondary splits,
picksplit is (rarely) called with exactly two items to split.
Formerly, due to special-case handling of the last item, this would
lead to access to an uninitialized cache entry; prior to PG 13 this
might have been harmless or at worst led to an incorrect union datum,
but in 13 onwards it can cause a backend crash from using an
uninitialized pointer.
Repair by removing the special case, which was deemed not to have been
appropriate anyway. Backpatch all the way, because this bug has
existed since pg_trgm was added.
Per report on IRC from user "ftzdomino". Analysis and testing by me,
patch from Alexander Korotkov.
Discussion: https://postgr.es/m/87k0usfdxg.fsf@news-spur.riddles.org.uk
M contrib/pg_trgm/trgm_gist.c
Fix typo in contrib/pg_trgm/pg_trgm--1.4--1.5.sql
commit : 6058f22324c8781b1914551ab4da431224c3a7dd
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 08:55:09 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 08:55:09 +0300
Backpatch-through: 13
M contrib/pg_trgm/pg_trgm–1.4–1.5.sql
Fix name of the macro for getting signature length trgm_gist.c
commit : 065683bbdbab7fe51c59a479f4136328577bbecd
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 06:19:16 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Thu, 12 Nov 2020 06:19:16 +0300
911e702077 has introduced the opclass parameters including signature length
for a set of GiST opclasses. Due to copy-pasting, macro for getting the
signature length in trgm_gist.c was named LTREE_GET_ASIGLEN(). Fix that by
renaming this macro to just GET_SIGLEN().
Backpatch-through: 13
M contrib/pg_trgm/trgm_gist.c
Remove useless SHA256 initialization when not using backup manifests
commit : 9a94b925317ec963befffaa7e5edc38a62c2b88f
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Nov 2020 10:56:40 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 12 Nov 2020 10:56:40 +0900
Attempting to take a base backup with Postgres linking to a build of
OpenSSL with FIPS enabled currently fails with or even without a backup
manifest requested because of this mandatory SHA256 initialization used
for the manifest file itself. However, there is no need to do this
initialization at all if backup manifests are not needed because there
is no data to append to the manifest.
Note that being able to use backup manifests with OpenSSL+FIPS requires
a switch of the SHA2 implementation to use EVP, which would cause an ABI
breakage so this cannot be backpatched to 13 as it has been already
released, but at least avoiding this SHA256 initialization gives users
the possibility to take a base backup even when specifying --no-manifest
with pg_basebackup.
Author: Michael Paquier
Discussion: https://postgr.es/m/20201110020014.GE1887@paquier.xyz
Backpatch-through: 13
M src/backend/replication/backup_manifest.c
Remove duplicate code in brin_memtuple_initialize
commit : c885e4a2f93cff543a7e0ccaaf82b8148fc53401
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 11 Nov 2020 18:37:36 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Wed, 11 Nov 2020 18:37:36 +0100
Commit 8bf74967dab moved some of the code from brin_new_memtuple to
brin_memtuple_initialize, but this resulted in some of the code being
duplicate. Fix by removing the duplicate lines and backpatch to 10.
Author: Tomas Vondra
Backpatch-through: 10
Discussion: https://postgr.es/m/5eb50c97-9a8e-b691-8c40-1b2a55611c4c%40enterprisedb.com
M src/backend/access/brin/brin_tuple.c
Fix and simplify some usages of TimestampDifference().
commit : afce7908d7062d94ac60fd4de5f98aaed134c2c7
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 22:51:19 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 22:51:19 -0500
Introduce TimestampDifferenceMilliseconds() to simplify callers
that would rather have the difference in milliseconds, instead of
the select()-oriented seconds-and-microseconds format. This gets
rid of at least one integer division per call, and it eliminates
some apparently-easy-to-mess-up arithmetic.
Two of these call sites were in fact wrong:
* pg_prewarm's autoprewarm_main() forgot to multiply the seconds
by 1000, thus ending up with a delay 1000X shorter than intended.
That doesn't quite make it a busy-wait, but close.
* postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute
microseconds not milliseconds, thus ending up with a delay 1000X longer
than intended. Somebody along the way had noticed this problem but
misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather
than fixing the units. This was relatively harmless in context, because
we don't care that much about exactly how long this delay is; still,
it's wrong.
There are a few more callers of TimestampDifference() that don't
have a direct need for seconds-and-microseconds, but can't use
TimestampDifferenceMilliseconds() either because they do need
microsecond precision or because they might possibly deal with
intervals long enough to overflow 32-bit milliseconds. It might be
worth inventing another API to improve that, but that seems outside
the scope of this patch; so those callers are untouched here.
Given the fact that we are fixing some bugs, and the likelihood
that future patches might want to back-patch code that uses this
new API, back-patch to all supported branches.
Alexey Kondratov and Tom Lane
Discussion: https://postgr.es/m/3b1c053a21c07c1ed5e00be3b2b855ef@postgrespro.ru
M contrib/pg_prewarm/autoprewarm.c
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xlog.c
M src/backend/replication/walreceiverfuncs.c
M src/backend/replication/walsender.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/timestamp.h
doc: fix spelling "connction" to "connection"
commit : 19fd4f20b6a75058ca5be8037da529bb8cd55898
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Nov 2020 19:18:35 -0500
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 10 Nov 2020 19:18:35 -0500
Was wrong in commit 1a9388bd0f.
Reported-by: Tom Lane, Justin Pryzby
Discussion: https://postgr.es/m/20201102063333.GE22691@telsasoft.com
Backpatch-through: 9.5
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
Work around cross-version-upgrade issues created by commit 9e38c2bb5.
commit : 5c456d30807136e47ea936d67946cfada3f0e71c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 18:32:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 10 Nov 2020 18:32:36 -0500
Summarily changing the STYPE of regression-test aggregates that
depend on array_append or array_cat is an issue for the buildfarm's
cross-version-upgrade tests, because those aggregates (as defined
in the back branches) now won't load into HEAD. Although this seems
like only a minimal risk for genuine user-defined aggregates, we
need to do something for the buildfarm. Hence, adjust the aggregate
definitions, in both HEAD and the back branches.
Discussion: https://postgr.es/m/1401824.1604537031@sss.pgh.pa.us
Discussion: https://postgr.es/m/E1kaQ2c-0005lx-Eg@gemulon.postgresql.org
M src/test/regress/expected/polymorphism.out
M src/test/regress/sql/polymorphism.sql
Stamp 13.1.
commit : 6daf725a9c66e880fd76d25279ce00710535e030
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 17:24:30 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 17:24:30 -0500
M configure
M configure.in
Last-minute updates for release notes.
commit : 90cf59c8c8c66c8b0b4c719b6f7ba8fce60b87e1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 13:02:13 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 13:02:13 -0500
Security: CVE-2020-25694, CVE-2020-25695, CVE-2020-25696
M doc/src/sgml/release-13.sgml
Doc: clarify data type behavior of COALESCE and NULLIF.
commit : f47841fbad8a7f6dd579db68d782857697ad25c6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 12:02:24 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 9 Nov 2020 12:02:24 -0500
After studying the code, NULLIF is a lot more subtle than you might
have guessed.
Discussion: https://postgr.es/m/160486028730.25500.15740897403028593550@wrigleys.postgresql.org
M doc/src/sgml/func.sgml
M doc/src/sgml/typeconv.sgml
Ignore attempts to \gset into specially treated variables.
commit : 67029845b08d93108d53f572e2d58334c850126f
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
If an interactive psql session used \gset when querying a compromised
server, the attacker could execute arbitrary code as the operating
system account running psql. Using a prefix not found among specially
treated variables, e.g. every lowercase string, precluded the attack.
Fix by issuing a warning and setting no variable for the column in
question. Users wanting the old behavior can use a prefix and then a
meta-command like "\set HISTSIZE :prefix_HISTSIZE". Back-patch to 9.5
(all supported versions).
Reviewed by Robert Haas. Reported by Nick Cleaton.
Security: CVE-2020-25696
M src/bin/psql/common.c
M src/bin/psql/variables.c
M src/bin/psql/variables.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
In security-restricted operations, block enqueue of at-commit user code.
commit : c90c84b3f797a54a40ebc6795fbd743bdf44adad
author : Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
committer: Noah Misch <noah@leadboat.com>
date : Mon, 9 Nov 2020 07:32:09 -0800
Specifically, this blocks DECLARE ... WITH HOLD and firing of deferred
triggers within index expressions and materialized view queries. An
attacker having permission to create non-temp objects in at least one
schema could execute arbitrary SQL functions under the identity of the
bootstrap superuser. One can work around the vulnerability by disabling
autovacuum and not manually running ANALYZE, CLUSTER, REINDEX, CREATE
INDEX, VACUUM FULL, or REFRESH MATERIALIZED VIEW. (Don't restore from
pg_dump, since it runs some of those commands.) Plain VACUUM (without
FULL) is safe, and all commands are fine when a trusted user owns the
target object. Performance may degrade quickly under this workaround,
however. Back-patch to 9.5 (all supported versions).
Reviewed by Robert Haas. Reported by Etienne Stalmans.
Security: CVE-2020-25695
M contrib/postgres_fdw/connection.c
M src/backend/access/transam/xact.c
M src/backend/commands/portalcmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Translation updates
commit : 62e7ae75f441e7c91f446b05f5b206fe01e34f0c
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 Nov 2020 12:34:05 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 9 Nov 2020 12:34:05 +0100
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 2ffedf5ea37677f39cdc1eb92a1e78762cd3fb0e
M src/backend/po/de.po
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ja.po
M src/backend/po/ko.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/backend/po/uk.po
M src/bin/initdb/po/cs.po
M src/bin/initdb/po/ko.po
M src/bin/initdb/po/ru.po
M src/bin/initdb/po/uk.po
M src/bin/pg_archivecleanup/po/cs.po
M src/bin/pg_archivecleanup/po/ko.po
M src/bin/pg_archivecleanup/po/uk.po
M src/bin/pg_basebackup/po/cs.po
M src/bin/pg_basebackup/po/ko.po
M src/bin/pg_basebackup/po/ru.po
M src/bin/pg_basebackup/po/uk.po
M src/bin/pg_checksums/po/cs.po
M src/bin/pg_checksums/po/ko.po
M src/bin/pg_checksums/po/uk.po
M src/bin/pg_config/po/cs.po
M src/bin/pg_config/po/ko.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_controldata/po/cs.po
M src/bin/pg_controldata/po/ko.po
M src/bin/pg_controldata/po/uk.po
M src/bin/pg_ctl/po/cs.po
M src/bin/pg_ctl/po/ko.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/po/cs.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_dump/po/ko.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
M src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/po/cs.po
M src/bin/pg_resetwal/po/ko.po
M src/bin/pg_resetwal/po/uk.po
M src/bin/pg_rewind/po/cs.po
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ko.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/uk.po
M src/bin/pg_upgrade/po/cs.po
M src/bin/pg_upgrade/po/de.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/fr.po
M src/bin/pg_upgrade/po/ko.po
M src/bin/pg_upgrade/po/ru.po
M src/bin/pg_upgrade/po/sv.po
M src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/nls.mk
M src/bin/pg_verifybackup/po/es.po
A src/bin/pg_verifybackup/po/ko.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_verifybackup/po/uk.po
M src/bin/pg_waldump/po/cs.po
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/uk.po
M src/bin/psql/po/cs.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ko.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/uk.po
M src/bin/scripts/po/cs.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/ko.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/uk.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/preproc/po/cs.po
M src/interfaces/ecpg/preproc/po/de.po
M src/interfaces/ecpg/preproc/po/fr.po
M src/interfaces/ecpg/preproc/po/ko.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/po/cs.po
M src/interfaces/libpq/po/ko.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/uk.po
M src/pl/plperl/po/uk.po
M src/pl/plpgsql/src/po/cs.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/ko.po
M src/pl/plpgsql/src/po/ru.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpython/po/uk.po
M src/pl/tcl/po/uk.po
Remove incorrect %s in string
commit : 7d94c017b94577548a0656b2aef7ce8be18fbc1b
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 9 Nov 2020 10:36:49 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 9 Nov 2020 10:36:49 +0100
Appears to have been a copy/paste error in the original commit that
moved the messages to fe_utils/.
Author: Tang, Haiying <tanghy.fnst@cn.fujitsu.com>
Backpatch-through: 13
Discussion: https://postgr.es/m/3321cbcea76d4d2c8320a05c19b9304a@G08CNEXMBPEKD05.g08.fujitsu.local
M src/fe_utils/cancel.c
Release notes for 13.1, 12.5, 11.10, 10.15, 9.6.20, 9.5.24.
commit : 213774a45b9259ea797cd6b5e1e9a74bd7d12d28
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 15:16:12 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 15:16:12 -0500
M doc/src/sgml/release-13.sgml
In INSERT/UPDATE, use the table's real tuple descriptor as target.
commit : 7aeb6404f0aa250e75b7156d21ebe12d0ec2d1c8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 13:08:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 8 Nov 2020 13:08:36 -0500
This back-patches commit 20d3fe900 into the v12 and v13 branches.
At the time I thought that commit was not fixing any observable
bug, but Bertrand Drouvot showed otherwise: adding a dropped column
to the previously-considered scenario crashes v12 and v13, unless the
dropped column happens to be an integer. That is, of course, because
the tupdesc we derive from the plan output tlist fails to describe
the dropped column accurately, so that we'll do the wrong thing with
a tuple in which that column isn't NULL.
There is no bug in pre-v12 branches because they already did use
the table's real tuple descriptor for any trigger-returned tuple.
It seems that this set of bugs can be blamed on the changes that
removed es_trig_tuple_slot, though I've not attempted to pin that
down precisely.
Although there's no code change needed in HEAD, update the test case
to include a dropped column there too.
Discussion: https://postgr.es/m/db5d97c8-f48a-51e2-7b08-b73d5434d425@amazon.com
Discussion: https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org
M src/backend/commands/trigger.c
M src/backend/executor/execJunk.c
M src/backend/executor/nodeModifyTable.c
M src/include/executor/executor.h
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix test for error message change
commit : 5ca6f685b834518187b15926d196c5dbb086efe7
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 8 Nov 2020 07:49:07 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 8 Nov 2020 07:49:07 +0100
fix for f3ad4fddfaf71e8f6f037cd627f398ba43625ca1
M src/test/ssl/t/002_scram.pl
Message style improvements
commit : 99f9384ea91761cdb957d051b0ca64179320b2ed
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 7 Nov 2020 19:33:43 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 7 Nov 2020 19:33:43 -0300
* Avoid pointlessly highlighting that an index vacuum was executed by a
parallel worker; user doesn't care.
* Don't give the impression that a non-concurrent reindex of an invalid
index on a TOAST table would work, because it wouldn't.
* Add a "translator:" comment for a mysterious message.
Discussion: https://postgr.es/m/20201107034943.GA16596@alvherre.pgsql
Reviewed-by: Michael Paquier <michael@paquier.xyz>
M src/backend/access/heap/vacuumlazy.c
M src/backend/commands/indexcmds.c
M src/backend/libpq/be-secure-openssl.c
Fix redundant error messages in client tools
commit : f3ad4fddfaf71e8f6f037cd627f398ba43625ca1
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 22:15:52 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 7 Nov 2020 22:15:52 +0100
A few client tools duplicate error messages already provided by libpq.
Discussion: https://www.postgresql.org/message-id/flat/3e937641-88a1-e697-612e-99bba4b8e5e4%40enterprisedb.com
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/psql/startup.c
Avoid re-using output variables in new ecpg test case.
commit : 3459f4169ba9665fbc7965165ec4ef83170b748b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 16:25:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 16:25:42 -0500
The buildfarm thinks this leads to memory stomps, though annoyingly
I can't duplicate that here. The existing code in strings.pgc is
doing something that doesn't seem to be sanctioned at all really
by the documentation, but I'm disinclined to try to make that nicer
right now. Let's just declare some more output variables in hopes
of working around it.
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/preproc/strings.h
M src/interfaces/ecpg/test/preproc/strings.pgc
Doc: small release note updates.
commit : 7fb326e04b5c367f03b2ebb85348e79a722afef9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 15:27:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 15:27:36 -0500
Add items committed in the last 24 hours. Also correct my failure
to credit Nikita Glukhov for 52ad1e659, which did all the heavy
lifting for 3db322eaa.
M doc/src/sgml/release-13.sgml
Fix ecpg's mishandling of B'...' and X'...' literals.
commit : 1bccb159af5813b8f34fd177acdbdb2ad82cd596
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 15:03:44 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 7 Nov 2020 15:03:44 -0500
These were broken in multiple ways:
* The xbstart and xhstart lexer actions neglected to set
"state_before_str_start" before transitioning to the xb/xh states,
thus possibly resulting in "internal error: unreachable state" later.
* The test for valid string contents at the end of xb state was flat out
wrong, as it accounted incorrectly for the "b" prefix that the xbstart
action had injected. Meanwhile, the xh state had no such check at all.
* The generated literal value failed to include any quote marks.
* The grammar did the wrong thing anyway, typically ignoring the
literal value and emitting something else, since BCONST and XCONST
tokens were handled randomly differently from SCONST tokens.
The first of these problems is evidently an oversight in commit
7f380c59f, but the others seem to be very ancient. The lack of
complaints shows that ECPG users aren't using these syntaxes much
(although I do vaguely remember one previous complaint).
As written, this patch is dependent on 7f380c59f, so it can't go
back further than v13. Given the shortage of complaints, I'm not
excited about adapting the patch to prior branches.
Report and patch by Shenhao Wang (test case adjusted by me)
Discussion: https://postgr.es/m/d6402f1bacb74ecba22ef715dbba17fd@G08CNEXMBPEKD06.g08.fujitsu.local
M src/interfaces/ecpg/preproc/ecpg.trailer
M src/interfaces/ecpg/preproc/ecpg.type
M src/interfaces/ecpg/preproc/parse.pl
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-strings.c
M src/interfaces/ecpg/test/expected/preproc-strings.stderr
M src/interfaces/ecpg/test/expected/preproc-strings.stdout
M src/interfaces/ecpg/test/preproc/strings.pgc
Plug memory leak in index_get_partition
commit : d94d37f8c0c77cf1b9c5ae924bb6cfc12f4bc692
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 6 Nov 2020 22:52:15 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 6 Nov 2020 22:52:15 -0300
The list of indexes was being leaked when asked for an index that
doesn't have an index partition in the table partition. Not a common
case admittedly --and in most cases where it occurs, caller throws an
error anyway-- but worth fixing for cleanliness and in case any
third-party code is calling this function.
While at it, remove use of lfirst_oid() to obtain a value we already
have.
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20201105203606.GF22691@telsasoft.com
M src/backend/catalog/partition.c
Properly detoast data in brin_form_tuple
commit : 6a7b55f3716fad9c40ecb960cb7b7d616d5b02fd
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 7 Nov 2020 00:40:06 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sat, 7 Nov 2020 00:40:06 +0100
brin_form_tuple failed to consider the values may be toasted, inserting
the toast pointer into the index. This may easily result in index
corruption, as the toast data may be deleted and cleaned up by vacuum.
The cleanup however does not care about indexes, leaving invalid toast
pointers behind, which triggers errors like this:
ERROR: missing chunk number 0 for toast value 16433 in pg_toast_16426
A less severe consequence are inconsistent failures due to the index row
being too large, depending on whether brin_form_tuple operated on plain
or toasted version of the row. For example
CREATE TABLE t (val TEXT);
INSERT INTO t VALUES ('... long value ...')
CREATE INDEX idx ON t USING brin (val);
would likely succeed, as the row would likely include toast pointer.
Switching the order of INSERT and CREATE INDEX would likely fail:
ERROR: index row size 8712 exceeds maximum 8152 for index "idx"
because this happens before the row values are toasted.
The bug exists since PostgreSQL 9.5 where BRIN indexes were introduced.
So backpatch all the way back.
Author: Tomas Vondra
Reviewed-by: Alvaro Herrera
Backpatch-through: 9.5
Discussion: https://postgr.es/m/20201001184133.oq5uq75sb45pu3aw@development
Discussion: https://postgr.es/m/20201104010544.zexj52mlldagzowv%40development
M src/backend/access/brin/brin_tuple.c
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
First-draft release notes for 13.1.
commit : c66a3225e07b5098a796f24588a6b81bfdedd2fd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 17:05:43 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 17:05:43 -0500
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
Also as usual for a .1 release, there are some entries here that
are not really relevant for v13 because they already appeared in 13.0.
Those'll be removed later.
M doc/src/sgml/release-13.sgml
Revert "Accept relations of any kind in LOCK TABLE".
commit : 4352c2394a547e5797c512cbaf0d0b7ca824747a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 16:17:56 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 16:17:56 -0500
Revert 59ab4ac32, as well as the followup fix 33862cb9c, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Revert "pg_dump: Lock all relations, not just plain tables".
commit : fa3840c8800f290b8352c5c81714a4439d2e1f46
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 15:48:21 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 6 Nov 2020 15:48:21 -0500
Revert 403a3d91c, as well as the followup fix 7f4235032, in all
branches. We need to think a bit harder about what the behavior
of LOCK TABLE on views should be, and there's no time for that
before next week's releases. We'll take another crack at this
later.
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
Don't throw an error for LOCK TABLE on a self-referential view.
commit : 44b973b91029cb5aecf09d589bdf3f05cfddaa60
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 Nov 2020 11:44:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 5 Nov 2020 11:44:32 -0500
LOCK TABLE has complained about "infinite recursion" when applied
to a self-referential view, ever since we made it recurse into views
in v11. However, that breaks pg_dump's new assumption that it's
okay to lock every relation. There doesn't seem to be any good
reason to throw an error: if we just abandon the recursion, we've
still satisfied the requirement of locking every referenced relation.
Per bug #16703 from Andrew Bille (via Alexander Lakhin).
Discussion: https://postgr.es/m/16703-e348f58aab3cf6cc@postgresql.org
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
Fix nbtree cleanup-only VACUUM stats inaccuracies.
commit : 02c9386ca4f706364904be2720e2d09916e2b619
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 4 Nov 2020 18:42:24 -0800
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 4 Nov 2020 18:42:24 -0800
Logic for counting heap TIDs from posting list tuples (added by commit
0d861bbb) was faulty. It didn't count any TIDs/index tuples in the
event of no callback being set. This meant that we incorrectly counted
no index tuples in clean-up only VACUUMs, which could lead to
pg_class.reltuples being spuriously set to 0 in affected indexes.
To fix, go back to counting items from the page in cases where there is
no callback. This approach isn't very accurate, but it works well
enough in practice while avoiding the expense of accessing every index
tuple during cleanup-only VACUUMs.
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
https://postgr.es/m/20201023174451.69e358f1@firost
Backpatch: 13-, where nbtree deduplication was introduced
M src/backend/access/nbtree/nbtree.c
Enable hash partitioning of text arrays
commit : 82d4a2a7d63e79f6a6724f366cfaa4beed6b8326
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Nov 2020 07:47:06 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 4 Nov 2020 07:47:06 +0100
hash_array_extended() needs to pass PG_GET_COLLATION() to the hash
function of the element type. Otherwise, the hash function of a
collation-aware data type such as text will error out, since the
introduction of nondeterministic collation made hash functions require
a collation, too.
The consequence of this is that before this change, hash partitioning
using an array over text in the partition key would not work.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/32c1fdae-95c6-5dc6-058a-a90330a3b621%40enterprisedb.com
M src/backend/utils/adt/arrayfuncs.c
M src/test/regress/expected/collate.icu.utf8.out
M src/test/regress/sql/collate.icu.utf8.sql
Use INT64_FORMAT to print int64 variables in sort debug
commit : 7d39586a59c6f5816a068bd38e1e4887d4c984ff
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:43:12 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:43:12 +0100
Commit 6ee3b5fb99 cleaned up most of the long/int64 confusion related to
incremental sort, but the sort debug messages were still using %ld for
int64 variables. So fix that.
Author: Haiying Tang
Backpatch-through: 13, where the incremental sort code was added
Discussion: https://postgr.es/m/4250be9d350c4992abb722a76e288aef%40G08CNEXMBPEKD05.g08.fujitsu.local
M src/backend/executor/nodeIncrementalSort.c
Fix get_useful_pathkeys_for_relation for volatile expressions
commit : 2d26c4ac703447a002a02124c1edd01e70a5d1ee
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:07:23 +0100
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Tue, 3 Nov 2020 20:07:23 +0100
When considering Incremental Sort below a Gather Merge, we need to be
a bit more careful when matching pathkeys to EC members. It's not enough
to find a member whose Vars are all in the current relation's target;
volatile expressions in particular need to be contained in the target,
otherwise it's too early to use the pathkey.
Reported-by: Jaime Casanova
Author: James Coleman
Reviewed-by: Tomas Vondra
Backpatch-through: 13, where the incremental sort code was added
Discussion: https://postgr.es/m/CAJGNTeNaxpXgBVcRhJX%2B2vSbq%2BF2kJqGBcvompmpvXb7pq%2BoFA%40mail.gmail.com
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/equivclass.c
M src/include/optimizer/paths.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/sql/incremental_sort.sql
Guard against core dump from uninitialized subplan.
commit : 936043c9eacb9e9c7356a8190a410d2c4e4ea03a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 16:16:36 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 16:16:36 -0500
If the planner erroneously puts a non-parallel-safe SubPlan into
a parallelized portion of the query tree, nodeSubplan.c will fail
in the worker processes because it finds a null in es_subplanstates,
which it's unable to cope with. It seems worth a test-and-elog to
make that an error case rather than a core dump case.
This probably should have been included in commit 16ebab688, which
was responsible for allowing nulls to appear in es_subplanstates
to begin with. So, back-patch to v10 where that came in.
Discussion: https://postgr.es/m/924226.1604422326@sss.pgh.pa.us
M src/backend/executor/nodeSubplan.c
Allow users with BYPASSRLS to alter their own passwords.
commit : 768dbef0d49826c2e404ceb1567b3cc9e2bbc30a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:41:32 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 3 Nov 2020 15:41:32 -0500
The intention in commit 491c029db was to require superuserness to
change the BYPASSRLS property, but the actual effect of the coding
in AlterRole() was to require superuserness to change anything at all
about a BYPASSRLS role. Other properties of a BYPASSRLS role should
be changeable under the same rules as for a normal role, though.
Fix that, and also take care of some documentation omissions related
to BYPASSRLS and REPLICATION role properties.
Tom Lane and Stephen Frost, per bug report from Wolfgang Walther.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/a5548a9f-89ee-3167-129d-162b5985fcf8@technowledgy.de
M doc/src/sgml/ref/alter_role.sgml
M doc/src/sgml/ref/create_role.sgml
M src/backend/commands/user.c
Disallow ALTER TABLE ONLY / DROP EXPRESSION
commit : 53977598174652dcb06bc2b26674c29b9ae601cc
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 15:14:50 +0100
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 3 Nov 2020 15:14:50 +0100
The current implementation cannot handle this correctly, so just
forbid it for now.
GENERATED clauses must be attached to the column definition and cannot
be added later like DEFAULT, so if a child table has a generation
expression that the parent does not have, the child column will
necessarily be an attlocal column. So to implement ALTER TABLE ONLY /
DROP EXPRESSION, we'd need extra code to update attislocal of the
direct child tables, somewhat similar to how DROP COLUMN does it, so
that the resulting state can be properly dumped and restored.
Discussion: https://www.postgresql.org/message-id/flat/15830.1575468847%40sss.pgh.pa.us
M src/backend/commands/tablecmds.c
M src/test/regress/expected/generated.out
M src/test/regress/sql/generated.sql
Fix unportable use of getnameinfo() in pg_hba_file_rules view.
commit : a58a631b4af0c027c07ea7cc4110a60b5f279ddf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 21:11:50 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 21:11:50 -0500
fill_hba_line() thought it could get away with passing sizeof(struct
sockaddr_storage) rather than the actual addrlen previously returned
by getaddrinfo(). While that appears to work on many platforms,
it does not work on FreeBSD 11: you get back a failure, which leads
to the view showing NULL for the address and netmask columns in all
rows. The POSIX spec for getnameinfo() is pretty clearly on
FreeBSD's side here: you should pass the actual address length.
So it seems plausible that there are other platforms where this
coding also fails, and we just hadn't noticed.
Also, IMO the fact that getnameinfo() failure leads to a NULL output
is pretty bogus in itself. Our pg_getnameinfo_all() wrapper is
careful to emit "???" on failure, and we should use that in such
cases. NULL should only be emitted in rows that don't have IP
addresses.
Per bug #16695 from Peter Vandivier. Back-patch to v10 where this
code was added.
Discussion: https://postgr.es/m/16695-a665558e2f630be7@postgresql.org
M src/backend/libpq/hba.c
M src/include/libpq/hba.h
Second thoughts on TOAST decompression.
commit : 7957e75c588c0b17210d4379afb50ea2673b0d20
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 11:25:18 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 2 Nov 2020 11:25:18 -0500
On detecting a corrupted match tag, pglz_decompress() should just
summarily return -1. Breaking out of the loop, as I did in dfc797730,
doesn't quite guarantee that will happen. Also, we can use
unlikely() on that check, just in case it helps.
Backpatch to v13, like the previous patch.
M src/common/pg_lzcompress.c
Add missing comma in list of SSL versions
commit : 57fae192f8f7d094159c913f10fcfd11cd827332
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 2 Nov 2020 15:20:19 +0100
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 2 Nov 2020 15:20:19 +0100
M doc/src/sgml/sslinfo.sgml
Fix some grammar and typos in comments and docs
commit : 796885a0713374f7bdcc3edec136e0527fcafafa
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 15:15:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 15:15:20 +0900
The documentation fixes are backpatched down to where they apply.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20201031020801.GD3080@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/auto-explain.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ref/pg_rewind.sgml
Extend PageIsVerified() to handle more custom options
commit : 017e78a3edc261143c7d791cafca5a5ad326a679
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 10:41:23 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 2 Nov 2020 10:41:23 +0900
This is useful for checks of relation pages without having to load the
pages into the shared buffers, and two cases can make use of that: page
verification in base backups and the online, lock-safe, flavor.
Compatibility is kept with past versions using a routine that calls the
new extended routine with the set of options compatible with the
original version. Contrary to d401c576, a macro cannot be used as there
may be external code relying on the presence of the original routine.
This is applied down to 11, where this will be used by a follow-up
commit addressing a set of issues with page verification in base
backups.
Extracted from a larger patch by the same author.
Author: Anastasia Lubennikova
Reviewed-by: Michael Paquier, Julien Rouhaud
Discussion: https://postgr.es/m/608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru
Backpatch-through: 11
M src/backend/catalog/storage.c
M src/backend/storage/buffer/bufmgr.c
M src/backend/storage/page/bufpage.c
M src/include/storage/bufpage.h
Fix two issues in TOAST decompression.
commit : 2330f4d3a87ac43b6ecd31bfd698384888ed03cb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 18:38:42 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 18:38:42 -0500
pglz_maximum_compressed_size() potentially underestimated the amount
of compressed data required to produce N bytes of decompressed data;
this is a fault in commit 11a078cf8.
Separately from that, pglz_decompress() failed to protect itself
against corrupt compressed data, particularly off == 0 in a match
tag. Commit c60e520f6 turned such a situation into an infinite loop,
where before it'd just have resulted in garbage output.
The combination of these two bugs seems like it may explain bug #16694
from Tom Vijlbrief, though it's impossible to be quite sure without
direct inspection of the failing session. (One needs to assume that
the pglz_maximum_compressed_size() bug caused us to fail to fetch the
second byte of a match tag, and what happened to be there instead was
a zero. The reported infinite loop is hard to explain without off == 0,
though.)
Aside from fixing the bugs, rewrite associated comments for more
clarity.
Back-patch to v13 where both these commits landed.
Discussion: https://postgr.es/m/16694-f107871e499ec114@postgresql.org
M src/common/pg_lzcompress.c
Avoid null pointer dereference if error result lacks SQLSTATE.
commit : 0041941f5bbe48ff3a05942efc6aa65f4f389efc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 11:26:16 -0500
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 1 Nov 2020 11:26:16 -0500
Although error results received from the backend should always have
a SQLSTATE field, ones generated by libpq won't, making this code
vulnerable to a crash after, say, untimely loss of connection.
Noted by Coverity.
Oversight in commit 403a3d91c. Back-patch to 9.5, as that was.
M src/bin/pg_dump/pg_backup_db.c
Preserve index data in pg_statistic across REINDEX CONCURRENTLY
commit : bb62df46bcaa109d5eb1907392034024dde0886e
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 1 Nov 2020 21:24:10 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 1 Nov 2020 21:24:10 +0900
Statistics associated to an index got lost after running REINDEX
CONCURRENTLY, while the non-concurrent case preserves these correctly.
The concurrent and non-concurrent operations need to be consistent for
the end-user, and missing statistics would force to wait for a new
analyze to happen, which could take some time depending on the activity
of the existing autovacuum workers. This issue is fixed by copying any
existing entries in pg_statistic associated to the old index to the new
one. Note that this copy is already done with the data of the index in
the stats collector.
Reported-by: Fabrízio de Royes Mello
Author: Michael Paquier, Fabrízio de Royes Mello
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com
Backpatch-through: 12
M src/backend/catalog/heap.c
M src/backend/catalog/index.c
M src/include/catalog/heap.h
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Reproduce debug_query_string==NULL on parallel workers.
commit : ab2e2ce466683b6af5ec956106cd905380d3d349
author : Noah Misch <noah@leadboat.com>
date : Sat, 31 Oct 2020 08:43:28 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 31 Oct 2020 08:43:28 -0700
Certain background workers initiate parallel queries while
debug_query_string==NULL, at which point they attempted strlen(NULL) and
died to SIGSEGV. Older debug_query_string observers allow NULL, so do
likewise in these newer ones. Back-patch to v11, where commit
7de4a1bcc56f494acbd0d6e70781df877dc8ecb5 introduced the first of these.
Discussion: https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/nbtree/nbtsort.c
Stabilize timetz test across DST transitions.
commit : ee03baad267dd83fa66a0ca4c1a7635f549fc1a6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Oct 2020 15:28:14 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 29 Oct 2020 15:28:14 -0400
The timetz test cases I added in commit a9632830b were unintentionally
sensitive to whether or not DST is active in the PST8PDT time zone.
Thus, they'll start failing this coming weekend, as reported by
Bernhard M. Wiedemann in bug #16689. Fortunately, DST-awareness is
not significant to the purpose of these test cases, so we can just
force them all to PDT (DST hours) to preserve stability of the
results.
Back-patch to v10, as the prior patch was.
Discussion: https://postgr.es/m/16689-57701daa23b377bf@postgresql.org
M src/test/regress/expected/timetz.out
M src/test/regress/sql/timetz.sql
Use mode "r" for popen() in psql's evaluate_backtick().
commit : ba4f5413e357faac2f33cd5d22db2a21c0be7727
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 14:35:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 14:35:53 -0400
In almost all other places, we use plain "r" or "w" mode in popen()
calls (the exceptions being for COPY data). This one has been
overlooked (possibly because it's buried in a ".l" flex file?),
but it's using PG_BINARY_R.
Kensuke Okamura complained in bug #16688 that we fail to strip \r
when stripping the trailing newline from a backtick result string.
That's true enough, but we'd also fail to convert embedded \r\n
cleanly, which also seems undesirable. Fixing the popen() mode
seems like the best way to deal with this.
It's been like this for a long time, so back-patch to all supported
branches.
Discussion: https://postgr.es/m/16688-c649c7b69cd7e6f8@postgresql.org
M src/bin/psql/psqlscanslash.l
Calculate extraUpdatedCols in query rewriter, not parser.
commit : 70492195be5e0283cf134c3c531ca0a23fdf9919
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 13:47:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 28 Oct 2020 13:47:02 -0400
It's unsafe to do this at parse time because addition of generated
columns to a table would not invalidate stored rules containing
UPDATEs on the table ... but there might now be dependent generated
columns that were not there when the rule was made. This also fixes
an oversight that rewriteTargetView failed to update extraUpdatedCols
when transforming an UPDATE on an updatable view. (Since the new
calculation is downstream of that, rewriteTargetView doesn't actually
need to do anything; but before, there was a demonstrable bug there.)
In v13 and HEAD, this leads to easily-visible bugs because (since
commit c6679e4fc) we won't recalculate generated columns that aren't
listed in extraUpdatedCols. In v12 this bitmap is mostly just used
for trigger-firing decisions, so you'd only notice a problem if a
trigger cared whether a generated column had been updated.
I'd complained about this back in May, but then forgot about it
until bug #16671 from Michael Paul Killian revived the issue.
Back-patch to v12 where this field was introduced. If existing
stored rules contain any extraUpdatedCols values, they'll be
ignored because the rewriter will overwrite them, so the bug will
be fixed even for existing rules. (But note that if someone were
to update to 13.1 or 12.5, store some rules with UPDATEs on tables
having generated columns, and then downgrade to a prior minor version,
they might observe issues similar to what this patch fixes. That
seems unlikely enough to not be worth going to a lot of effort to fix.)
Discussion: https://postgr.es/m/10206.1588964727@sss.pgh.pa.us
Discussion: https://postgr.es/m/16671-2fa55851859fb166@postgresql.org
M src/backend/optimizer/plan/setrefs.c
M src/backend/parser/analyze.c
M src/backend/replication/logical/worker.c
M src/backend/rewrite/rewriteHandler.c
M src/include/nodes/parsenodes.h
M src/include/parser/analyze.h
M src/include/rewrite/rewriteHandler.h
M src/test/regress/expected/updatable_views.out
M src/test/regress/sql/updatable_views.sql
Makefile comment: remove reference to tools/thread/thread_test
commit : b4395cc87ed5c49e0b9bfd0770383d4086fd9f12
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 14:00:38 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 14:00:38 -0400
You can't compile thread_test alone anymore, and the location moved too.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/1062278.1603819969@sss.pgh.pa.us
Backpatch-through: 9.5
M src/template/netbsd
pg_dump: Lock all relations, not just plain tables
commit : 64fc3e03495162154797f7d01e871462b1f42979
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 14:31:37 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 14:31:37 -0300
Now that LOCK TABLE can take any relation type, acquire lock on all
relations that are to be dumped. This prevents schema changes or
deadlock errors that could cause a dump to fail after expending much
effort. The server is tested to have the capability and the feature
disabled if it doesn't, so that a patched pg_dump doesn't fail when
connecting to an unpatched server.
Backpatch to 9.5.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_backup_db.h
M src/bin/pg_dump/pg_dump.c
Accept relations of any kind in LOCK TABLE
commit : 272bff6a35ef85b0226ac536c58ca24881c3c8d2
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 13:49:19 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 27 Oct 2020 13:49:19 -0300
The restriction that only tables and views can be locked by LOCK TABLE
is quite arbitrary, since the underlying mechanism can lock any relation
type. Drop the restriction so that programs such as pg_dump can lock
all relations they're interested in, preventing schema changes that
could cause a dump to fail after expending much effort.
Backpatch to 9.5.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
M doc/src/sgml/ref/lock.sgml
M src/backend/commands/lockcmds.c
M src/test/regress/expected/lock.out
M src/test/regress/sql/lock.sql
docs: remove reference to src/tools/thread
commit : d04c4a8f7066bdac71d012fa868ba9d84538632e
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 12:43:11 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 27 Oct 2020 12:43:11 -0400
This directory and the ability to build the thread test independently
were removed in commit 8a2121185b.
Reported-by: e.indrupskaya@postgrespro.ru
Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
doc: simplify wording of function error affects
commit : a645c08838d6d4ae55ac4df57cfd91f0b9e4f812
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 22:38:11 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 22:38:11 -0400
Reported-by: bob.henkel@gmail.com
Discussion: https://postgr.es/m/160324449781.693.8298142858847611071@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/plpgsql.sgml
doc: make blooms docs match reality
commit : 4747655111b0bae6e45729ee4b06156ea6c40a77
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 19:17:05 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 26 Oct 2020 19:17:05 -0400
Parallel execution changed the way bloom queries are executed, so update
the EXPLAIN output, and restructure the docs to be clearer and more
accurate.
Reported-by: Daniel Westermann
Discussion: https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM
Author: Daniel Westermann and me
Backpatch-through: 9.6
M doc/src/sgml/bloom.sgml
Fix corner case for a BEFORE ROW UPDATE trigger returning OLD.
commit : d88d8ad28484a745fffe036a4085f4674b3064bc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Oct 2020 13:57:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 25 Oct 2020 13:57:46 -0400
If the old row has any "missing" attributes that are supposed to
be retrieved from an associated tuple descriptor, the wrong things
happened because the trigger result is shoved directly into an
executor slot that lacks the missing-attribute data. Notably,
CHECK-constraint verification would incorrectly see those columns
as NULL, and so would RETURNING-list evaluation.
Band-aid around this by forcibly expanding the tuple before passing
it to the trigger function. (IMO it was a fundamental misdesign to
put the missing-attribute data into tuple constraints, which so
much of the system considers to be optional. But we're probably
stuck with that now, and will have to continue to apply band-aids
as we find other places with similar issues.)
Back-patch to v12. v11 would also have the issue, except that
commit 920311ab1 already applied a similar band-aid. That forced
expansion in more cases than seem really necessary, though, so
this isn't a directly equivalent fix.
Amit Langote, with some cosmetic changes by me
Discussion: https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Fix incorrect parameter name in a function header comment
commit : 563973bf066893b9386c1d026433708797361575
author : David Rowley <drowley@postgresql.org>
date : Sun, 25 Oct 2020 22:39:00 +1300
committer: David Rowley <drowley@postgresql.org>
date : Sun, 25 Oct 2020 22:39:00 +1300
Author: Zhijie Hou
Discussion: https://postgr.es/m/14cd74ea00204cc8a7ea5d738ac82cd1@G08CNEXMBPEKD05.g08.fujitsu.local
Backpatch-through: 12, where the mistake was introduced
M src/backend/utils/cache/lsyscache.c
Fix ancient bug in ecpg's pthread_once() emulation for Windows.
commit : fd048e0cb5aa45ef67b5b2b0446f4b1b6bdefbf3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Oct 2020 13:12:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 24 Oct 2020 13:12:08 -0400
We must not set the "done" flag until after we've executed the
initialization function. Otherwise, other threads can fall through
the initial unlocked test before initialization is really complete.
This has been seen to cause rare failures of ecpg's thread/descriptor
test, and it could presumably cause other sorts of misbehavior in
threaded ECPG-using applications, since ecpglib relies on
pthread_once() in several places.
Diagnosis and patch by me, based on investigation by Alexander Lakhin.
Back-patch to all supported branches (the bug dates to 2007).
Discussion: https://postgr.es/m/16685-d6cd241872c101d3@postgresql.org
M src/interfaces/ecpg/ecpglib/misc.c
Fix broken XML formatting in EXPLAIN output for incremental sorts.
commit : e4538708d58400c8c0336ebabca0b7bdb72e0ff6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 11:32:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 23 Oct 2020 11:32:33 -0400
The ExplainCloseGroup arguments for incremental sort usage data
didn't match the corresponding ExplainOpenGroup. This only matters
for XML-format output, which is probably why we'd not noticed.
Daniel Gustafsson, per bug #16683 from Frits Jalvingh
Discussion: https://postgr.es/m/16683-8005033324ad34e9@postgresql.org
M src/backend/commands/explain.c
Update time zone data files to tzdata release 2020d.
commit : 96ed2ae9360d9b89f695f00c2b6417c4e4d9fcba
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:23:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:23:47 -0400
DST law changes in Palestine, with a whopping 120 hours' notice.
Also some historical corrections for Palestine.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020d.
commit : 0e551533b46069ec3b909ce5159eed51b768ca52
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:15:22 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 22 Oct 2020 21:15:22 -0400
There's no functional change at all here, but I'm curious to see
whether this change successfully shuts up Coverity's warning about
a useless strcmp(), which appeared with the previous update.
Discussion: http://mm.icann.org/pipermail/tz/2020-October/029370.html
M src/timezone/README
M src/timezone/zic.c
Fix connection string handling in psql's \connect command.
commit : 2e4af411075c10c5007eb09bcb67abf7f825572d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 16:18:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 21 Oct 2020 16:18:40 -0400
psql's \connect claims to be able to re-use previous connection
parameters, but in fact it only re-uses the database name, user name,
host name (and possibly hostaddr, depending on version), and port.
This is problematic for assorted use cases. Notably, pg_dump[all]
emits "\connect databasename" commands which we would like to have
re-use all other parameters. If such a script is loaded in a psql run
that initially had "-d connstring" with some non-default parameters,
those other parameters would be lost, potentially causing connection
failure. (Thus, this is the same kind of bug addressed in commits
a45bc8a4f and 8e5793ab6, although the details are much different.)
To fix, redesign do_connect() so that it pulls out all properties
of the old PGconn using PQconninfo(), and then replaces individual
properties in that array. In the case where we don't wish to re-use
anything, get libpq's default settings using PQconndefaults() and
replace entries in that, so that we don't need different code paths
for the two cases.
This does result in an additional behavioral change for cases where
the original connection parameters allowed multiple hosts, say
"psql -h host1,host2", and the \connect request allows re-use of the
host setting. Because the previous coding relied on PQhost(), it
would only permit reconnection to the same host originally selected.
Although one can think of scenarios where that's a good thing, there
are others where it is not. Moreover, that behavior doesn't seem to
meet the principle of least surprise, nor was it documented; nor is
it even clear it was intended, since that coding long pre-dates the
addition of multi-host support to libpq. Hence, this patch is content
to drop it and re-use the host list as given.
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
Use fast checkpoint in PostgresNode::backup()
commit : ddc728d437a08c68638101ef01d742d6f4476675
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Oct 2020 14:37:26 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 21 Oct 2020 14:37:26 -0300
Should cause tests to be a bit faster
M src/test/perl/PostgresNode.pm
Fix ALTER TABLE .. ENABLE/DISABLE TRIGGER recursion
commit : 5f6463a20af183db10d372f16ddeb5690a92aa1b
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 20 Oct 2020 19:22:09 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 20 Oct 2020 19:22:09 -0300
More precisely, correctly handle the ONLY flag indicating not to
recurse. This was implemented in 86f575948c77 by recursing in
trigger.c, but that's the wrong place; use ATSimpleRecursion instead,
which behaves properly. However, because legacy inheritance has never
recursed in that situation, make sure to do that only for new-style
partitioning.
I noticed this problem while testing a fix for another bug in the
vicinity.
This has been wrong all along, so backpatch to 11.
Discussion: https://postgr.es/m/20201016235925.GA29829@alvherre.pgsql
M src/backend/commands/tablecmds.c
M src/backend/commands/trigger.c
M src/test/regress/expected/triggers.out
M src/test/regress/sql/triggers.sql
Avoid invalid alloc size error in shm_mq
commit : 1f53d0b9f45521a85e85b6dcab7c15a7d8b4b973
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:52:25 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 19 Oct 2020 08:52:25 +0200
In shm_mq_receive(), a huge payload could trigger an unjustified
"invalid memory alloc request size" error due to the way the buffer
size is increased.
Add error checks (documenting the upper limit) and avoid the error by
limiting the allocation size to MaxAllocSize.
Author: Markus Wanner <markus.wanner@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com
M src/backend/storage/ipc/shm_mq.c
Fix typo in commit 99ae342fc4.
commit : 0a1377760bcdfe837ea5f602a800ea97c668bf16
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 20 Oct 2020 08:31:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 20 Oct 2020 08:31:43 +0530
In v13, the id for max_parallel_maintenance_workers is defined differently
as compared to HEAD in docs, so adjust the docs accordingly.
Reported-by: Magnus Hagander and Tom Lane
Discussion: https://postgr.es/m/CABUevEyAFQZ_jvjY_KtRUWbci4YMyQC1QAMzDQAbLs=XCo3m5Q@mail.gmail.com
M doc/src/sgml/ref/vacuum.sgml
Fix connection string handling in src/bin/scripts/ programs.
commit : 1814f915b526d5022b3e2a6ce4ea3bcbe59abe2c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 19:03:46 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 19:03:46 -0400
When told to process all databases, clusterdb, reindexdb, and vacuumdb
would reconnect by replacing their --maintenance-db parameter with the
name of the target database. If that parameter is a connstring (which
has been allowed for a long time, though we failed to document that
before this patch), we'd lose any other options it might specify, for
example SSL or GSS parameters, possibly resulting in failure to connect.
Thus, this is the same bug as commit a45bc8a4f fixed in pg_dump and
pg_restore. We can fix it in the same way, by using libpq's rules for
handling multiple "dbname" parameters to add the target database name
separately. I chose to apply the same refactoring approach as in that
patch, with a struct to handle the command line parameters that need to
be passed through to connectDatabase. (Maybe someday we can unify the
very similar functions here and in pg_dump/pg_restore.)
Per Peter Eisentraut's comments on bug #16604. Back-patch to all
supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/createdb.sgml
M doc/src/sgml/ref/dropdb.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M src/bin/scripts/clusterdb.c
M src/bin/scripts/common.c
M src/bin/scripts/common.h
M src/bin/scripts/createdb.c
M src/bin/scripts/createuser.c
M src/bin/scripts/dropdb.c
M src/bin/scripts/dropuser.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/scripts_parallel.c
M src/bin/scripts/scripts_parallel.h
M src/bin/scripts/vacuumdb.c
Fix list-munging bug that broke SQL function result coercions.
commit : 25378db74fd97f2b10ad44d1f0b2e1f8b0a651f2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 14:33:02 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 14:33:02 -0400
Since commit 913bbd88d, check_sql_fn_retval() can either insert type
coercion steps in-line in the Query that produces the SQL function's
results, or generate a new top-level Query to perform the coercions,
if modifying the Query's output in-place wouldn't be safe. However,
it appears that the latter case has never actually worked, because
the code tried to inject the new Query back into the query list it was
passed ... which is not the list that will be used for later processing
when we execute the SQL function "normally" (without inlining it).
So we ended up with no coercion happening at run-time, leading to
wrong results or crashes depending on the datatypes involved.
While the regression tests look like they cover this area well enough,
through a huge bit of bad luck all the test cases that exercise the
separate-Query path were checking either inline-able cases (which
accidentally didn't have the bug) or cases that are no-ops at runtime
(e.g., varchar to text), so that the failure to perform the coercion
wasn't obvious. The fact that the cases that don't work weren't
allowed at all before v13 probably contributed to not noticing the
problem sooner, too.
To fix, get rid of the separate "flat" list of Query nodes and instead
pass the real two-level list that is going to be used later. I chose
to make the same change in check_sql_fn_statements(), although that has
no actual bug, just so that we don't need that data structure at all.
This is an API change, as evidenced by the adjustments needed to
callers outside functions.c. That's a bit scary to be doing in a
released branch, but so far as I can tell from a quick search,
there are no outside callers of these functions (and they are
sufficiently specific to our semantics for SQL-language functions that
it's not apparent why any extension would need to call them). In any
case, v13 already changed the API of check_sql_fn_retval() compared to
prior branches.
Per report from pinker. Back-patch to v13 where this code came in.
Discussion: https://postgr.es/m/1603050466566-0.post@n3.nabble.com
M src/backend/catalog/pg_proc.c
M src/backend/executor/functions.c
M src/backend/optimizer/util/clauses.c
M src/include/executor/functions.h
M src/test/regress/expected/rangefuncs.out
M src/test/regress/sql/rangefuncs.sql
Misc documentation fixes.
commit : 33acc6bc8795d8e3c2802c78e17e57a75a143904
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:28:54 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:28:54 +0300
- Misc grammar and punctuation fixes.
- Stylistic cleanup: use spaces between function arguments and JSON fields
in examples. For example "foo(a,b)" -> "foo(a, b)". Add semicolon after
last END in a few PL/pgSQL examples that were missing them.
- Make sentence that talked about "..." and ".." operators more clear,
by avoiding to end the sentence with "..". That makes it look the same
as "..."
- Fix syntax description for HAVING: HAVING conditions cannot be repeated
Patch by Justin Pryzby, per Yaroslav Schekin's report. Backpatch to all
supported versions, to the extent that the patch applies easily.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/dblink.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/ref/select_into.sgml
M doc/src/sgml/rules.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml
Fix TRUNCATE doc: ALTER SEQUENCE RESTART is now transactional.
commit : 3b5bf7b893fc70d298881cff14f7f0c82d8fee34
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:02:25 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 19:02:25 +0300
ALTER SEQUENCE RESTART was made transactional in commit 3d79013b97.
Backpatch to v10, where that was introduced.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/ref/truncate.sgml
Fix output of tsquery example in docs.
commit : f0b3d3bb89ae34281edc49de24f865af85671a37
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 18:50:33 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 18:50:33 +0300
The output for this query changed in commit 4e2477b7b8. Backport to 9.6
like that commit.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
In libpq for Windows, call WSAStartup once and WSACleanup not at all.
commit : d2074daebe1699f0d48fa63f6ba196b6426023ad
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 11:23:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 19 Oct 2020 11:23:51 -0400
The Windows documentation insists that every WSAStartup call should
have a matching WSACleanup call. However, if that ever had actual
relevance, it wasn't in this century. Every remotely-modern Windows
kernel is capable of cleaning up when a process exits without doing
that, and must be so to avoid resource leaks in case of a process
crash. Moreover, Postgres backends have done WSAStartup without
WSACleanup since commit 4cdf51e64 in 2004, and we've never seen any
indication of a problem with that.
libpq's habit of doing WSAStartup during connection start and
WSACleanup during shutdown is also rather inefficient, since a
series of non-overlapping connection requests leads to repeated,
quite expensive DLL unload/reload cycles. We document a workaround
for that (having the application call WSAStartup for itself), but
that's just a kluge. It's also worth noting that it's far from
uncommon for applications to exit without doing PQfinish, and
we've not heard reports of trouble from that either.
However, the real reason for acting on this is that recent
experiments by Alexander Lakhin show that calling WSACleanup
during PQfinish is triggering the symptom we occasionally see
that a process using libpq fails to emit expected stdio output.
Therefore, let's change libpq so that it calls WSAStartup only
once per process, during the first connection attempt, and never
calls WSACleanup at all.
While at it, get rid of the only other WSACleanup call in our code
tree, in pg_dump/parallel.c; that presumably is equally useless.
Back-patch of HEAD commit 7d00a6b2d.
Discussion: https://postgr.es/m/ac976d8c-03df-d6b8-025c-15a2de8d9af1@postgrespro.ru
M doc/src/sgml/libpq.sgml
M src/bin/pg_dump/parallel.c
M src/interfaces/libpq/fe-connect.c
Fix doc for full text search distance operator.
commit : f2f03f9bb10b0749eddea5ad1dd730926421f090
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 17:58:38 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 19 Oct 2020 17:58:38 +0300
Commit 028350f619 changed its behavior from "at most" to "exactly", but
forgot to update the documentation. Backpatch to 9.6.
Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
M doc/src/sgml/textsearch.sgml
Update link for pllua
commit : d49357c0dc51378aeded51ecb8a290d761ac5b8c
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 19 Oct 2020 13:47:09 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 19 Oct 2020 13:47:09 +0200
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/A05874AE-8771-4C61-A24E-0B6249B8F3C2@yesql.se
M doc/src/sgml/external-projects.sgml
Relax some asserts in merge join costing code
commit : 33a332bc1cff27c5138fe117ae56b6a6f476f30c
author : David Rowley <drowley@postgresql.org>
date : Tue, 20 Oct 2020 00:06:40 +1300
committer: David Rowley <drowley@postgresql.org>
date : Tue, 20 Oct 2020 00:06:40 +1300
In the planner, it was possible, given an extreme enough case containing a
large number of joins for the number of estimated rows to become infinite.
This could cause problems in initial_cost_mergejoin() where we perform
some calculations based on those row estimates.
A problem case, presented by Onder Kalaci showed an Assert failure from
an Assert checking outerstartsel <= outerendsel. In his test case this
was effectively NaN <= Inf, which is false. The NaN outerstartsel came
from multiplying the infinite outer_path_rows by 0.0.
In master, this problem was fixed by a90c950fc, however, that fix was too
invasive for the backbranches. Here we just relax the Asserts to allow
them to pass. The worst that appears to happen from this is that we show
NaN cost values and infinite row estimates in EXPLAIN. add_path() would
have had a hard time doing anything useful with such costs, but that does
not really matter as if the row estimates were even close to accurate,
such plan would not complete this side of the heat death of the universe.
Reported-by: Onder Kalaci
Backpatch: 9.5 to 13
Discussion: https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com
M src/backend/optimizer/path/costsize.c
Change the docs for PARALLEL option of Vacuum.
commit : 99ae342fc4ffe5f9a6ec7f540c5a31fb483b06e6
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Oct 2020 09:34:04 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 19 Oct 2020 09:34:04 +0530
The rules to choose the number of parallel workers to perform parallel
vacuum operation were not clearly specified.
Reported-by: Peter Eisentraut
Author: Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/36aa8aea-61b7-eb3c-263b-648e0cb117b7@2ndquadrant.com
M doc/src/sgml/ref/vacuum.sgml
Fix potential memory leak in pgcrypto
commit : 1bd9b2b2369608de7763b3947af2f59292152268
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Oct 2020 09:37:50 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 19 Oct 2020 09:37:50 +0900
When allocating a EVP context, it would have been possible to leak some
memory allocated directly by OpenSSL, that PostgreSQL lost track of if
the initialization of the context allocated failed. The cleanup can be
done with EVP_MD_CTX_destroy().
Note that EVP APIs exist since OpenSSL 0.9.7 and we have in the tree
equivalent implementations for older versions since ce9b75d (code
removed with 9b7cd59a as of 10~). However, in 9.5 and 9.6, the existing
code makes use of EVP_MD_CTX_destroy() and EVP_MD_CTX_create() without
an equivalent implementation when building the tree with OpenSSL 0.9.6
or older, meaning that this code is in reality broken with such versions
since it got introduced in e2838c5. As we have heard no complains about
that, it does not seem worth bothering with in 9.5 and 9.6, so I have
left that out for simplicity.
Author: Michael Paquier
Discussion: https://postgr.es/m/20201015072212.GC2305@paquier.xyz
Backpatch-through: 9.5
M contrib/pgcrypto/openssl.c
Doc: caution against misuse of 'now' and related datetime literals.
commit : d317fd7570ca69553eef9a9ec1825967c2680927
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 17 Oct 2020 16:02:47 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 17 Oct 2020 16:02:47 -0400
Section 8.5.1.4, which defines these literals, made only a vague
reference to the fact that they might be evaluated too soon to be
safe in non-interactive contexts. Provide a more explicit caution
against misuse. Also, generalize the wording in the related tip in
section 9.9.4: while it clearly described this problem, it implied
(or really, stated outright) that the problem only applies to table
DEFAULT clauses.
Per gripe from Tijs van Dam. Back-patch to all supported branches.
Discussion: https://postgr.es/m/c2LuRv9BiRT3bqIo5mMQiVraEXey_25B4vUn0kDqVqilwOEu_iVF1tbtvLnyQK7yDG3PFaz_GxLLPil2SDkj1MCObNRVaac-7j1dVdFERk8=@thalex.com
M doc/src/sgml/datatype.sgml
M doc/src/sgml/func.sgml
Update time zone data files to tzdata release 2020c.
commit : 3f26dca76343d2e4aca5e2070875c93057925dca
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:53:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:53:33 -0400
DST law changes in Morocco, Canadian Yukon, Fiji, Macquarie Island,
Casey Station (Antarctica). Historical corrections for France,
Hungary, Monaco.
M src/timezone/data/tzdata.zi
Sync our copy of the timezone library with IANA release tzcode2020c.
commit : e0cf5e9b226aae3df8ab8df7f5f7c8fe295be24e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:40:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 21:40:16 -0400
This changes zic's default output format from "-b fat" to "-b slim".
We were already using "slim" in v13/HEAD, so those branches drop
the explicit -b switch in the Makefiles. Instead, add an explicit
"-b fat" in v12 and before, so that we don't change the output file
format in those branches. (This is perhaps excessively conservative,
but we decided not to do so in a12079109, and I'll stick with that.)
Other non-cosmetic changes are to drop support for zic's long-obsolete
"-y" switch, and to ensure that strftime() does not change errno
unless it fails.
As usual with tzcode changes, back-patch to all supported branches.
M src/timezone/Makefile
M src/timezone/README
M src/timezone/strftime.c
M src/timezone/zic.c
M src/tools/msvc/Install.pm
Add missing error check in pgcrypto/crypt-md5.c.
commit : 3d338a46a4c342fa1a4e276e0c84532141f4264b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:59:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:59:13 -0400
In theory, the second px_find_digest call in px_crypt_md5 could fail
even though the first one succeeded, since resource allocation is
required. Don't skip testing for a failure. (If one did happen,
the likely result would be a crash rather than clean recovery from
an OOM failure.)
The code's been like this all along, so back-patch to all supported
branches.
Daniel Gustafsson
Discussion: https://postgr.es/m/AA8D6FE9-4AB2-41B4-98CB-AE64BA668C03@yesql.se
M contrib/pgcrypto/crypt-md5.c
Doc: tweak column widths in synchronous-commit-matrix table.
commit : 2cde0fd6fcaf70a8cf59972c36e86c00cc97e90c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:36:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 16 Oct 2020 11:36:34 -0400
Commit a97e85f2b caused "exceed the available area" warnings in PDF
builds. Fine-tune colwidth values to avoid that.
Back-patch to 9.6, like the prior patch. (This is of dubious value
before v13, since we were far from free of such warnings in older
branches. But we might as well keep the SGML looking the same in all
branches.)
Per buildfarm.
M doc/src/sgml/config.sgml
llvmjit: Work around bug in LLVM 3.9 causing crashes after 72559438f92.
commit : efc9a8e9800cc04ff7460c3c78f9dc2120f3aea6
author : Andres Freund <andres@anarazel.de>
date : Thu, 15 Oct 2020 17:38:00 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 15 Oct 2020 17:38:00 -0700
Unfortunately in LLVM 3.9 LLVMGetAttributeCountAtIndex(func, index)
crashes when called with an index that has 0 attributes. Since there's
no way to work around this in the C API, add a small C++ wrapper doing
so.
The only reason this didn't fail before 72559438f92 is that there
always are function attributes...
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20201016001254.w2nfj7gd74jmb5in@alap3.anarazel.de
Backpatch: 11-, like 72559438f92
M src/backend/jit/llvm/llvmjit.c
M src/backend/jit/llvm/llvmjit_wrap.cpp
M src/include/jit/llvmjit.h
pg_upgrade: remove C99 compiler req. from commit 3c0471b5fd
commit : 79fe23465d56e7a3e649fd95bdb4e8b0af27a376
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 20:37:20 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 20:37:20 -0400
This commit required support for inline variable definition, which is
not a requirement.
RELEASE NOTE AUTHOR: the author of commit 3c0471b5fd
(pg_upgrade/tablespaces) was Justin Pryzby, not me.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20201016001959.h24fkywfubkv2pc5@alap3.anarazel.de
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
pg_upgrade: generate check error for left-over new tablespace
commit : 59cfff65b10036b637a3f6e50d8f654e855d3b69
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 19:33:36 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 19:33:36 -0400
Previously, if pg_upgrade failed, and the user recreated the cluster but
did not remove the new cluster tablespace directory, a later pg_upgrade
would fail since the new tablespace directory would already exists.
This adds error reporting for this during check.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20200925005531.GJ23631@telsasoft.com
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
llvmjit: Also copy parameter / return value attributes from template functions.
commit : ae3e75abab222519aef90eb130d93c1ea745ac2e
author : Andres Freund <andres@anarazel.de>
date : Thu, 15 Oct 2020 13:39:41 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 15 Oct 2020 13:39:41 -0700
Previously we only copied the function attributes. That caused problems at
least on s390x: Because we didn't copy the 'zeroext' attribute for
ExecAggTransReparent()'s *IsNull parameters, expressions invoking it didn't
ensure that the upper bytes of the registers were zeroed. In the - relatively
rare - cases where not, ExecAggTransReparent() wrongly ended up in the
newValueIsNull branch due to the register not being zero. Subsequently causing
a crash.
It's quite possible that this would cause problems on other platforms, and in
other places than just ExecAggTransReparent() on s390x.
Thanks to Christoph (and the Debian project) for providing me with access to a
s390x machine, allowing me to debug this.
Reported-By: Christoph Berg
Author: Andres Freund
Discussion: https://postgr.es/m/20201015083246.kie5726xerdt3ael@alap3.anarazel.de
Backpatch: 11-, where JIT was added
M src/backend/jit/llvm/llvmjit.c
doc: improve description of synchronous_commit modes
commit : 264e517e68190767de7cd50734956db02c6dbef1
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 15:15:29 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 15 Oct 2020 15:15:29 -0400
Previously it wasn't clear exactly what each of the synchronous_commit
modes accomplished. This clarifies that, and adds a table describing it.
Only backpatched through 9.6 since 9.5 doesn't have all the options.
Reported-by: kghost0@gmail.com
Discussion: https://postgr.es/m/159741195522.14321.13812604195366728976@wrigleys.postgresql.org
Backpatch-through: 9.6
M doc/src/sgml/config.sgml
M src/include/access/xact.h
Fix query in new test to check tables are synced
commit : 9f783aea669f56c2e7c875ee1391949f234a2257
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 15 Oct 2020 09:48:36 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 15 Oct 2020 09:48:36 -0300
Rather than looking for tablesync workers, it is more reliable to see
the sync state of the tables.
Per note from Amit Kapila.
Discussion: https://postgr.es/m/CAA4eK1JSSD7FVwq+_rOme86jUZTQFzjsNU06hQ4-LiRt1xFmSg@mail.gmail.com
M src/test/subscription/t/100_bugs.pl
Handle EACCES errors from kevent() better.
commit : 47522ee00ddbe77280e4c063605b443ec1de3881
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 15 Oct 2020 18:23:30 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 15 Oct 2020 18:23:30 +1300
While registering for postmaster exit events, we have to handle a couple
of edge cases where the postmaster is already gone. Commit 815c2f09
missed one: EACCES must surely imply that PostmasterPid no longer
belongs to our postmaster process (or alternatively an unexpected
permissions model has been imposed on us). Like ESRCH, this should be
treated as a WL_POSTMASTER_DEATH event, rather than being raised with
ereport().
No known problems reported in the wild. Per code review from Tom Lane.
Back-patch to 13.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3624029.1602701929%40sss.pgh.pa.us
M src/backend/storage/ipc/latch.c
doc: Mention that toast_tuple_target affects also column marked as Main.
commit : 53c07dbbf36855a4f5b11654106654b18e00ee15
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 15 Oct 2020 11:04:07 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 15 Oct 2020 11:04:07 +0900
Previously it was documented that toast_tuple_target affected column
marked as only External or Extended. But this description is not correct
and toast_tuple_target affects also column marked as Main.
Back-patch to v11 where toast_tuple_target reloption was introduced.
Author: Shinya Okano
Reviewed-by: Tatsuhito Kasahara, Fujii Masao
Discussion: https://postgr.es/m/93f46e311a67422e89e770d236059817@oss.nttdata.com
M doc/src/sgml/ref/create_table.sgml
Restore replication protocol's duplicate command tags
commit : 72e43fc313e93c95704c574bcf98805805668063
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 14 Oct 2020 20:12:26 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 14 Oct 2020 20:12:26 -0300
I removed the duplicate command tags for START_REPLICATION inadvertently
in commit 07082b08cc5d, but the replication protocol requires them. The
fact that the replication protocol was broken was not noticed because
all our test cases use an optimized code path that exits early, failing
to verify that the behavior is correct for non-optimized cases. Put
them back.
Also document this protocol quirk.
Add a test case that shows the failure. It might still succeed even
without the patch when run on a fast enough server, but it suffices to
show the bug in enough cases that it would be noticed in buildfarm.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Henry Hinze <henry.hinze@gmail.com>
Reviewed-by: Petr Jelínek <petr.jelinek@2ndquadrant.com>
Discussion: https://postgr.es/m/16643-eaadeb2a1a58d28c@postgresql.org
M doc/src/sgml/protocol.sgml
M src/backend/replication/logical/worker.c
M src/backend/replication/walsender.c
M src/test/subscription/t/100_bugs.pl
Make WL_POSTMASTER_DEATH level-triggered on kqueue builds.
commit : e0950135ae5d50140feecc7bc87c018019c6e406
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 15 Oct 2020 11:31:20 +1300
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 15 Oct 2020 11:31:20 +1300
If WaitEventSetWait() reports that the postmaster has gone away, later
calls to WaitEventSetWait() should continue to report that. Otherwise
further waits that occur in the proc_exit() path after we already
noticed the postmaster's demise could block forever.
Back-patch to 13, where the kqueue support landed.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3624029.1602701929%40sss.pgh.pa.us
M src/backend/storage/ipc/latch.c
Paper over regression failures in infinite_recurse() on PPC64 Linux.
commit : 855b6f287100f3eab24df0a83998db251ac4fd09
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Oct 2020 17:44:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 13 Oct 2020 17:44:56 -0400
Our infinite_recurse() test to verify sane stack-overrun behavior
is affected by a bug of the Linux kernel on PPC64: it will get SIGSEGV
if it receives a signal when the stack depth is (a) over 1MB and
(b) within a few kB of filling the current physical stack allocation.
See https://bugzilla.kernel.org/show_bug.cgi?id=205183.
Since this test is a bit time-consuming and we run it in parallel with
test scripts that do a lot of DDL, it can be expected to get an sinval
catchup interrupt at some point, leading to failure if the timing is
wrong. This has caused more than 100 buildfarm failures over the
past year or so.
While a fix exists for the kernel bug, it might be years before that
propagates into all production kernels, particularly in some of the
older distros we have in the buildfarm. For now, let's just back off
and not run this test on Linux PPC64; that loses nothing in test
coverage so far as our own code is concerned.
To do that, split this test into a new script infinite_recurse.sql
and skip the test when the platform name is powerpc64...-linux-gnu.
Back-patch to v12. Branches before that have not been seen to get
this failure. No doubt that's because the "errors" test was not
run in parallel with other tests before commit 798070ec0, greatly
reducing the odds of an sinval catchup being necessary.
I also back-patched 3c8553547 into v12, just so the new regression
script would look the same in all branches having it.
Discussion: https://postgr.es/m/3479046.1602607848@sss.pgh.pa.us
Discussion: https://postgr.es/m/20190723162703.GM22387%40telsasoft.com
M src/test/regress/expected/errors.out
A src/test/regress/expected/infinite_recurse.out
A src/test/regress/expected/infinite_recurse_1.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
M src/test/regress/sql/errors.sql
A src/test/regress/sql/infinite_recurse.sql
Fix GiST buffering build to work when there are included columns.
commit : 962ab473ec3d4c1090ba75fa677167126956c1ee
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Oct 2020 18:01:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Oct 2020 18:01:34 -0400
gistRelocateBuildBuffersOnSplit did not get the memo about which
attribute count to use. This could lead to a crash if there were
included columns and buffering build was chosen. (Because there
are random page-split decisions elsewhere in GiST index build,
the crashes are not entirely deterministic.)
Back-patch to v12 where GiST gained support for included columns.
Pavel Borisov
Discussion: https://postgr.es/m/CALT9ZEECCV5m7wvxg46PC-7x-EybUmnpupBGhSFMoAAay+r6HQ@mail.gmail.com
M src/backend/access/gist/gistbuildbuffers.c
Fix memory leak when guc.c decides a setting can't be applied now.
commit : 9343bfefa4514e5623cfc2610c44e3d93d776e64
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Oct 2020 13:31:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 12 Oct 2020 13:31:24 -0400
The prohibitValueChange code paths in set_config_option(), which
are executed whenever we re-read a PGC_POSTMASTER variable from
postgresql.conf, neglected to free anything before exiting. Thus
we'd leak the proposed new value of a PGC_STRING variable, as noted
by BoChen in bug #16666. For all variable types, if the check hook
creates an "extra" chunk, we'd also leak that.
These are malloc not palloc chunks, so there is no mechanism for
recovering the leaks before process exit. Fortunately, the values
are typically not very large, meaning you'd have to go through an
awful lot of SIGHUP configuration-reload cycles to make the leakage
amount to anything. Still, for a long-lived postmaster process it
could potentially be a problem.
Oversight in commit 2594cf0e8. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16666-2c41a4eec61b03e1@postgresql.org
M src/backend/utils/misc/guc.c
Choose ppc compare_exchange constant path for more operand values.
commit : 5efa788e1d070dd14cb94a8e087184dda36dc3ea
author : Noah Misch <noah@leadboat.com>
date : Sun, 11 Oct 2020 21:31:37 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 11 Oct 2020 21:31:37 -0700
The implementation uses smaller code when the "expected" operand is a
small constant, but the implementation needlessly defined the set of
acceptable constants more narrowly than the ABI does. Core PostgreSQL
and PGXN don't use the constant path at all, so this is future-proofing.
Back-patch to v13, where commit 30ee5d17c20dbb282a9952b3048d6ad52d56c371
introduced this code.
Reviewed by Tom Lane. Reported by Christoph Berg.
Discussion: https://postgr.es/m/20201009092825.GD889580@msg.df7cb.de
M src/include/port/atomics/arch-ppc.h
M src/test/regress/regress.c
For ppc gcc, implement 64-bit compare_exchange and fetch_add with asm.
commit : d41cb63ff4d114d856837fbf61ba2872c5076ac2
author : Noah Misch <noah@leadboat.com>
date : Sun, 11 Oct 2020 21:31:37 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 11 Oct 2020 21:31:37 -0700
While xlc defines __64BIT__, gcc does not. Due to this oversight in
commit 30ee5d17c20dbb282a9952b3048d6ad52d56c371, gcc builds continued
implementing 64-bit atomics by way of intrinsics. Back-patch to v13,
where that commit first appeared.
Reviewed by Tom Lane.
Discussion: https://postgr.es/m/20201011051043.GA1724101@rfd.leadboat.com
M src/include/port/atomics/arch-ppc.h
Fix optimization hazard in gram.y's makeOrderedSetArgs(), redux.
commit : dc14aa038e20d0287a569c36498da9469fe9d4e3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 18:41:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 18:41:39 -0400
It appears that commit cf63c641c, which intended to prevent
misoptimization of the result-building step in makeOrderedSetArgs,
didn't go far enough: buildfarm member hornet's version of xlc
is now optimizing back to the old, broken behavior in which
list_length(directargs) is fetched only after list_concat() has
changed that value. I'm not entirely convinced whether that's
an undeniable compiler bug or whether it can be justified by a
sufficiently aggressive interpretation of C sequence points.
So let's just change the code to make it harder to misinterpret.
Back-patch to all supported versions, just in case.
Discussion: https://postgr.es/m/1830491.1601944935@sss.pgh.pa.us
M src/backend/parser/gram.y
Prevent internal overflows in date-vs-timestamp and related comparisons.
commit : 5ed20a689e3d5d47a70b971f388e9da2a996dea9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 17:10:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 17:10:26 -0400
The date-vs-timestamp, date-vs-timestamptz, and timestamp-vs-timestamptz
comparators all worked by promoting the first type to the second and
then doing a simple same-type comparison. This works fine, except
when the conversion result is out of range, in which case we throw an
entirely avoidable error. The sources of such failures are
(a) type date can represent dates much farther in the future than
the timestamp types can;
(b) timezone rotation might cause a just-in-range timestamp value to
become a just-out-of-range timestamptz value.
Up to now we just ignored these corner-case issues, but now we have
an actual user complaint (bug #16657 from Huss EL-Sheikh), so let's
do something about it.
It turns out that commit 52ad1e659 already built all the necessary
infrastructure to support error-free comparisons, but neglected to
actually use it in the main-line code paths. Fix that, do a little
bit of code style review, and remove the now-duplicate logic in
jsonpath_exec.c.
Back-patch to v13 where 52ad1e659 came in. We could take this back
further by back-patching said infrastructure, but given the small
number of complaints so far, I don't feel a great need to.
Discussion: https://postgr.es/m/16657-cde2f876d8cc7971@postgresql.org
M src/backend/utils/adt/date.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/date.h
M src/include/utils/timestamp.h
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
Rethink recent fix for pg_dump's handling of extension config tables.
commit : 2ea624b4b51caa0e82a4084d2499f5fc72cbe418
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 12:50:54 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 7 Oct 2020 12:50:54 -0400
Commit 3eb3d3e78 was a few bricks shy of a load: while it correctly
set the table's "interesting" flag when deciding to dump the data of
an extension config table, it was not correct to clear that flag
if we concluded we shouldn't dump the data. This led to the crash
reported in bug #16655, because in fact we'll traverse dumpTableSchema
anyway for all extension tables (to see if they have user-added
seclabels or RLS policies).
The right thing to do is to force "interesting" true in makeTableDataInfo,
and otherwise leave the flag alone. (Doing it there is more future-proof
in case additional calls are added, and it also avoids setting the flag
unnecessarily if that function decides the table is non-dumpable.)
This investigation also showed that while only the --inserts code path
had an obvious failure in the case considered by 3eb3d3e78, the COPY
code path also has a problem with not having loaded table subsidiary
data. That causes fmtCopyColumnList to silently return an empty string
instead of the correct column list. That accidentally mostly works,
which perhaps is why we didn't notice this before. It would only fail
if the restore column order is different from the dump column order,
which only happens in weird inheritance cases, so it's not surprising
nobody had hit the case with an extension config table. Nonetheless,
it's a bug, and it goes a long way back, not just to v12 where the
--inserts code path started to have a problem with this.
In hopes of catching such cases a bit sooner in future, add some
Asserts that "interesting" has been set in both dumpTableData and
dumpTableSchema. Adjust the test case added by 3eb3d3e78 so that it
checks the COPY rather than INSERT form of that bug, allowing it to
detect the longer-standing symptom.
Per bug #16655 from Cameron Daniel. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/16655-5c92d6b3a9438137@postgresql.org
Discussion: https://postgr.es/m/18048b44-3414-b983-8c7c-9165b177900d@2ndQuadrant.com
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
pg_upgrade: remove pre-8.4 code and >= 8.4 check
commit : be304cf9f5665e1c19d02f1144d4f0affa0034c7
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 14:31:22 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 14:31:22 -0400
We only support upgrading from >= 8.4 so no need for this code or tests.
Reported-by: Magnus Hagander
Discussion: https://postgr.es/m/CABUevEx-D0PNVe00tkeQRGennZQwDtBJn=493MJt-x6sppbUxA@mail.gmail.com
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/relfilenode.c
pg_upgrade; change major version comparisons to use <=, not <
commit : 2cb4b8e0ae16e7091a7bd5cf94f21d2719b108e0
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 12:12:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 6 Oct 2020 12:12:09 -0400
This makes checking for older major versions more consistent.
Backpatch-through: 9.5
M src/bin/pg_upgrade/check.c
M src/bin/pg_upgrade/controldata.c
M src/bin/pg_upgrade/exec.c
M src/bin/pg_upgrade/function.c
M src/bin/pg_upgrade/pg_upgrade.c
M src/bin/pg_upgrade/server.c
Build EC members for child join rels in the right memory context.
commit : b7f166efade004ba293f52b672961ae064d202cd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Oct 2020 11:43:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 6 Oct 2020 11:43:53 -0400
This patch prevents crashes or wrong plans when partition-wise joins
are considered during GEQO planning, as a consequence of the
EquivalenceClass data structures becoming corrupt after a GEQO
context reset.
A remaining problem is that successive GEQO cycles will make multiple
copies of the required EC members, since add_child_join_rel_equivalences
has no idea that such members might exist already. For now we'll just
live with that. The lack of field complaints of crashes suggests that
this is a mighty little-used situation.
Back-patch to v12 where this code was introduced.
Discussion: https://postgr.es/m/1683100.1601860653@sss.pgh.pa.us
M src/backend/optimizer/path/equivclass.c
Further improvements on documentation for pg_dump -t
commit : 96423711918f44600c9ef91f4342984624f053bb
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 6 Oct 2020 15:50:03 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 6 Oct 2020 15:50:03 +0200
Ian submitted an updated patch just as I was pushing the previous one,
so use this newer wording instead.
Author: Ian Barwick
M doc/src/sgml/ref/pg_dump.sgml
Clarify documentation around pg_dump -t option
commit : 0639f9b8c251d152695a968c3978edca844c3cad
author : Magnus Hagander <magnus@hagander.net>
date : Tue, 6 Oct 2020 15:46:36 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Tue, 6 Oct 2020 15:46:36 +0200
The behavior is different for different types of objects, so make that
more clear.
Author: Ian Barwick
M doc/src/sgml/ref/pg_dump.sgml
doc: show functions returning record types and use of ROWS FROM
commit : e9703ce6e7ac263e0e5a6fca9266e09284195dc7
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 Oct 2020 16:27:33 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 Oct 2020 16:27:33 -0400
Previously it was unclear exactly how ROWS FROM behaved and how to cast
the data types of columns returned by FROM functions. Also document
that only non-OUT record functions can have their columns cast to data
types.
Reported-by: guyren@gmail.com
Discussion: https://postgr.es/m/158638264419.662.2482095087061084020@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/queries.sgml
docs: clarify the interaction of clientcert and cert auth.
commit : ef40ab77d5143385d15dcfd08c5a7d66719ef7a3
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 5 Oct 2020 16:07:15 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 5 Oct 2020 16:07:15 -0400
This is the first paragraph change of master-only commit 253f1025da.
Backpatch-through: PG 12-13 only
M doc/src/sgml/client-auth.sgml
Fix two latent(?) bugs in equivclass.c.
commit : d1c23d726d50e10179235b6cee6b34543a879b19
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Oct 2020 13:15:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Oct 2020 13:15:39 -0400
get_eclass_for_sort_expr() computes expr_relids and nullable_relids
early on, even though they won't be needed unless we make a new
EquivalenceClass, which we often don't. Aside from the probably-minor
inefficiency, there's a memory management problem: these bitmapsets will
be built in the caller's context, leading to dangling pointers if that
is shorter-lived than root->planner_cxt. This would be a live bug if
get_eclass_for_sort_expr() could be called with create_it = true during
GEQO join planning. So far as I can find, the core code never does
that, but it's hard to be sure that no extensions do, especially since
the comments make it clear that that's supposed to be a supported case.
Fix by not computing these values until we've switched into planner_cxt
to build the new EquivalenceClass.
generate_join_implied_equalities() uses inner_rel->relids to look up
relevant eclasses, but it ought to be using nominal_inner_relids.
This is presently harmless because a child RelOptInfo will always have
exactly the same eclass_indexes as its topmost parent; but that might
not be true forever, and anyway it makes the code confusing.
The first of these is old (introduced by me in f3b3b8d5b), so back-patch
to all supported branches. The second only dates to v13, but we might
as well back-patch it to keep the code looking similar across branches.
Discussion: https://postgr.es/m/1508010.1601832581@sss.pgh.pa.us
M src/backend/optimizer/path/equivclass.c
Doc: fix parameter names in the docs of a couple of functions.
commit : 019eb962fb869b55ac8db173c4424a5de6cfee61
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Oct 2020 11:42:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 5 Oct 2020 11:42:33 -0400
The descriptions of make_interval() and pg_options_to_table()
were randomly different from the reality embedded in pg_proc.
(These are not all the discrepancies I found in a quick search,
but the others perhaps require more discussion, since there's
at least a case to be made for changing pg_proc not the docs.)
make_interval issue noted by Thomas Kellerer.
Discussion: https://postgr.es/m/7b154ef0-9f22-90b9-7734-4bf23686695b@gmx.net
M doc/src/sgml/func.sgml
Improve stability of identity.sql regression test.
commit : e01e339560ea7d8716924f3b014e902ef646729c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Oct 2020 20:45:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 4 Oct 2020 20:45:06 -0400
I noticed while trying to run the regression tests under a low
geqo_threshold that one query on information_schema.columns had
unstable (as in, variable from one run to the next) output order.
This is pretty unsurprising given the complexity of the underlying
plan. Interestingly, of this test's three nigh-identical queries on
information_schema.columns, the other two already had ORDER BY clauses
guaranteeing stable output. Let's make this one look the same.
Back-patch to v10 where this test was added. We've not heard field
reports of the test failing, but this experience shows that it can
happen when testing under even slightly unusual conditions.
M src/test/regress/expected/identity.out
M src/test/regress/sql/identity.sql
doc: libpq connection options can override command-line flags
commit : d2c9ef1c80b6adfbeee49507445c1b2eb3d08783
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 22:19:31 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 22:19:31 -0400
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/16486-b9c93d71c02c4907@postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/ref/clusterdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_isready.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M doc/src/sgml/ref/pg_recvlogical.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
doc: clarify the use of ssh port forwarding
commit : 566c6d4fd8bb76a761ae144a33514428ab048880
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 21:39:33 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 2 Oct 2020 21:39:33 -0400
Reported-by: karimelghazouly@gmail.com
Discussion: https://postgr.es/m/159854511172.24991.4373145230066586863@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/runtime.sgml
Put back explicit setting of replication values within TAP tests.
commit : 6731f1ef19fdf912cfbf2686a4d344a022b7704b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Oct 2020 10:59:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 1 Oct 2020 10:59:20 -0400
Commit 151c0c5f7 neglected the possibility that a TEMP_CONFIG file
would explicitly set max_wal_senders=0; as indeed buildfarm member
thorntail does, so that it can test wal_level=minimal in other test
suites. Hence, rather than assuming that max_wal_senders=10 will
prevail if we say nothing, set it explicitly.
Set max_replication_slots=10 explicitly too, just to be safe.
Back-patch to v10, like the previous patch.
Discussion: https://postgr.es/m/723911.1601417626@sss.pgh.pa.us
M src/test/perl/PostgresNode.pm
Fix incorrect assertion on number of array dimensions.
commit : 3c85489ec9cd08ce43976afa75f613f939635cb9
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 1 Oct 2020 11:48:48 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Thu, 1 Oct 2020 11:48:48 +0300
This has been wrong ever since the support for multi-dimensional
arrays as PL/python function arguments and return values was
introduced in commit 94aceed317.
Backpatch-through: 10
Discussion: https://www.postgresql.org/message-id/61647b8e-961c-0362-d5d3-c8a18f4a7ec6%40iki.fi
M src/pl/plpython/plpy_typeio.c
Reword partitioning error message
commit : 49433744ff65d8d799572cd616aecaf3074bcda5
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 30 Sep 2020 18:25:22 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 30 Sep 2020 18:25:22 -0300
The error message about columns in the primary key not including all of
the partition key was unclear; reword it.
Backpatch all the way to pg11, where it appeared.
Reported-by: Nagaraj Raj <nagaraj.sf@yahoo.com>
Discussion: https://postgr.es/m/64062533.78364.1601415362244@mail.yahoo.com
M src/backend/commands/indexcmds.c
M src/test/regress/expected/indexing.out
Fix handling of BC years in to_date/to_timestamp.
commit : 99fd38c02299acdc2282ac2dea8057a7a8f5f807
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Sep 2020 15:40:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 30 Sep 2020 15:40:23 -0400
Previously, a conversion such as
to_date('-44-02-01','YYYY-MM-DD')
would result in '0045-02-01 BC', as the code attempted to interpret
the negative year as BC, but failed to apply the correction needed
for our internal handling of BC years. Fix the off-by-one problem.
Also, arrange for the combination of a negative year and an
explicit "BC" marker to cancel out and produce AD. This is how
the negative-century case works, so it seems sane to do likewise.
Continue to read "year 0000" as 1 BC. Oracle would throw an error,
but we've accepted that case for a long time so I'm hesitant to
change it in a back-patch.
Per bug #16419 from Saeed Hubaishan. Back-patch to all supported
branches.
Dar Alathar-Yemen and Tom Lane
Discussion: https://postgr.es/m/16419-d8d9db0a7553f01b@postgresql.org
M doc/src/sgml/func.sgml
M src/backend/utils/adt/formatting.c
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
Remove obsolete replication settings within TAP tests.
commit : db8e60b82d6af88a4c8e1f9572abd5f5d84906b2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Sep 2020 20:02:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Sep 2020 20:02:58 -0400
PostgresNode.pm set "max_wal_senders = 5" for replication testing,
but this seems to be slightly too low for our current test suite.
Slower buildfarm members frequently report "number of requested standby
connections exceeds max_wal_senders" failures, due to old walsenders
not exiting instantaneously. Usually, the test does not fail overall
because of automatic walreceiver restart, but sometimes the failure
becomes visible; and in any case such retries slow down the test.
That value came in with commit 89ac7004d, but was soon obsoleted by
f6d6d2920, which raised the built-in default from zero to 10; so that
PostgresNode.pm is actually setting it to less than the conservative
built-in default. That seems pretty pointless, so let's remove the
special setting and let the default prevail, in hopes of making
the TAP tests more robust.
Likewise, the setting "max_replication_slots = 5" is obsolete and
can be removed.
While here, reverse-engineer a comment about why we're choosing
less-than-default values for some other settings.
(Note: before v12, max_wal_senders counted against max_connections
so that the latter setting also needs some fiddling with.)
Back-patch to v10 where the subscription tests were added.
It's likely that the older branches aren't pushing the boundaries
of max_wal_senders, but I'm disinclined to spend time trying to
figure out exactly when it started to be a problem.
Discussion: https://postgr.es/m/723911.1601417626@sss.pgh.pa.us
M src/test/perl/PostgresNode.pm
Doc: Improve clarity on partitioned table limitations
commit : 5610ffaf00a53877ec973881b9b0b7a1acad689a
author : David Rowley <drowley@postgresql.org>
date : Wed, 30 Sep 2020 13:03:01 +1300
committer: David Rowley <drowley@postgresql.org>
date : Wed, 30 Sep 2020 13:03:01 +1300
Explicitly mention that primary key constraints are also included in the
limitation that the constraint columns must be a superset of the partition key
columns.
Wording suggestion from Tom Lane.
Discussion: https://postgr.es/m/64062533.78364.1601415362244@mail.yahoo.com
Backpatch-through: 11, where unique constraints on partitioned tables were added
M doc/src/sgml/ddl.sgml
Fix memory leak in plpgsql's CALL processing.
commit : f0e4ec74e452f55922b52f50da4ba4834771a268
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Sep 2020 11:18:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 29 Sep 2020 11:18:30 -0400
When executing a CALL or DO in a non-atomic context (i.e., not inside
a function or query), plpgsql creates a new plan each time through,
as a rather hacky solution to some resource management issues. But
it failed to free this plan until exit of the current procedure or DO
block, resulting in serious memory bloat in procedures that called
other procedures many times. Fix by remembering to free the plan,
and by being more honest about restoring the previous state (otherwise,
recursive procedure calls have a problem).
There was also a smaller leak associated with recalculation of the
"target" list of output variables. Fix that by using the statement-
lifespan context to hold non-permanent values.
Back-patch to v11 where procedures were introduced.
Pavel Stehule and Tom Lane
Discussion: https://postgr.es/m/CAFj8pRDiiU1dqym+_P4_GuTWm76knJu7z9opWayBJTC0nQGUUA@mail.gmail.com
M src/pl/plpgsql/src/pl_exec.c
Support for ISO 8601 in the jsonpath .datetime() method
commit : 651bdbc811652638e1205440c3181a18feb8f967
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Sep 2020 11:41:46 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Sep 2020 11:41:46 +0300
The SQL standard doesn't require jsonpath .datetime() method to support the
ISO 8601 format. But our to_json[b]() functions convert timestamps to text in
the ISO 8601 format in the sake of compatibility with javascript. So, we add
support of the ISO 8601 to the jsonpath .datetime() in the sake compatibility
with to_json[b]().
The standard mode of datetime parsing currently supports just template patterns
and separators in the format string. In order to implement ISO 8601, we have to
add support of the format string double quotes to the standard parsing mode.
Discussion: https://postgr.es/m/94321be0-cc96-1a81-b6df-796f437f7c66%40postgrespro.ru
Author: Nikita Glukhov, revised by me
Backpatch-through: 13
M src/backend/utils/adt/formatting.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Remove excess space from jsonpath .datetime() default format string
commit : abcc0ab163003d2ab7c82a1e810ba257ebbec15f
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Sep 2020 11:00:22 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Tue, 29 Sep 2020 11:00:22 +0300
bffe1bd684 has introduced jsonpath .datetime() method, but default formats
for time and timestamp contain excess space between time and timezone. This
commit removes this excess space making behavior of .datetime() method
standard-compliant.
Discussion: https://postgr.es/m/94321be0-cc96-1a81-b6df-796f437f7c66%40postgrespro.ru
Author: Nikita Glukhov
Backpatch-through: 13
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Archive timeline history files in standby if archive_mode is set to "always".
commit : 059caf36c3074afd998b6e5f36ea9da460dcaee8
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 29 Sep 2020 16:21:46 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 29 Sep 2020 16:21:46 +0900
Previously the standby server didn't archive timeline history files
streamed from the primary even when archive_mode is set to "always",
while it archives the streamed WAL files. This could cause the PITR to
fail because there was no required timeline history file in the archive.
The cause of this issue was that walreceiver didn't mark those files as
ready for archiving.
This commit makes walreceiver mark those streamed timeline history
files as ready for archiving if archive_mode=always. Then the archiver
process archives the marked timeline history files.
Back-patch to all supported versions.
Reported-by: Grigory Smolkin
Author: Grigory Smolkin, Fujii Masao
Reviewed-by: David Zhang, Anastasia Lubennikova
Discussion: https://postgr.es/m/54b059d4-2b48-13a4-6f43-95a087c92367@postgrespro.ru
M doc/src/sgml/high-availability.sgml
M src/backend/replication/walreceiver.c
Fix progress reporting of REINDEX CONCURRENTLY
commit : 1aedaba78aa8617b24b7a703abd1359f9d78f62a
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Sep 2020 14:16:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 29 Sep 2020 14:16:12 +0900
This addresses a couple of issues with the so-said subject:
- Report the correct parent relation with the index actually being
rebuilt or validated. Previously, the command status remained set to
the last index created for the progress of the index build and
validation, which would be incorrect when working on a table that has
more than one index.
- Use the correct phase when waiting before the drop of the old
indexes. Previously, this was reported with the same status as when
waiting before the old indexes are marked as dead.
Author: Matthias van de Meent, Michael Paquier
Discussion: https://postgr.es/m/CAEze2WhqFgcwe1_tv=sFYhLWV2AdpfukumotJ6JNcAOQs3jufg@mail.gmail.com
Backpatch-through: 12
M src/backend/commands/indexcmds.c
Add for_each_from, to simplify loops starting from non-first list cells.
commit : 67b2ceea01576933a1dc881ef6a65403e03483ee
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Sep 2020 20:32:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Sep 2020 20:32:53 -0400
We have a dozen or so places that need to iterate over all but the
first cell of a List. Prior to v13 this was typically written as
for_each_cell(lc, lnext(list_head(list)))
Commit 1cff1b95a changed these to
for_each_cell(lc, list, list_second_cell(list))
This patch introduces a new macro for_each_from() which expresses
the start point as a list index, allowing these to be written as
for_each_from(lc, list, 1)
This is marginally more efficient, since ForEachState.i can be
initialized directly instead of backing into it from a ListCell
address. It also seems clearer and less typo-prone.
Some of the remaining uses of for_each_cell() look like they could
profitably be changed to for_each_from(), but here I confined myself
to changing uses of list_second_cell().
Also, fix for_each_cell_setup() and for_both_cell_setup() to
const-ify their arguments; that's a simple oversight in 1cff1b95a.
Back-patch into v13, on the grounds that (1) the const-ification
is a minor bug fix, and (2) it's better for back-patching purposes
if we only have two ways to write these loops rather than three.
In HEAD, also remove list_third_cell() and list_fourth_cell(),
which were also introduced in 1cff1b95a, and are unused as of
cc99baa43. It seems unlikely that any third-party code would
have started to use them already; anyone who has can be directed
to list_nth_cell instead.
Discussion: https://postgr.es/m/CAApHDvpo1zj9KhEpU2cCRZfSM3Q6XGdhzuAS2v79PH7WJBkYVA@mail.gmail.com
M src/backend/commands/tablecmds.c
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/plan/createplan.c
M src/backend/optimizer/plan/planner.c
M src/backend/parser/parse_agg.c
M src/backend/utils/adt/jsonpath_gram.y
M src/backend/utils/adt/ruleutils.c
M src/backend/utils/adt/selfuncs.c
M src/include/nodes/pg_list.h
Assign collations in partition bound expressions.
commit : 61a78c71a656593bf4121e624348a990ba5b91da
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Sep 2020 14:12:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 28 Sep 2020 14:12:38 -0400
Failure to do this can result in errors during evaluation of
the bound expression, as illustrated by the new regression test.
Back-patch to v12 where the ability for partition bounds to be
expressions was added.
Discussion: https://postgr.es/m/CAJV4CdrZ5mKuaEsRSbLf2URQ3h6iMtKD=hik8MaF5WwdmC9uZw@mail.gmail.com
M src/backend/parser/parse_utilcmd.c
M src/test/regress/expected/create_table.out
M src/test/regress/sql/create_table.sql
Revise RelationBuildRowSecurity() to avoid memory leaks.
commit : f7873900f353ff210ef2ef2aa587e39196b8bf5a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Sep 2020 16:04:06 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 26 Sep 2020 16:04:06 -0400
This function leaked some memory while loading qual clauses for
an RLS policy. While ordinarily negligible, that could build up
in some repeated-reload cases, as reported by Konstantin Knizhnik.
We can improve matters by borrowing the coding long used in
RelationBuildRuleLock: build stringToNode's result directly in
the target context, and remember to explicitly pfree the
input string.
This patch by no means completely guarantees zero leaks within
this function, since we have no real guarantee that the catalog-
reading subroutines it calls don't leak anything. However,
practical tests suggest that this is enough to resolve the issue.
In any case, any remaining leaks are similar to those risked by
RelationBuildRuleLock and other relcache-loading subroutines.
If we need to fix them, we should adopt a more global approach
such as that used by the RECOVER_RELATION_BUILD_MEMORY hack.
While here, let's remove the need for an expensive PG_TRY block by
using MemoryContextSetParent to reparent an initially-short-lived
context for the RLS data.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/21356c12-8917-8249-b35f-1c447231922b@postgrespro.ru
M src/backend/commands/policy.c
Fix handling of -d "connection string" in pg_dump/pg_restore.
commit : cb8885ac49697eb2568c4764ae3565cea52be92b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Sep 2020 18:19:38 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 24 Sep 2020 18:19:38 -0400
Parallel pg_dump failed if its -d parameter was a connection string
containing any essential information other than host, port, or username.
The same was true for pg_restore with --create.
The reason is that these scenarios failed to preserve the connection
string from the command line; the code felt free to replace that with
just the database name when reconnecting from a pg_dump parallel worker
or after creating the target database. By chance, parallel pg_restore
did not suffer this defect, as long as you didn't say --create.
In practice it seems that the error would be obvious only if the
connstring included essential, non-default SSL or GSS parameters.
This may explain why it took us so long to notice. (It also makes
it very difficult to craft a regression test case illustrating the
problem, since the test would fail in builds without those options.)
Fix by refactoring so that ConnectDatabase always receives all the
relevant options directly from the command line, rather than
reconstructed values. Inject a different database name, when necessary,
by relying on libpq's rules for handling multiple "dbname" parameters.
While here, let's get rid of the essentially duplicate _connectDB
function, as well as some obsolete nearby cruft.
Per bug #16604 from Zsolt Ero. Back-patch to all supported branches.
Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_backup_archiver.c
M src/bin/pg_dump/pg_backup_archiver.h
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_restore.c
Fix missing fsync of SLRU directories.
commit : 052014a2066827cb96dbc9ef464ce44293585601
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 24 Sep 2020 09:26:09 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 24 Sep 2020 09:26:09 +1200
Harmonize behavior by moving reponsibility for fsyncing directories down
into slru.c. In 10 and later, only the multixact directories were
missed (see commit 1b02be21), and in older branches all SLRUs were
missed.
Back-patch to all supported releases.
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKGLtsTUOScnNoSMZ-2ZLv%2BwGh01J6kAo_DM8mTRq1sKdSQ%40mail.gmail.com
M src/backend/access/transam/clog.c
M src/backend/access/transam/commit_ts.c
M src/backend/access/transam/slru.c
Avoid possible dangling-pointer access in tsearch_readline_callback.
commit : 569f6a89a9153ee05ab429522e835b00b11ad7f9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Sep 2020 11:36:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 23 Sep 2020 11:36:13 -0400
tsearch_readline() saves the string pointer it returns to the caller
for possible use in the associated error context callback. However,
the caller will usually pfree that string sometime before it next
calls tsearch_readline(), so that there is a window where an ereport
will try to print an already-freed string.
The built-in users of tsearch_readline() happen to all do that pfree
at the bottoms of their loops, so that the window is effectively
empty for them. However, this is not documented as a requirement,
and contrib/dict_xsyn doesn't do it like that, so it seems likely
that third-party dictionaries might have live bugs here.
The practical consequences of this seem pretty limited in any case,
since production builds wouldn't clobber the freed string immediately,
besides which you'd not expect syntax errors in dictionary files
being used in production. Still, it's clearly a bug waiting to bite
somebody.
Fix by pstrdup'ing the string to be saved for the error callback,
and then pfree'ing it next time through. It's been like this for
a long time, so back-patch to all supported branches.
Discussion: https://postgr.es/m/48A4FA71-524E-41B9-953A-FD04EF36E2E7@yesql.se
M src/backend/tsearch/ts_locale.c
Stamp 13.0.
commit : 29be9983a64c011eac0b9ee29895cce71e15ea77
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Sep 2020 16:47:36 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Sep 2020 16:47:36 -0400
M configure
M configure.in
Doc: improve v13 release note item about autovacuum and INSERTs.
commit : 4406364e2bf421459be7bd21503da093d910e0c3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Sep 2020 13:30:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Sep 2020 13:30:18 -0400
The previous text was confusing, per off-list discussion with
Bruce Momjian.
M doc/src/sgml/release-13.sgml
Copy editing: fix a bunch of misspellings and poor wording.
commit : e62c5ea22c12f63d8d5ca3b228a458dfc10ae314
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Sep 2020 12:43:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 21 Sep 2020 12:43:42 -0400
99% of this is docs, but also a couple of comments. No code changes.
Justin Pryzby
Discussion: https://postgr.es/m/20200919175804.GE30557@telsasoft.com
M doc/src/sgml/btree.sgml
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/ref/alter_statistics.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/reindex.sgml
M doc/src/sgml/ref/reindexdb.sgml
M doc/src/sgml/ref/vacuumdb.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/sources.sgml
M src/backend/access/gin/README
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/partition_join.sql
Translation updates
commit : d83268ae10cdeb2aa88e32286e94a8a8f59653a0
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Sep 2020 10:06:30 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Sep 2020 10:06:30 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: cdd5cffbddac2869f3eed0a6a37cba71ce2332cd
M src/backend/po/es.po
M src/backend/po/fr.po
M src/backend/po/ru.po
M src/backend/po/sv.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_dump/po/fr.po
M src/bin/pg_rewind/po/es.po
M src/bin/pg_rewind/po/fr.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_rewind/po/sv.po
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_verifybackup/po/es.po
M src/bin/pg_verifybackup/po/fr.po
M src/bin/pg_verifybackup/po/ru.po
M src/bin/pg_verifybackup/po/sv.po
M src/bin/pg_waldump/po/es.po
M src/bin/psql/po/es.po
M src/bin/psql/po/fr.po
M src/bin/psql/po/ru.po
M src/bin/scripts/po/es.po
M src/bin/scripts/po/fr.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
M src/interfaces/libpq/po/fr.po
M src/pl/plpgsql/src/po/es.po
M src/pl/plpgsql/src/po/fr.po
Update list of acknowledgments in release notes
commit : f6727f29d52072bd0e87fbc9ed7af0d880db0d5c
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Sep 2020 09:05:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 21 Sep 2020 09:05:13 +0200
M doc/src/sgml/release-13.sgml
Use factorial rather than numeric_fac in create_operator.sql.
commit : 9ab5ed4194f3863ff744e195027da747d4db4106
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Sep 2020 18:03:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 18 Sep 2020 18:03:44 -0400
These two SQL functions are aliases for the same C function, so this
change has no semantic effect. However, because we dropped the
numeric_fac alias in HEAD (commit 76f412ab3), operator definitions
based on that one don't port forward, causing problems for cross-version
upgrade tests based on the regression database.
Patch all active back branches to dodge the problem.
Discussion: https://postgr.es/m/449144.1600439950@sss.pgh.pa.us
M src/test/regress/expected/create_operator.out
M src/test/regress/sql/create_operator.sql
Fix comments in heapam.c.
commit : f083afac9df707778288b9ab448be169a25e3ea6
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 18 Sep 2020 09:40:04 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 18 Sep 2020 09:40:04 +0530
After commits 85f6b49c2c and 3ba59ccc89, we can allow parallel inserts
which was earlier not possible as parallel group members won't conflict
for relation extension and page lock. In those commits, we forgot to
update comments at few places.
Author: Amit Kapila
Reviewed-by: Robert Haas and Dilip Kumar
Backpatch-through: 13
Discussion: https://postgr.es/m/CAFiTN-tMrQh5FFMPx5aWJ+1gi1H6JxktEhq5mDwCHgnEO5oBkA@mail.gmail.com
M src/backend/access/heap/heapam.c
Update parallel BTree scan state when the scan keys can't be satisfied.
commit : 0abd9cd2f3a1cad201ca28767aa0a720cc341179
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 17 Sep 2020 15:16:46 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 17 Sep 2020 15:16:46 +0530
For parallel btree scan to work for array of scan keys, it should reach
BTPARALLEL_DONE state once for every distinct combination of array keys.
This is required to ensure that the parallel workers don't try to seize
blocks at the same time for different scan keys. We missed to update this
state when we discovered that the scan keys can't be satisfied.
Author: James Hunter
Reviewed-by: Amit Kapila
Tested-by: Justin Pryzby
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/4248CABC-25E3-4809-B4D0-128E1BAABC3C@amazon.com
M src/backend/access/nbtree/nbtsearch.c
Fix bogus completion tag usage in walsender
commit : bfb12cd2b59da2ce51a9c86bf2c468202d8f96ee
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 16 Sep 2020 13:04:38 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 16 Sep 2020 13:04:38 -0300
Since commit fd5942c18f97 (2012, 9.3-era), walsender has been sending
completion tags for certain replication commands twice -- and they're
not even consistent. Apparently neither libpq nor JDBC have a problem
with it, but it's not kosher. Fix by remove the EndCommand() call in
the common code path for them all, and inserting specific calls to
EndReplicationCommand() specifically in those places where it's needed.
EndReplicationCommand() is a new simple function to send the completion
tag for replication commands. Do this instead of sending a generic
SELECT completion tag for them all, which was also pretty bogus (if
innocuous). While at it, change StartReplication() to use
EndReplicationCommand() instead of pg_puttextmessage().
In commit 2f9661311b83, I failed to realize that replication commands
are not close-enough kin of regular SQL commands, so the
DROP_REPLICATION_SLOT tag I added is undeserved and a type pun. Take it
out.
Backpatch to 13, where the latter commit appeared. The duplicate tag
has been sent since 9.3, but since nothing is broken, it doesn't seem
worth fixing.
Per complaints from Tom Lane.
Discussion: https://postgr.es/m/1347966.1600195735@sss.pgh.pa.us
M src/backend/replication/walsender.c
M src/backend/tcop/dest.c
M src/include/tcop/cmdtaglist.h
M src/include/tcop/dest.h
Fix amcheck child check pg_upgrade bug.
commit : c287f585865b81c96602db995dacf2c006c79d58
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 16 Sep 2020 10:42:28 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 16 Sep 2020 10:42:28 -0700
Commit d114cc53 overlooked the fact that pg_upgrade'd B-Tree indexes
have leaf page high keys whose offset numbers do not match the one from
the copy of the tuple one level up (the copy stored with a downlink for
leaf page's right sibling page). This led to false positive reports of
corruption from bt_index_parent_check() when it was called to verify a
pg_upgrade'd index.
To fix, skip comparing the offset number on pg_upgrade'd B-Tree indexes.
Author: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Andrew Bille <andrewbille@gmail.com>
Diagnosed-By: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Bug: #16619
Discussion: https://postgr.es/m/16619-aaba10f83fdc1c3c@postgresql.org
Backpatch: 13-, where child check was enhanced.
M contrib/amcheck/verify_nbtree.c
Avoid unnecessary recursion to child tables in ALTER TABLE SET NOT NULL.
commit : 17280b31c2f218f1b1f0c1fbacae7e781010a01b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Sep 2020 13:38:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Sep 2020 13:38:26 -0400
If a partitioned table's column is already marked NOT NULL, there is
no need to examine its partitions, because we can rely on previous
DDL to have enforced that the child columns are NOT NULL as well.
(Unfortunately, the same cannot be said for traditional inheritance,
so for now we have to restrict the optimization to partitioned tables.)
Hence, we may skip recursing to child tables in this situation.
The reason this case is worth worrying about is that when pg_dump dumps
a partitioned table having a primary key, it will include the requisite
NOT NULL markings in the CREATE TABLE commands, and then add the
primary key as a separate step. The primary key addition generates a
SET NOT NULL as a subcommand, just to be sure. So the situation where
a SET NOT NULL is redundant does arise in the real world.
Skipping the recursion does more than just save a few cycles: it means
that a command such as "ALTER TABLE ONLY partition_parent ADD PRIMARY
KEY" will take locks only on the partition parent table, not on the
partitions. It turns out that parallel pg_restore is effectively
assuming that that's true, and has little choice but to do so because
the dependencies listed for such a TOC entry don't include the
partitions. pg_restore could thus issue this ALTER while data restores
on the partitions are still in progress. Taking unnecessary locks on
the partitions not only hurts concurrency, but can lead to actual
deadlock failures, as reported by Domagoj Smoljanovic.
(A contributing factor in the deadlock is that TRUNCATE on a child
partition wants a non-exclusive lock on the parent. This seems
likewise unnecessary, but the fix for it is more invasive so we
won't consider back-patching it. Fortunately, getting rid of one
of these two poor behaviors is enough to remove the deadlock.)
Although support for partitioned primary keys came in with v11,
this patch is dependent on the SET NOT NULL refactoring done by
commit f4a3fdfbd, so we can only patch back to v12.
Patch by me; thanks to Alvaro Herrera and Amit Langote for review.
Discussion: https://postgr.es/m/VI1PR03MB31670CA1BD9625C3A8C5DD05EB230@VI1PR03MB3167.eurprd03.prod.outlook.com
M src/backend/commands/tablecmds.c
Fix bogus cache-invalidation logic in logical replication worker.
commit : 3e3f8f20206cbbb8d30be528d2a640d14a95c25c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Sep 2020 12:07:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 16 Sep 2020 12:07:31 -0400
The code recorded cache invalidation events by zeroing the "localreloid"
field of affected cache entries. However, it's possible for an inval
event to occur even while we have the entry open and locked. So an
ill-timed inval could result in "cache lookup failed for relation 0"
errors, if the worker's code tried to use the cleared field. We can
fix that by creating a separate bool field to record whether the entry
needs to be revalidated. (In the back branches, cram the bool into
what had been padding space, to avoid an ABI break in the somewhat
unlikely event that any extension is looking at this struct.)
Also, rearrange the logic in logicalrep_rel_open so that it
does the right thing in cases where table_open would fail.
We should retry the lookup by name in that case, but we didn't.
The real-world impact of this is probably small. In the first place,
the error conditions are very low probability, and in the second place,
the worker would just exit and get restarted. We only noticed because
in a CLOBBER_CACHE_ALWAYS build, the failure can occur repeatedly,
preventing the worker from making progress. Nonetheless, it's clearly
a bug, and it impedes a useful type of testing; so back-patch to v10
where this code was introduced.
Discussion: https://postgr.es/m/1032727.1600096803@sss.pgh.pa.us
M src/backend/replication/logical/relation.c
M src/include/replication/logicalrelation.h
Change LogicalTapeSetBlocks() to use nBlocksWritten.
commit : 6e146a663536f86c8421ac6ed08c4eb9a69979fd
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 15 Sep 2020 21:34:05 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 15 Sep 2020 21:34:05 -0700
Previously, it was based on nBlocksAllocated to account for tapes with
open write buffers that may not have made it to the BufFile yet.
That was unnecessary, because callers do not need to get the number of
blocks while a tape has an open write buffer; and it also conflicted
with the preallocation logic added for HashAgg.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/ce5af05900fdbd0e9185747825a7423c48501964.camel@j-davis.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/backend/utils/sort/logtape.c
HashAgg: release write buffers sooner by rewinding tape.
commit : 42a46f5a76ecb435ac3a29fc3a0d03f1cfff17ab
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 15 Sep 2020 21:16:31 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 15 Sep 2020 21:16:31 -0700
This was an oversight. The purpose of 7fdd919ae7 was to avoid keeping
tape buffers around unnecessisarily, but HashAgg didn't rewind early
enough.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/1fb1151c2cddf8747d14e0532da283c3f97e2685.camel@j-davis.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
Fix use-after-free bug with event triggers in an extension script
commit : 873cb8fca9b14bde3e1d5577fcbb7b76d303076d
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 15 Sep 2020 21:03:14 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 15 Sep 2020 21:03:14 -0300
ALTER TABLE commands in an extension script are added to an event
trigger command list; but starting with commit b5810de3f4 they do so in
a memory context that's too short-lived, so when execution ends and time
comes to use the entries, they've already been freed.
(This would also be a problem with ALTER TABLE commands in a
multi-command query string, but these serendipitously end in
PortalContext -- which probably explains why it took so long for this to
be reported.)
Fix by using the memory context specifically set for that, instead.
Backpatch to 13, where the aforementioned commit appeared.
Reported-by: Philippe Beaudoin
Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Discussion: https://postgr.es/m/20200902193715.6e0269d4@firost
M src/backend/commands/event_trigger.c
M src/test/modules/test_extensions/Makefile
M src/test/modules/test_extensions/expected/test_extensions.out
M src/test/modules/test_extensions/sql/test_extensions.sql
A src/test/modules/test_extensions/test_ext_evttrig–1.0–2.0.sql
A src/test/modules/test_extensions/test_ext_evttrig–1.0.sql
A src/test/modules/test_extensions/test_ext_evttrig.control
Doc: improve release notes' info about FROM UNPACKAGED feature removal.
commit : d42c6176446440b185fcb95c214b7e40d5758b60
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Sep 2020 14:29:43 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Sep 2020 14:29:43 -0400
Per gripe from Jonathan Katz.
Discussion: https://postgr.es/m/e0a4d177-d003-8ebb-5296-5a445472b66f@postgresql.org
M doc/src/sgml/release-13.sgml
Doc: fix misstatement in v13 release notes.
commit : 001d2c5f15bf8d554a7fe28af033d82c24de4e44
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Sep 2020 10:58:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 15 Sep 2020 10:58:37 -0400
Parallel vacuuming isn't restricted to b-tree indexes.
Noted by Peter Eisentraut.
Discussion: https://postgr.es/m/f1c43223-3987-a23f-2063-18fd0aa4f0d4@2ndquadrant.com
M doc/src/sgml/release-13.sgml
Stamp 13rc1.
commit : efea2b85fa2c3dec1c8039f1c97fcfe53ee5e82c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Sep 2020 16:08:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 14 Sep 2020 16:08:07 -0400
M configure
M configure.in
Translation updates
commit : bab6f77f24407e0924dac292af9e65016fce99bf
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Sep 2020 13:14:53 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Sep 2020 13:14:53 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 00c0d74fc1f1f2a831077fdf3655c6ae5eeceac3
M src/backend/nls.mk
M src/backend/po/de.po
M src/backend/po/es.po
D src/backend/po/id.po
M src/backend/po/ja.po
D src/backend/po/pl.po
D src/backend/po/pt_BR.po
M src/backend/po/ru.po
M src/backend/po/sv.po
A src/backend/po/uk.po
M src/bin/initdb/nls.mk
M src/bin/initdb/po/es.po
D src/bin/initdb/po/he.po
D src/bin/initdb/po/it.po
M src/bin/initdb/po/ja.po
D src/bin/initdb/po/pl.po
D src/bin/initdb/po/pt_BR.po
M src/bin/initdb/po/ru.po
D src/bin/initdb/po/vi.po
M src/bin/pg_archivecleanup/nls.mk
M src/bin/pg_archivecleanup/po/es.po
M src/bin/pg_archivecleanup/po/ja.po
D src/bin/pg_archivecleanup/po/pl.po
M src/bin/pg_archivecleanup/po/ru.po
D src/bin/pg_archivecleanup/po/vi.po
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_basebackup/po/es.po
D src/bin/pg_basebackup/po/he.po
D src/bin/pg_basebackup/po/it.po
M src/bin/pg_basebackup/po/ja.po
D src/bin/pg_basebackup/po/pl.po
D src/bin/pg_basebackup/po/pt_BR.po
M src/bin/pg_basebackup/po/ru.po
A src/bin/pg_basebackup/po/uk.po
D src/bin/pg_basebackup/po/vi.po
M src/bin/pg_checksums/po/es.po
M src/bin/pg_checksums/po/ja.po
M src/bin/pg_checksums/po/ru.po
M src/bin/pg_config/nls.mk
M src/bin/pg_config/po/es.po
M src/bin/pg_config/po/ja.po
D src/bin/pg_config/po/nb.po
D src/bin/pg_config/po/ro.po
M src/bin/pg_config/po/ru.po
D src/bin/pg_config/po/ta.po
D src/bin/pg_config/po/zh_TW.po
M src/bin/pg_controldata/nls.mk
M src/bin/pg_controldata/po/es.po
M src/bin/pg_controldata/po/ja.po
D src/bin/pg_controldata/po/pl.po
D src/bin/pg_controldata/po/pt_BR.po
M src/bin/pg_controldata/po/ru.po
D src/bin/pg_controldata/po/vi.po
M src/bin/pg_ctl/nls.mk
M src/bin/pg_ctl/po/es.po
D src/bin/pg_ctl/po/he.po
M src/bin/pg_ctl/po/ja.po
D src/bin/pg_ctl/po/pl.po
D src/bin/pg_ctl/po/pt_BR.po
M src/bin/pg_ctl/po/ru.po
M src/bin/pg_dump/nls.mk
M src/bin/pg_dump/po/es.po
D src/bin/pg_dump/po/he.po
D src/bin/pg_dump/po/it.po
M src/bin/pg_dump/po/ja.po
D src/bin/pg_dump/po/pl.po
D src/bin/pg_dump/po/pt_BR.po
M src/bin/pg_dump/po/ru.po
M src/bin/pg_dump/po/sv.po
A src/bin/pg_dump/po/uk.po
M src/bin/pg_resetwal/nls.mk
M src/bin/pg_resetwal/po/es.po
D src/bin/pg_resetwal/po/it.po
M src/bin/pg_resetwal/po/ja.po
D src/bin/pg_resetwal/po/pl.po
D src/bin/pg_resetwal/po/pt_BR.po
M src/bin/pg_resetwal/po/ru.po
M src/bin/pg_rewind/nls.mk
M src/bin/pg_rewind/po/de.po
M src/bin/pg_rewind/po/es.po
D src/bin/pg_rewind/po/it.po
M src/bin/pg_rewind/po/ja.po
D src/bin/pg_rewind/po/pl.po
D src/bin/pg_rewind/po/pt_BR.po
M src/bin/pg_rewind/po/ru.po
M src/bin/pg_test_fsync/po/es.po
M src/bin/pg_test_fsync/po/ru.po
M src/bin/pg_test_timing/po/es.po
M src/bin/pg_upgrade/nls.mk
M src/bin/pg_upgrade/po/es.po
M src/bin/pg_upgrade/po/ja.po
M src/bin/pg_upgrade/po/ru.po
A src/bin/pg_upgrade/po/uk.po
M src/bin/pg_verifybackup/nls.mk
A src/bin/pg_verifybackup/po/de.po
A src/bin/pg_verifybackup/po/es.po
A src/bin/pg_verifybackup/po/ru.po
A src/bin/pg_verifybackup/po/uk.po
M src/bin/pg_waldump/nls.mk
M src/bin/pg_waldump/po/es.po
M src/bin/pg_waldump/po/ja.po
M src/bin/pg_waldump/po/ru.po
D src/bin/pg_waldump/po/vi.po
M src/bin/psql/nls.mk
M src/bin/psql/po/de.po
M src/bin/psql/po/es.po
D src/bin/psql/po/he.po
M src/bin/psql/po/ja.po
D src/bin/psql/po/pl.po
D src/bin/psql/po/pt_BR.po
M src/bin/psql/po/ru.po
M src/bin/psql/po/sv.po
D src/bin/psql/po/zh_TW.po
M src/bin/scripts/nls.mk
M src/bin/scripts/po/de.po
M src/bin/scripts/po/es.po
D src/bin/scripts/po/he.po
D src/bin/scripts/po/it.po
M src/bin/scripts/po/ja.po
D src/bin/scripts/po/pl.po
D src/bin/scripts/po/pt_BR.po
M src/bin/scripts/po/ru.po
M src/bin/scripts/po/sv.po
A src/bin/scripts/po/uk.po
M src/interfaces/ecpg/ecpglib/po/es.po
M src/interfaces/ecpg/ecpglib/po/ru.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/ecpg/preproc/po/ru.po
M src/interfaces/libpq/nls.mk
M src/interfaces/libpq/po/es.po
D src/interfaces/libpq/po/he.po
M src/interfaces/libpq/po/ja.po
D src/interfaces/libpq/po/pl.po
D src/interfaces/libpq/po/pt_BR.po
M src/interfaces/libpq/po/ru.po
M src/interfaces/libpq/po/sv.po
D src/interfaces/libpq/po/zh_TW.po
M src/pl/plperl/nls.mk
M src/pl/plperl/po/es.po
M src/pl/plperl/po/ru.po
A src/pl/plperl/po/uk.po
D src/pl/plperl/po/zh_TW.po
M src/pl/plpgsql/src/nls.mk
M src/pl/plpgsql/src/po/es.po
D src/pl/plpgsql/src/po/ro.po
M src/pl/plpgsql/src/po/ru.po
D src/pl/plpgsql/src/po/zh_TW.po
M src/pl/plpython/po/es.po
M src/pl/plpython/po/ru.po
M src/pl/tcl/nls.mk
M src/pl/tcl/po/es.po
D src/pl/tcl/po/pt_BR.po
D src/pl/tcl/po/ro.po
M src/pl/tcl/po/ru.po
D src/pl/tcl/po/zh_TW.po
Fix interpolation in test name.
commit : 6fb1c5b528267918a88c4143985a08a3c997e528
author : Noah Misch <noah@leadboat.com>
date : Sun, 13 Sep 2020 23:29:51 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 13 Sep 2020 23:29:51 -0700
A pre-commit review had reported the problem, but the fix reached only
v10 and earlier. Back-patch to v11.
Discussion: https://postgr.es/m/20200423.140546.1055476118690602079.horikyota.ntt@gmail.com
M src/test/recovery/t/020_archive_status.pl
Message fixes and style improvements
commit : b1b53f15bbac106e241b14ae1bc13f2708fe74c8
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Sep 2020 06:42:07 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 14 Sep 2020 06:42:07 +0200
M src/backend/access/heap/vacuumlazy.c
M src/backend/access/transam/xlog.c
M src/backend/commands/opclasscmds.c
M src/backend/commands/tablecmds.c
M src/backend/libpq/hba.c
M src/backend/nodes/params.c
M src/backend/parser/gram.y
M src/backend/parser/parse_clause.c
M src/backend/replication/basebackup.c
M src/backend/storage/ipc/procarray.c
M src/backend/utils/adt/jsonpath_exec.c
M src/backend/utils/fmgr/fmgr.c
M src/backend/utils/misc/guc.c
M src/backend/utils/sort/sharedtuplestore.c
M src/bin/pg_verifybackup/parse_manifest.c
M src/bin/pg_verifybackup/pg_verifybackup.c
M src/bin/pg_verifybackup/t/005_bad_manifest.pl
M src/bin/pgbench/t/001_pgbench_with_server.pl
M src/bin/scripts/vacuumdb.c
M src/fe_utils/archive.c
M src/test/modules/test_misc/t/001_constraint_validation.pl
M src/test/regress/expected/alter_generic.out
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/expected/limit.out
Use the properly transformed RangeVar for expandTableLikeClause().
commit : b380484a850b6bf7d9fc0d85c555a2366e38451f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 13 Sep 2020 12:51:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 13 Sep 2020 12:51:21 -0400
transformCreateStmt() adjusts the transformed statement's RangeVar
to specify the target schema explicitly, for the express reason
of making sure that auxiliary statements derived by parse
transformation operate on the right table. But the refactoring
I did in commit 502898192 got this wrong and passed the untransformed
RangeVar to expandTableLikeClause(). This could lead to assertion
failures or weird misbehavior if the wrong table was accessed.
Per report from Alexander Lakhin. Like the previous patch, back-patch
to all supported branches.
Discussion: https://postgr.es/m/05051f9d-b32b-cb35-6735-0e9f2ab86b5f@gmail.com
M src/backend/tcop/utility.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
doc: Don't hide the "Up" link when it is the same as "Home"
commit : e6bbe07deec9824eb62fbbf38c4bfe7aaf674d37
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 6 Sep 2020 16:46:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 6 Sep 2020 16:46:13 +0200
The original stylesheets seemed to think this was a good idea, but our
users find it confusing and unhelpful, so undo that logic.
Reported-by: Fabien COELHO <coelho@cri.ensmp.fr>
Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.22.394.2006210914370.859381%40pseudo
M doc/src/sgml/stylesheet.xsl
logtape.c: do not preallocate for tapes when sorting
commit : 93106d71a18afdda2b9bf6e6b8e6c7f9cea2d0ef
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 11 Sep 2020 17:10:02 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 11 Sep 2020 17:10:02 -0700
The preallocation logic is only useful for HashAgg, so disable it when
sorting.
Also, adjust an out-of-date comment.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-Wzn_o7tE2+hRVvwSFghRb75AJ5g-nqGzDUqLYMexjOAe=g@mail.gmail.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/tuplesort.c
M src/include/utils/logtape.h
psql: Display stats target of extended statistics
commit : aeb781107a7ca0cfe109c188534ecbf9c392f6ba
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Sep 2020 16:15:47 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Sep 2020 16:15:47 -0300
The stats target can be set since commit d06215d03, but wasn't shown by
psql.
Author: Justin Pryzby <justin@telsasoft.com>
Discussion: https://postgr.es/m/20200831050047.GG5450@telsasoft.com
Reviewed-by: Georgios Kokolatos <gkokolatos@protonmail.com>
Reviewed-by: Tatsuro Yamada <tatsuro.yamada.tf@nttcom.co.jp>
M src/bin/psql/describe.c
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Update copyright year
commit : 6dcec8fe13b41b0773e9122168a1b53f3e458206
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Sep 2020 12:53:25 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 11 Sep 2020 12:53:25 -0300
Thinko in 40b3e2c201af.
Reported-by: "Wang, Shenhao" <wangsh.fnst@cn.fujitsu.com>
Discussion: https://postgr.es/m/ed98706b82694b57a8c0d339a10732aa@G08CNEXMBPEKD06.g08.fujitsu.local
M src/backend/catalog/pg_cast.c
Doc: some more v13 release note tweaking.
commit : 9892564121d425291c4fd06ff083147dd70b9156
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 18:42:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 18:42:57 -0400
Justin Pryzby
Discussion: https://postgr.es/m/20200910222705.GJ18552@telsasoft.com
M doc/src/sgml/release-13.sgml
Doc: update v13 release notes through today, do a copy-editing pass.
commit : 3965de54e718600a4703233936e56a3202caf73f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 17:43:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 17:43:16 -0400
Also set the release date ... hopefully we won't have to change that.
M doc/src/sgml/release-13.sgml
Doc: fill in "major enhancements" list in v13 release notes.
commit : 3d92252d7d8bf7080ba61f1bda3d27bd8a3617e1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 13:14:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 13:14:09 -0400
Jonathan S. Katz, minor tweaks by me
Discussion: https://postgr.es/m/448a382b-ae07-3126-5a08-aacda9aa28ea@postgresql.org
M doc/src/sgml/release-13.sgml
Use _exit(2) for SIGQUIT during ProcessStartupPacket, too.
commit : 3f29aa48b6df318e43d0efe5735f61175ef38574
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 12:06:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 10 Sep 2020 12:06:26 -0400
Bring the signal handling for startup-packet collection into line
with the policy established in commits bedadc732 and 8e19a8264,
namely don't risk running atexit callbacks when handling SIGQUIT.
Ideally, we'd not do so for SIGTERM or timeout interrupts either,
but that change seems a bit too risky for the back branches.
For now, just improve the comments in this area to describe the risk.
Also relocate where BackendInitialize re-disables these interrupts,
to minimize the code span where they're active. This doesn't buy
a whole lot of safety, but it can't hurt.
In passing, rename startup_die() to remove confusion about whether
it is for the startup process.
Like the previous commits, back-patch to all supported branches.
Discussion: https://postgr.es/m/1850884.1599601164@sss.pgh.pa.us
M src/backend/postmaster/postmaster.c
doc: Remove buggy ICU collation from documentation
commit : 9f358c5ef31327f7a67af783f5f37468bbac3aed
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 10 Sep 2020 15:31:09 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 10 Sep 2020 15:31:09 +0200
We have had multiple reports that point to the
'@colReorder=latn-digit' collation customization being buggy. We have
reported this to ICU and are waiting for a fix. In the meantime,
remove references to this from the documentation and replace it by
another reordering example. Apparently, many users have been picking
up this example specifically from the documentation.
Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Discussion: https://www.postgresql.org/message-id/flat/153201618542.1404.3611626898935613264%40wrigleys.postgresql.org
M doc/src/sgml/charset.sgml
Fix title in reference section
commit : 727f6fb8f71f7ed64e6883164765bbf6cb7684ec
author : Magnus Hagander <magnus@hagander.net>
date : Thu, 10 Sep 2020 14:15:26 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Thu, 10 Sep 2020 14:15:26 +0200
Reported-by: Robert Kahlert
Author: Daniel Gustafsson
M doc/src/sgml/biblio.sgml
Clean up some code and comments in partbounds.c.
commit : d601a930756b96fe05beb75a1b624a8155c98cc8
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 10 Sep 2020 18:00:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 10 Sep 2020 18:00:01 +0900
Do some minor cleanup for commit c8434d64c: 1) remove a useless
assignment (in normal builds) and 2) improve comments a little.
Back-patch to v13 where the aforementioned commit went in.
Author: Etsuro Fujita
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/CAPmGK16yCd2R4=bQ4g8N2dT9TtA5ZU+qNmJ3LPc_nypbNy4_2A@mail.gmail.com
M src/backend/partitioning/partbounds.c
Doc: clean up contributor names.
commit : 32a433455a0a6918b9c8f300d6c0be73ef06e9f9
author : Etsuro Fujita <efujita@postgresql.org>
date : Thu, 10 Sep 2020 16:15:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Thu, 10 Sep 2020 16:15:00 +0900
Standardize Japanese names as given-name-first.
Author: Etsuro Fujita
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/CAPmGK14S5frHWzsKS14R2LeMjKkjr5PeqRGiKZ0os0A+o8BWuQ@mail.gmail.com
M doc/src/sgml/release-13.sgml
doc: Fix some grammar and inconsistencies
commit : b3d89b7a889614a5ad549bae49b8a6fdda3a6bdd
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Sep 2020 15:50:42 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 10 Sep 2020 15:50:42 +0900
Some comments are fixed while on it.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20200818171702.GK17022@telsasoft.com
Backpatch-through: 9.6
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_subscription.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/sources.sgml
M src/backend/replication/logical/relation.c
M src/backend/storage/lmgr/proc.c
Fix rd_firstRelfilenodeSubid for nailed relations, in parallel workers.
commit : 6f15be5bedfc423b8a8f2f0282e2a0eb20a16663
author : Noah Misch <noah@leadboat.com>
date : Wed, 9 Sep 2020 18:50:24 -0700
committer: Noah Misch <noah@leadboat.com>
date : Wed, 9 Sep 2020 18:50:24 -0700
Move applicable code out of RelationBuildDesc(), which nailed relations
bypass. Non-assert builds experienced no known problems. Back-patch to
v13, where commit c6b92041d38512a4176ed76ad06f713d2e6c01a8 introduced
rd_firstRelfilenodeSubid.
Kyotaro Horiguchi. Reported by Justin Pryzby.
Discussion: https://postgr.es/m/20200907023737.GA7158@telsasoft.com
M src/backend/utils/cache/relcache.c
M src/test/regress/expected/reindex_catalog.out
M src/test/regress/sql/reindex_catalog.sql
Make archiver's SIGQUIT handler exit via _exit().
commit : 35e59398abbb562f1e831af206f1d1cc8c3cb7db
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 15:32:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 15:32:34 -0400
Commit 8e19a8264 changed the SIGQUIT handlers of almost all server
processes not to run atexit callbacks. The archiver process was
skipped, perhaps because it's not connected to shared memory; but
it's just as true here that running atexit callbacks in a signal
handler is unsafe. So let's make it work like the rest.
In HEAD and v13, we can use the common SignalHandlerForCrashExit
handler. Before that, just tweak pgarch_exit to use _exit(2)
explicitly.
Like the previous commit, back-patch to all supported branches.
Kyotaro Horiguchi, back-patching by me
Discussion: https://postgr.es/m/1850884.1599601164@sss.pgh.pa.us
M src/backend/postmaster/pgarch.c
Doc: adjust documentation related to index support functions.
commit : 6b473ab4f23736d67c420909ab65c55228dcd6e6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 12:00:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 12:00:49 -0400
Commit 15cb2bd27 neglected to make the running text match the
tables, leaving the reader with the strong impression that
we cannot count. Also, don't drop an unrelated para between
a table and the para describing it.
M doc/src/sgml/xindex.sgml
Minor fixes in docs and error messages.
commit : aa33187164e1d8571f1568f0470aaace8f791876
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 11:53:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 9 Sep 2020 11:53:39 -0400
Alexander Lakhin
Discussion: https://postgr.es/m/ce7debdd-c943-d7a7-9b41-687107b27831@gmail.com
M doc/src/sgml/bgworker.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/intarray.sgml
M src/backend/replication/backup_manifest.c
M src/backend/utils/misc/guc.c
Add missing quote in docs
commit : 1bf0b9c5f58c61c160519a77a8f9dd24cea68b32
author : Magnus Hagander <magnus@hagander.net>
date : Wed, 9 Sep 2020 12:20:53 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Wed, 9 Sep 2020 12:20:53 +0200
Mistake in commit 68b603e1a9.
Reported-by: Ian Barwick
M doc/src/sgml/file-fdw.sgml
Check default partitions constraints while descending
commit : d0230a43fcae6f923fcedfe6f27db7fca8760d95
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 8 Sep 2020 19:35:15 -0300
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 8 Sep 2020 19:35:15 -0300
Partitioning tuple route code assumes that the partition chosen while
descending the partition hierarchy is always the correct one. This is
true except when the partition is the default partition and another
partition has been added concurrently: the partition constraint changes
and we don't recheck it. This can lead to tuples mistakenly being added
to the default partition that should have been rejected.
Fix by rechecking the default partition constraint while descending the
hierarchy.
An isolation test based on the reproduction steps described by Hao Wu
(with tweaks for extra coverage) is included.
Backpatch to 12, where this bug came in with 898e5e3290a7.
Reported by: Hao Wu <hawu@vmware.com>
Author: Amit Langote <amitlangote09@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CA+HiwqFqBmcSSap4sFnCBUEL_VfOMmEKaQ3gwUhyfa4c7J_-nA@mail.gmail.com
Discussion: https://postgr.es/m/DM5PR0501MB3910E97A9EDFB4C775CF3D75A42F0@DM5PR0501MB3910.namprd05.prod.outlook.com
M src/backend/executor/execPartition.c
A src/test/isolation/expected/partition-concurrent-attach.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/partition-concurrent-attach.spec
Adjust cost model for HashAgg that spills to disk.
commit : b61d048e0d480f4311c62bf3026879c83ba9aaad
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 7 Sep 2020 13:31:59 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 7 Sep 2020 13:31:59 -0700
Tomas Vondra observed that the IO behavior for HashAgg tends to be
worse than for Sort. Penalize HashAgg IO costs accordingly.
Also, account for the CPU effort of spilling the tuples and reading
them back.
Discussion: https://postgr.es/m/20200906212112.nzoy5ytrzjjodpfh@development
Reviewed-by: Tomas Vondra
Backpatch-through: 13
M src/backend/optimizer/path/costsize.c
Clarify comments in enforce_generic_type_consistency().
commit : e02c99bff6fcf7a14292cf99b16e4810ea89a2de
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Sep 2020 14:52:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 7 Sep 2020 14:52:33 -0400
Some of the pre-existing comments were vague about whether they
referred to all polymorphic types or only the old-style ones.
Also be more consistent about using the "family 1" vs "family 2"
terminology.
Himanshu Upadhyaya and Tom Lane
Discussion: https://postgr.es/m/CAPF61jBUg9XoMPNuLpoZ+h6UZ2VxKdNt3rQL1xw1GOBwjWzAXQ@mail.gmail.com
M src/backend/parser/parse_coerce.c
Update list of acknowledgments in release notes
commit : 1e83138da14d61d572b0b58a9da6a2cf535ea198
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 7 Sep 2020 09:08:58 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 7 Sep 2020 09:08:58 +0200
current through b7cf9e42ac2d4b153eb781195ebf369d4b8b566e
M doc/src/sgml/release-13.sgml
doc: Tweak sentence for pg_checksums when enabling checksums
commit : b7cf9e42ac2d4b153eb781195ebf369d4b8b566e
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Sep 2020 14:35:13 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 7 Sep 2020 14:35:13 +0900
The previous version of the docs mentioned that files are rewritten,
implying that a second copy of each file gets created, but each file is
updated in-place.
Author: Michael Banck
Reviewed-by: Daniel Gustafsson, Michael Paquier
Discussion: https://postgr.es/m/858086b6a42fb7d17995b6175856f7e7ec44d0a2.camel@credativ.de
Backpatch-through: 12
M doc/src/sgml/ref/pg_checksums.sgml
Change path in example of file_fdw for logs
commit : a82919afe26ad93a135aa5f017a3f31ec5a6547a
author : Magnus Hagander <magnus@hagander.net>
date : Sun, 6 Sep 2020 19:28:32 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Sun, 6 Sep 2020 19:28:32 +0200
It's better to use a relative path into the data directory, than to a
hardcoded home directory of user 'josh'.
Discussion: https://postgr.es/m/CABUevEyuf67Yu_r9gpDMs5MKifK7+-+pe=ZjKzya4JEn9kUk1w@mail.gmail.com
M doc/src/sgml/file-fdw.sgml
Fix misleading error message about inconsistent moving-aggregate types.
commit : f04203ab7e683bc3e961a40b002cf3c8d1d12100
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 12:55:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 12:55:13 -0400
We reported the wrong types when complaining that an aggregate's
moving-aggregate implementation is inconsistent with its regular
implementation.
This was wrong since the feature was introduced, so back-patch
to all supported branches.
Jeff Janes
Discussion: https://postgr.es/m/CAMkU=1x808LH=LPhZp9mNSP0Xd1xDqEd+XeGcvEe48dfE6xV=A@mail.gmail.com
M src/backend/catalog/pg_aggregate.c
Remove useless lstat() call in pg_rewind.
commit : e7f06ea60a3c07128176b294ce3fb0555edd15a5
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 11:50:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 6 Sep 2020 11:50:40 -0400
This is duplicative of an lstat that was just done by the calling
function (traverse_datadir), besides which we weren't really doing
anything with the results. There's not much point in checking to
see if someone removed the file since the previous lstat, since the
FILE_ACTION_REMOVE code would have to deal with missing-file cases
anyway. Moreover, the "exists = false" assignment was a dead store;
nothing was done with that value later.
A syscall saved is a syscall earned, so back-patch to 9.5
where this code was introduced.
Discussion: https://postgr.es/m/1221796.1599329320@sss.pgh.pa.us
M src/bin/pg_rewind/filemap.c
Make new authentication test case more robust.
commit : 8df601bd4067ecdf373ebe43bdf03159a12e2e9d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 21:01:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 21:01:59 -0400
I happened to notice that the new test case I added in b55b4dad9
falls over if one runs "make check" repeatedly; though not in branches
after v10. That's because it was assuming that tmp_check/pgpass
wouldn't exist already. However, it's only been since v11 that the
Makefiles forcibly remove all of tmp_check/ before starting a TAP run.
This fix to unlink the file is therefore strictly necessary only in
v10 ... but it seems wisest to do it across the board, rather than
let the test rely on external logic to get the conditions right.
M src/test/authentication/t/001_password.pl
Fix over-eager ping'ing in logical replication receiver.
commit : 9b81a30f924cf30e6bf3abb3366706440351e163
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 20:20:05 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 20:20:05 -0400
Commit 3f60f690f only partially fixed the broken-status-tracking
issue in LogicalRepApplyLoop: we need ping_sent to have the same
lifetime as last_recv_timestamp. The effects are much less serious
than what that commit fixed, though. AFAICS this would just lead to
extra ping requests being sent, once per second until the sender
responds. Still, it's a bug, so backpatch to v10 as before.
Discussion: https://postgr.es/m/959627.1599248476@sss.pgh.pa.us
M src/backend/replication/logical/worker.c
Fix bogus MaxAllocSize check in logtape.c.
commit : 4a4f3bf983b4abd908585a8d752eee0e47627034
author : Jeff Davis <jdavis@postgresql.org>
date : Fri, 4 Sep 2020 12:01:58 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Fri, 4 Sep 2020 12:01:58 -0700
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-Wz=NZPZc3-fkdmvu=w2itx0PiB-G6QpxHXZOjuvFAzPdZw@mail.gmail.com
Backpatch-through: 13
M src/backend/utils/sort/logtape.c
Collect attribute data on extension owned tables being dumped
commit : 72857482c135677703111855f33550c442108eaa
author : Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Sep 2020 13:53:09 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Fri, 4 Sep 2020 13:53:09 -0400
If this data is not collected, pg_dump segfaults if asked for column
inserts.
Fix by Fabrízio de Royes Mello
Backpatch to release 12 where the bug was introduced.
M src/bin/pg_dump/pg_dump.c
M src/test/modules/test_pg_dump/t/001_base.pl
M src/test/modules/test_pg_dump/test_pg_dump–1.0.sql
C comment: correct use of 64-"byte" cache line size
commit : 7574af1e1b94a4de7a6f63f3a7854a508ab7b0e0
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 4 Sep 2020 13:27:52 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 4 Sep 2020 13:27:52 -0400
Reported-by: Kelly Min
Discussion: https://postgr.es/m/CAPSbxatOiQO90LYpSC3+svAU9-sHgDfEP4oFhcEUt_X=DqFA9g@mail.gmail.com
Backpatch-through: 9.5
M src/include/storage/buf_internals.h
Fix rare deadlock failure in create_am regression test.
commit : afec6ba0bae0258835b81fcc0eeed3ff9c455427
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 12:40:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 4 Sep 2020 12:40:28 -0400
The "DROP ACCESS METHOD gist2" test will require locking the index
to be dropped and then its table; while most ordinary operations
lock a table first then its index. While no concurrent test scripts
should be touching fast_emp4000, autovacuum might chance to be
processing that table when the DROP runs, resulting in a deadlock
failure. This is pretty rare but we see it in the buildfarm from
time to time.
To fix, acquire a lock on fast_emp4000 before issuing the DROP.
Since the point of the exercise is mostly to prevent buildfarm
failures, back-patch to 9.6 where this test was introduced.
Discussion: https://postgr.es/m/839004.1599185607@sss.pgh.pa.us
M src/test/regress/expected/create_am.out
M src/test/regress/sql/create_am.sql
Avoid lockup of a parallel worker when reporting a long error message.
commit : 17424e79d9794b00bdb6653185fc27ba423d8d81
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Sep 2020 16:52:09 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Sep 2020 16:52:09 -0400
Because sigsetjmp() will restore the initial state with signals blocked,
the code path in bgworker.c for reporting an error and exiting would
execute that way. Usually this is fairly harmless; but if a parallel
worker had an error message exceeding the shared-memory communication
buffer size (16K) it would lock up, because it would wait for a
resume-sending signal from its parallel leader which it would never
detect.
To fix, just unblock signals at the appropriate point.
This can be shown to fail back to 9.6. The lack of parallel query
infrastructure makes it difficult to provide a simple test case for
9.5; but I'm pretty sure the issue exists in some form there as well,
so apply the code change there too.
Vignesh C, reviewed by Bharath Rupireddy, Robert Haas, and myself
Discussion: https://postgr.es/m/CALDaNm1d1hHPZUg3xU4XjtWBOLCrA+-2cJcLpw-cePZ=GgDVfA@mail.gmail.com
M src/backend/postmaster/bgworker.c
M src/test/regress/expected/select_parallel.out
M src/test/regress/sql/select_parallel.sql
Doc: mention packager-supplied tools for server start/stop, initdb, etc.
commit : 214bc9038e39e209924514b77db4ee491f95509a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Sep 2020 11:45:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 3 Sep 2020 11:45:26 -0400
The majority of our audience is probably using a pre-packaged Postgres
build rather than raw sources. For them, much of runtime.sgml is not
too relevant, and they should be reading the packager's docs instead.
Add some notes pointing that way in appropriate places.
Text by me; thanks to Daniel Gustafsson for review and discussion,
and to Laurenz Albe for an earlier version.
Discussion: https://postgr.es/m/159430831443.16535.11360317280100947016@wrigleys.postgresql.org
M doc/src/sgml/runtime.sgml
Fix typo in comment
commit : 352b8cf59f400e69a80db12f920adf12a1b0607c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Sep 2020 20:43:23 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Sep 2020 20:43:23 -0400
Introduced by 8b08f7d4820f; backpatch to 11.
Discussion: https://postgr.es/m/20200812214918.GA30353@alvherre.pgsql
M src/bin/pg_dump/pg_dump.h
doc: clarify that max_wal_size is "during" checkpoints
commit : 2c7db50ce83ec375ad6ddeba498f07c78bc10e97
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Sep 2020 17:00:10 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 1 Sep 2020 17:00:10 -0400
Previous wording was "between".
Reported-by: Pavel Luzanov
Discussion: https://postgr.es/m/26906a54-d7cb-2f8e-eed7-e31660024694@postgrespro.ru
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
Raise error on concurrent drop of partitioned index
commit : 15dad5776578e884ee7857abb278a116c0c3e578
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Sep 2020 13:40:43 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 1 Sep 2020 13:40:43 -0400
We were already raising an error for DROP INDEX CONCURRENTLY on a
partitioned table, albeit a different and confusing one:
ERROR: DROP INDEX CONCURRENTLY must be first action in transaction
Change that to throw a more comprehensible error:
ERROR: cannot drop partitioned index \"%s\" concurrently
Michael Paquier authored the test case for indexes on temporary
partitioned tables.
Backpatch to 11, where indexes on partitioned tables were added.
Reported-by: Jan Mussler <jan.mussler@zalando.de>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/16594-d2956ca909585067@postgresql.org
M doc/src/sgml/ref/drop_index.sgml
M src/backend/commands/tablecmds.c
M src/test/regress/expected/indexing.out
M src/test/regress/sql/indexing.sql
Teach libpq to handle arbitrary-length lines in .pgpass files.
commit : 4178b749963cbf28d438b81e38dedcf885ccdda3
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Sep 2020 13:14:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 1 Sep 2020 13:14:44 -0400
Historically there's been a hard-wired assumption here that no line of
a .pgpass file could be as long as NAMEDATALEN*5 bytes. That's a bit
shaky to start off with, because (a) there's no reason to suppose that
host names fit in NAMEDATALEN, and (b) this figure fails to allow for
backslash escape characters. However, it fails completely if someone
wants to use a very long password, and we're now hearing reports of
people wanting to use "security tokens" that can run up to several
hundred bytes. Another angle is that the file is specified to allow
comment lines, but there's no reason to assume that long comment lines
aren't possible.
Rather than guessing at what might be a more suitable limit, let's
replace the fixed-size buffer with an expansible PQExpBuffer. That
adds one malloc/free cycle to the typical use-case, but that's surely
pretty cheap relative to the I/O this code has to do.
Also, add TAP test cases to exercise this code, because there was no
test coverage before.
This reverts most of commit 2eb3bc588, as there's no longer a need for
a warning message about overlength .pgpass lines. (I kept the explicit
check for comment lines, though.)
In HEAD and v13, this also fixes an oversight in 74a308cf5: there's not
much point in explicit_bzero'ing the line buffer if we only do so in two
of the three exit paths.
Back-patch to all supported branches, except that the test case only
goes back to v10 where src/test/authentication/ was added.
Discussion: https://postgr.es/m/4187382.1598909041@sss.pgh.pa.us
M src/interfaces/libpq/fe-connect.c
M src/test/authentication/t/001_password.pl
doc: document how the backup manifest is transferred
commit : 73018f564af91b135e541b28133369c5e3de975d
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 18:48:38 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 18:48:38 -0400
Reported-by: Bernd Helmle
Discussion: https://postgr.es/m/31acf8b0f1f701d53245e0cae38abdf5c3a0d559.camel@oopsware.de
Backpatch-through: 13
M doc/src/sgml/protocol.sgml
doc: add commas after 'i.e.' and 'e.g.'
commit : 1d3ff89ecfbbdd8bf56d4773d8f2749156eeb7c1
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 18:33:37 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 18:33:37 -0400
This follows the American format,
https://jakubmarian.com/comma-after-i-e-and-e-g/. There is no intention
of requiring this format for future text, but making existing text
consistent every few years makes sense.
Discussion: https://postgr.es/m/20200825183619.GA22369@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/bki.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/datatype.sgml
M doc/src/sgml/ddl.sgml
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/glossary.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/mvcc.sgml
M doc/src/sgml/parallel.sgml
M doc/src/sgml/perform.sgml
M doc/src/sgml/postgres-fdw.sgml
M doc/src/sgml/protocol.sgml
M doc/src/sgml/queries.sgml
M doc/src/sgml/ref/create_database.sgml
M doc/src/sgml/ref/create_event_trigger.sgml
M doc/src/sgml/ref/create_function.sgml
M doc/src/sgml/ref/create_procedure.sgml
M doc/src/sgml/ref/create_statistics.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_restore.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/ref/prepare.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/release-13.sgml
M doc/src/sgml/replication-origins.sgml
M doc/src/sgml/runtime.sgml
M doc/src/sgml/sepgsql.sgml
M doc/src/sgml/sources.sgml
M doc/src/sgml/sslinfo.sgml
M doc/src/sgml/tableam.sgml
M doc/src/sgml/textsearch.sgml
M doc/src/sgml/wal.sgml
M doc/src/sgml/xfunc.sgml
M doc/src/sgml/xml2.sgml
C comment: remove mention of use of t_hoff WAL structure member
commit : 787ccf5a5fc92698e8cc8ebfb6fe1a0bf28981a4
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:51:31 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:51:31 -0400
Reported-by: Antonin Houska
Discussion: https://postgr.es/m/21643.1595353537@antos
Backpatch-through: 9.5
M src/include/access/heapam_xlog.h
pg_upgrade doc: mention saving postgresql.conf.auto files
commit : cef30b66fdf619a1b7aa0c2605719e64e980c6f2
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:36:23 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:36:23 -0400
Also mention files included by postgresql.conf.
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/08AD4526-75AB-457B-B2DD-099663F28040@yesql.se
Backpatch-through: 9.5
M doc/src/sgml/ref/pgupgrade.sgml
doc: Update partitioning limitation on BEFORE triggers
commit : 97dc0d1d58916e2f063ce8a7eec6bd4d3bc29197
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 31 Aug 2020 17:04:40 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 31 Aug 2020 17:04:40 -0400
Reported-by: Erwin Brandstetter <brsaweda@gmail.com>
Discussion: https://postgr.es/m/CAGHENJ6Le7S3qJJx2TvWvTwRNS3N=BtoNeb7AF2rZvfNBMeQcg@mail.gmail.com
M doc/src/sgml/ddl.sgml
docs: in mapping SQL to C data types, timestamp isn't a pointer
commit : bef7b917a76af659b4d6bce813702764e2c749db
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:05:53 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 17:05:53 -0400
It is an int64.
Reported-by: ajulien@shaktiware.fr
Discussion: https://postgr.es/m/159845038271.24995.15682121015698255155@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/xfunc.sgml
doc: cross-link file-fdw and CSV config log sections
commit : 9bb4f98bb7745b7339fb7d102cdc36d66c002f67
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:59:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:59:59 -0400
There is an file-fdw example that reads the server config file, so cross
link them.
Reported-by: Oleg Samoilov
Discussion: https://postgr.es/m/159800192078.2886.10431506404995508950@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
M doc/src/sgml/file-fdw.sgml
docs: clarify intermediate certificate creation instructions
commit : 8924ca865b18cac0b655170989bcd9507991d06f
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:21:03 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 16:21:03 -0400
Specifically, explain the v3_ca openssl specification.
Discussion: https://postgr.es/m/20200824175653.GA32411@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/runtime.sgml
docs: replace "stable storage" with "durable" in descriptions
commit : 1bb41c6ad795e018e396eb3e216b2d21fadc865a
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 15:23:19 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 15:23:19 -0400
For PG, "durable storage" has a clear meaning, while "stable storage"
does not, so use the former.
Discussion: https://postgr.es/m/20200817165222.GA31806@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/config.sgml
M doc/src/sgml/monitoring.sgml
doc: improve description of subscripting of arrays
commit : 8006ac1c5117a56fcd138d663a3fe3709c8f9988
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:49:17 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:49:17 -0400
It wasn't clear the non-integers are cast to integers for subscripting,
rather than throwing an error.
Reported-by: sean@materialize.io
Discussion: https://postgr.es/m/159538675800.624.7728794628229799531@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/syntax.sgml
docs: improve 'capitals' inheritance example
commit : 64fd65008c62ec011ba2d0673ca09849ae2b07c8
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:43:04 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:43:04 -0400
Adds constraints and improves wording.
Reported-by: 2552891@gmail.com
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/advanced.sgml
doc: clarify the useful features of procedures
commit : 785affc1d653bac43b7d4fad93f22e9e51aefcc1
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:20:04 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 31 Aug 2020 13:20:04 -0400
This was not clearly documented when procedures were added in PG 11.
Reported-by: Robin Abbi
Discussion: https://postgr.es/m/CAGmg_NX327KKVuJmbWZD=pGutYFxzZjX1rU+3ji8UuX=8ONn9Q@mail.gmail.com
Backpatch-through: 11
M doc/src/sgml/xfunc.sgml
Fix docs bug stating file_fdw requires absolute paths
commit : 3a1f6a2581e41ee2c724e8422675942d30a52ff7
author : Magnus Hagander <magnus@hagander.net>
date : Mon, 31 Aug 2020 13:03:54 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Mon, 31 Aug 2020 13:03:54 +0200
It has always (since the first commit) worked with relative paths, so
use the same wording as other parts of the documentation.
Author: Bruce Momjian
Discussion: https://postgr.es/m/CABUevExx-hm=cit+A9LeKBH39srvk8Y2tEZeEAj5mP8YfzNKUg@mail.gmail.com
M doc/src/sgml/file-fdw.sgml
Mark factorial operator, and postfix operators in general, as deprecated.
commit : 845cfe012fd15300cd090b05fb4029a26b848a67
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 30 Aug 2020 14:37:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 30 Aug 2020 14:37:24 -0400
Per discussion, we're planning to remove parser support for postfix
operators in order to simplify the grammar. So it behooves us to
put out a deprecation notice at least one release before that.
There is only one built-in postfix operator, ! for factorial.
Label it deprecated in the docs and in pg_description, and adjust
some examples that formerly relied on it. (The sister prefix
operator !! is also deprecated. We don't really have to remove
that one, but since we're suggesting that people use factorial()
instead, it seems better to remove both operators.)
Also state in the CREATE OPERATOR ref page that postfix operators
in general are going away.
Although this changes the initial contents of pg_description,
I did not force a catversion bump; it doesn't seem essential.
In v13, also back-patch 4c5cf5431, so that there's someplace for
the <link>s to point to.
Mark Dilger and John Naylor, with some adjustments by me
Discussion: https://postgr.es/m/BE2DF53D-251A-4E26-972F-930E523580E9@enterprisedb.com
M doc/src/sgml/func.sgml
M doc/src/sgml/ref/create_operator.sgml
M doc/src/sgml/syntax.sgml
M doc/src/sgml/typeconv.sgml
M src/include/catalog/pg_operator.dat
M src/include/catalog/pg_proc.dat
Fix code for re-finding scan position in a multicolumn GIN index.
commit : 1df14a5669428c0060ebdcebed8c1f807b659893
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Aug 2020 17:36:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 27 Aug 2020 17:36:13 -0400
collectMatchBitmap() needs to re-find the index tuple it was previously
looking at, after transiently dropping lock on the index page it's on.
The tuple should still exist and be at its prior position or somewhere
to the right of that, since ginvacuum never removes tuples but
concurrent insertions could add one. However, there was a thinko in
that logic, to the effect of expecting any inserted tuples to have the
same index "attnum" as what we'd been scanning. Since there's no
physical separation of tuples with different attnums, it's not terribly
hard to devise scenarios where this fails, leading to transient "lost
saved point in index" errors. (While I've duplicated this with manual
testing, it seems impossible to make a reproducible test case with our
available testing technology.)
Fix by just continuing the scan when the attnum doesn't match.
While here, improve the error message used if we do fail, so that it
matches the wording used in btree for a similar case.
collectMatchBitmap()'s posting-tree code path was previously not
exercised at all by our regression tests. While I can't make
a regression test that exhibits the bug, I can at least improve
the code coverage here, so do that. The test case I made for this
is an extension of one added by 4b754d6c1, so it only works in
HEAD and v13; didn't seem worth trying hard to back-patch it.
Per bug #16595 from Jesse Kinkead. This has been broken since
multicolumn capability was added to GIN (commit 27cb66fdf),
so back-patch to all supported branches.
Discussion: https://postgr.es/m/16595-633118be8eef9ce2@postgresql.org
M src/backend/access/gin/ginget.c
M src/test/regress/expected/gin.out
M src/test/regress/sql/gin.sql
docs: client certificates are always sent to the server
commit : 32c42b815ad2377c0bddc4f55b7bf32a5204a87d
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 25 Aug 2020 09:53:12 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 25 Aug 2020 09:53:12 -0400
They are not "requested" by the server.
Reported-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20200825.155320.986648039251743210.horikyota.ntt@gmail.com
Backpatch-through: 9.5
M doc/src/sgml/libpq.sgml
doc: Fix up title case
commit : 62340f8eca4c9d4e49097eaf0a341aca8450b56b
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Aug 2020 07:29:05 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 25 Aug 2020 07:29:05 +0200
This fixes some instances that were missed in earlier processings and
that now look a bit strange because they are inconsistent with nearby
titles.
M doc/src/sgml/dml.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/plpgsql.sgml
Improve the vacuum error context phase information.
commit : b4ef5ac0b7bfa734ad908bb5577cf4ee65ce2974
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 24 Aug 2020 08:22:54 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 24 Aug 2020 08:22:54 +0530
We were displaying the wrong phase information for 'info' message in the
index clean up phase because we were switching to the previous phase a bit
early. We were also not displaying context information for heap phase
unless the block number is valid which is fine for error cases but for
messages at 'info' or lower error level it appears to be inconsistent with
index phase information.
Reported-by: Sawada Masahiko
Author: Sawada Masahiko
Reviewed-by: Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CA+fd4k4HcbhPnCs7paRTw1K-AHin8y4xKomB9Ru0ATw0UeTy2w@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
Avoid pushing quals down into sub-queries that have grouping sets.
commit : de627adaad3a161b3ae0cafeb76c20519845799b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Aug 2020 14:46:40 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Aug 2020 14:46:40 -0400
The trouble with doing this is that an apparently-constant subquery
output column isn't really constant if it is a grouping column that
appears in only some of the grouping sets. A qual using such a
column would be subject to incorrect const-folding after push-down,
as seen in bug #16585 from Paul Sivash.
To fix, just disable qual pushdown altogether if the sub-query has
nonempty groupingSets. While we could imagine far less restrictive
solutions, there is not much point in working harder right now,
because subquery_planner() won't move HAVING clauses to WHERE within
such a subquery. If the qual stays in HAVING it's not going to be
a lot more useful than if we'd kept it at the outer level.
Having said that, this restriction could be removed if we used a
parsetree representation that distinguished such outputs from actual
constants, which is something I hope to do in future. Hence, make
the patch a minimal addition rather than integrating it more tightly
(e.g. by renumbering the existing items in subquery_is_pushdown_safe's
comment).
Back-patch to 9.5 where grouping sets were introduced.
Discussion: https://postgr.es/m/16585-9d8c340d23ade8c1@postgresql.org
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/groupingsets.out
M src/test/regress/sql/groupingsets.sql
Fix ALTER TABLE's scheduling rules for AT_AddConstraint subcommands.
commit : 47de6ac11b53cf24393a2ef048f9e8921163da0a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Aug 2020 12:34:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 22 Aug 2020 12:34:17 -0400
Commit 1281a5c90 rearranged the logic in this area rather drastically,
and it broke the case of adding a foreign key constraint in the same
ALTER that adds the pkey or unique constraint it depends on. While
self-referential fkeys are surely a pretty niche case, this used to
work so we shouldn't break it.
To fix, reorganize the scheduling rules in ATParseTransformCmd so
that a transformed AT_AddConstraint subcommand will be delayed into
a later pass in all cases, not only when it's been spit out as a
side-effect of parsing some other command type.
Also tweak the logic so that we won't run ATParseTransformCmd twice
while doing this. It seems to work even without that, but it's
surely wasting cycles to do so.
Per bug #16589 from Jeremy Evans. Back-patch to v13 where the new
code was introduced.
Discussion: https://postgr.es/m/16589-31c8d981ca503896@postgresql.org
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
doc: Fix format, incorrect structure names and markup inconsistencies
commit : 610394c7b876bc841464aad38f8abed09d63924e
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 22 Aug 2020 22:26:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 22 Aug 2020 22:26:18 +0900
Author: Alexander Lakhin
Discussion: https://postgr.es/m/a2345841-10a5-4eef-257c-02302347cf39@gmail.com
Backpatch-through: 13
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/protocol.sgml
docs: improve description of how to handle multiple databases
commit : 3669792934593a7b682c5b32dea41c9e3b48e6cc
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Aug 2020 20:23:09 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Aug 2020 20:23:09 -0400
This is a redesign of the intro to the managing databases chapter.
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org
Author: David G. Johnston
Backpatch-through: 9.5
M doc/src/sgml/manage-ag.sgml
docs: add COMMENT examples for new features, rename rtree
commit : 35f2d22ba909735448606fefc126b73e4fb3c627
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Aug 2020 18:29:37 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 21 Aug 2020 18:29:37 -0400
Reported-by: Jürgen Purtz
Discussion: https://postgr.es/m/15ec5428-d46a-1725-f38d-44986a977abb@purtz.de
Author: Jürgen Purtz
Backpatch-through: 11
M doc/src/sgml/ref/comment.sgml
Fix handling of CREATE TABLE LIKE with inheritance.
commit : 894f5dea76e1a59fa2f3e1905c6a44f254d9d08b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Aug 2020 15:00:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 21 Aug 2020 15:00:42 -0400
If a CREATE TABLE command uses both LIKE and traditional inheritance,
Vars in CHECK constraints and expression indexes that are absorbed
from a LIKE parent table tended to get mis-numbered, resulting in
wrong answers and/or bizarre error messages (though probably not any
actual crashes, thanks to validation occurring in the executor).
In v12 and up, the same could happen to Vars in GENERATED expressions,
even in cases with no LIKE clause but multiple traditional-inheritance
parents.
The cause of the problem for LIKE is that parse_utilcmd.c supposed
it could renumber such Vars correctly during transformCreateStmt(),
which it cannot since we have not yet accounted for columns added via
inheritance. Fix that by postponing processing of LIKE INCLUDING
CONSTRAINTS, DEFAULTS, GENERATED, INDEXES till after we've performed
DefineRelation().
The error with GENERATED and multiple inheritance is a simple oversight
in MergeAttributes(); it knows it has to renumber Vars in inherited
CHECK constraints, but forgot to apply the same processing to inherited
GENERATED expressions (a/k/a defaults).
Per bug #16272 from Tom Gottfried. The non-GENERATED variants of the
issue are ancient, presumably dating right back to the addition of
CREATE TABLE LIKE; hence back-patch to all supported branches.
Discussion: https://postgr.es/m/16272-6e32da020e9a9381@postgresql.org
M src/backend/commands/tablecmds.c
M src/backend/parser/parse_utilcmd.c
M src/backend/tcop/utility.c
M src/include/nodes/parsenodes.h
M src/include/parser/parse_utilcmd.h
M src/test/modules/test_ddl_deparse/expected/create_table.out
M src/test/modules/test_ddl_deparse/test_ddl_deparse.c
M src/test/regress/expected/create_table_like.out
M src/test/regress/sql/create_table_like.sql
Fix explain regression test failure.
commit : bc2ebf3acfd287247cd9ee286c44e969af1c81de
author : Fujii Masao <fujii@postgresql.org>
date : Sat, 22 Aug 2020 01:22:55 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Sat, 22 Aug 2020 01:22:55 +0900
Commit 9d701e624f caused the regression test for EXPLAIN to fail on
the buildfarm member prion. This happened because of instability of
test output, i.e., in text format, whether "Planning:" line is output
varies depending on the system state.
This commit updated the regression test so that it ignores that
"Planning:" line to produce more stable test output and get rid of
the test failure.
Back-patch to v13.
Author: Fujii Masao
Discussion: https://postgr.es/m/1803897.1598021621@sss.pgh.pa.us
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Rework EXPLAIN for planner's buffer usage.
commit : 674899ae02c375b03411c0676e7cfb4bafeebac9
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 21 Aug 2020 20:48:59 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 21 Aug 2020 20:48:59 +0900
Commit ce77abe63c allowed EXPLAIN (BUFFERS) to report the information
on buffer usage during planning phase. However three issues were
reported regarding this feature.
(1) Previously, EXPLAIN option BUFFERS required ANALYZE. So the query
had to be actually executed by specifying ANALYZE even when we
want to see only the planner's buffer usage. This was inconvenient
especially when the query was write one like DELETE.
(2) EXPLAIN included the planner's buffer usage in summary
information. So SUMMARY option had to be enabled to report that.
Also this format was confusing.
(3) The output structure for planning information was not consistent
between TEXT format and the others. For example, "Planning" tag
was output in JSON format, but not in TEXT format.
For (1), this commit allows us to perform EXPLAIN (BUFFERS) without
ANALYZE to report the planner's buffer usage.
For (2), this commit changed EXPLAIN output so that the planner's
buffer usage is reported before summary information.
For (3), this commit made the output structure for planning
information more consistent between the formats.
Back-patch to v13 where the planner's buffer usage was allowed to
be reported in EXPLAIN.
Reported-by: Pierre Giraud, David Rowley
Author: Fujii Masao
Reviewed-by: David Rowley, Julien Rouhaud, Pierre Giraud
Discussion: https://postgr.es/m/07b226e6-fa49-687f-b110-b7c37572f69e@dalibo.com
M doc/src/sgml/ref/explain.sgml
M src/backend/commands/explain.c
M src/test/regress/expected/explain.out
M src/test/regress/sql/explain.sql
Fix a few typos in JIT comments and README
commit : 0d7437de73b68b0105d00ff4fed3e7894b02f6d5
author : David Rowley <drowley@postgresql.org>
date : Fri, 21 Aug 2020 09:34:47 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 21 Aug 2020 09:34:47 +1200
Reviewed-by: Abhijit Menon-Sen
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAApHDvobgmCs6CohqhKTUf7D8vffoZXQTCBTERo9gbOeZmvLTw%40mail.gmail.com
Backpatch-through: 11, where JIT was added
M src/backend/jit/README
M src/include/jit/llvmjit_emit.h
Revise REINDEX CONCURRENTLY recovery instructions
commit : 2a9f042c47053e3ce73f634cce9622fa0151f63c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 20 Aug 2020 13:49:04 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 20 Aug 2020 13:49:04 -0400
When the leftover invalid index is "ccold", there's no need to re-run
the command. Reword the instructions to make that explicit.
Backpatch to 12, where REINDEX CONCURRENTLY appeared.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/20200819211312.GA15497@alvherre.pgsql
M doc/src/sgml/ref/reindex.sgml
Suppress unnecessary RelabelType nodes in yet more cases.
commit : 69ffc2f838990fb2c802087091ce7c2ff1b735eb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Aug 2020 14:07:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 19 Aug 2020 14:07:49 -0400
Commit a477bfc1d fixed eval_const_expressions() to ensure that it
didn't generate unnecessary RelabelType nodes, but I failed to notice
that some other places in the planner had the same issue. Really
noplace in the planner should be using plain makeRelabelType(), for
fear of generating expressions that should be equal() to semantically
equivalent trees, but aren't.
An example is that because canonicalize_ec_expression() failed
to be careful about this, we could end up with an equivalence class
containing both a plain Const, and a Const-with-RelabelType
representing exactly the same value. So far as I can tell this led to
no visible misbehavior, but we did waste a bunch of cycles generating
and evaluating "Const = Const-with-RelabelType" to prove such entries
are redundant.
Hence, move the support function added by a477bfc1d to where it can
be more generally useful, and use it in the places where planner code
previously used makeRelabelType.
Back-patch to v12, like the previous patch. While I have no concrete
evidence of any real misbehavior here, it's certainly possible that
I overlooked a case where equivalent expressions that aren't equal()
could cause a user-visible problem. In any case carrying extra
RelabelType nodes through planning to execution isn't very desirable.
Discussion: https://postgr.es/m/1311836.1597781384@sss.pgh.pa.us
M src/backend/nodes/nodeFuncs.c
M src/backend/optimizer/path/equivclass.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/nodeFuncs.h
Avoid non-constant format string argument to fprintf().
commit : bff41b2b8938f5e3e7d536da6d7a14a1648b4dec
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 18 Aug 2020 13:13:09 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Tue, 18 Aug 2020 13:13:09 +0300
As Tom Lane pointed out, it could defeat the compiler's printf() format
string verification.
Backpatch to v12, like that patch that introduced it.
Discussion: https://www.postgresql.org/message-id/1069283.1597672779%40sss.pgh.pa.us
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_rewind/pg_rewind.c
Disable autovacuum for BRIN test table
commit : b83f1bcca0bb460413da4ce856b030502b4f71c5
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 17 Aug 2020 16:20:06 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 17 Aug 2020 16:20:06 -0400
This should improve stability in the tests.
Per buildfarm member hyrax (CLOBBER_CACHE_ALWAYS) via Tom Lane.
Discussion: https://postgr.es/m/871534.1597503261@sss.pgh.pa.us
M src/test/regress/expected/brin.out
M src/test/regress/sql/brin.sql
Doc: fix description of UNION/CASE/etc type unification.
commit : 8216a1d3ed27364c7b2a9e6db76f15202a6070c1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Aug 2020 15:40:07 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 17 Aug 2020 15:40:07 -0400
The description of what select_common_type() does was not terribly
accurate. Improve it.
David Johnston and Tom Lane
Discussion: https://postgr.es/m/1019930.1597613200@sss.pgh.pa.us
M doc/src/sgml/typeconv.sgml
Fix printing last progress report line in client programs.
commit : 5ca1798f32b7fe730fef7ccd8d69e785a50134b8
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 17 Aug 2020 09:27:29 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Mon, 17 Aug 2020 09:27:29 +0300
A number of client programs have a "--progress" option that when printing
to a TTY, updates the current line by printing a '\r' and overwriting it.
After the last line, '\n' needs to be printed to move the cursor to the
next line. pg_basebackup and pgbench got this right, but pg_rewind and
pg_checksums were slightly wrong. pg_rewind printed the newline to stdout
instead of stderr, and pg_checksums printed the newline even when not
printing to a TTY. Fix them, and also add a 'finished' argument to
pg_basebackup's progress_report() function, to keep it consistent with
the other programs.
Backpatch to v12. pg_rewind's newline was broken with the logging changes
in commit cc8d415117 in v12, and pg_checksums was introduced in v12.
Discussion: https://www.postgresql.org/message-id/82b539e5-ae33-34b0-1aee-22b3379fd3eb@iki.fi
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_checksums/pg_checksums.c
M src/bin/pg_rewind/pg_rewind.c
M src/bin/pg_rewind/pg_rewind.h
doc: Fix description about bgwriter and checkpoint in HA section
commit : 3424c6bef0b897f9c6800f55d34a162a07319795
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Aug 2020 10:24:24 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 17 Aug 2020 10:24:24 +0900
Since 806a2ae, the work of the bgwriter is split the checkpointer, but a
portion of the documentation did not get the message.
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CA+fd4k6jXxjAtjMVC=wG3=QGpauZBtcgN3Jhw+oV7zXGKVLKzQ@mail.gmail.com
Backpatch-through: 9.5
M doc/src/sgml/high-availability.sgml
Doc: various improvements for pg_basebackup reference page.
commit : 277e49eca73a719695d0b74360b54124e76c6833
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 Aug 2020 15:43:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 15 Aug 2020 15:43:34 -0400
Put the -r option in the right section (it certainly isn't an
option controlling "the location and format of the output").
Clarify the behavior of the tablespace and waldir options
(that part per gripe from robert@interactive.co.uk).
Make a large number of small copy-editing fixes in text that
visibly wasn't written by native speakers, and try to avoid
grammatical inconsistencies between the descriptions of
the different options.
Back-patch to v13, since HEAD hasn't meaningfully diverged yet.
Discussion: https://postgr.es/m/159749418850.14322.216503677134569752@wrigleys.postgresql.org
M doc/src/sgml/ref/pg_basebackup.sgml
Prevent concurrent SimpleLruTruncate() for any given SLRU.
commit : 592a589a04bd456410b853d86bd05faa9432cbbb
author : Noah Misch <noah@leadboat.com>
date : Sat, 15 Aug 2020 10:15:53 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 15 Aug 2020 10:15:53 -0700
The SimpleLruTruncate() header comment states the new coding rule. To
achieve this, add locktype "frozenid" and two LWLocks. This closes a
rare opportunity for data loss, which manifested as "apparent
wraparound" or "could not access status of transaction" errors. Data
loss is more likely in pg_multixact, due to released branches' thin
margin between multiStopLimit and multiWrapLimit. If a user's physical
replication primary logged ": apparent wraparound" messages, the user
should rebuild standbys of that primary regardless of symptoms. At less
risk is a cluster having emitted "not accepting commands" errors or
"must be vacuumed" warnings at some point. One can test a cluster for
this data loss by running VACUUM FREEZE in every database. Back-patch
to 9.5 (all supported versions).
Discussion: https://postgr.es/m/20190218073103.GA1434723@rfd.leadboat.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/monitoring.sgml
M src/backend/access/transam/slru.c
M src/backend/access/transam/subtrans.c
M src/backend/commands/async.c
M src/backend/commands/vacuum.c
M src/backend/storage/lmgr/lmgr.c
M src/backend/storage/lmgr/lwlocknames.txt
M src/backend/utils/adt/lockfuncs.c
M src/include/storage/lmgr.h
M src/include/storage/lock.h
Be more careful about the shape of hashable subplan clauses.
commit : b538e83f17e36fd0fe0686815da0ff5866ce8a64
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 22:14:03 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 22:14:03 -0400
nodeSubplan.c expects that the testexpr for a hashable ANY SubPlan
has the form of one or more OpExprs whose LHS is an expression of the
outer query's, while the RHS is an expression over Params representing
output columns of the subquery. However, the planner only went as far
as verifying that the clauses were all binary OpExprs. This works
99.99% of the time, because the clauses have the right shape when
emitted by the parser --- but it's possible for function inlining to
break that, as reported by PegoraroF10. To fix, teach the planner
to check that the LHS and RHS contain the right things, or more
accurately don't contain the wrong things. Given that this has been
broken for years without anyone noticing, it seems sufficient to just
give up hashing when it happens, rather than go to the trouble of
commuting the clauses back again (which wouldn't necessarily work
anyway).
While poking at that, I also noticed that nodeSubplan.c had a baked-in
assumption that the number of hash clauses is identical to the number
of subquery output columns. Again, that's fine as far as parser output
goes, but it's not hard to break it via function inlining. There seems
little reason for that assumption though --- AFAICS, the only thing
it's buying us is not having to store the number of hash clauses
explicitly. Adding code to the planner to reject such cases would take
more code than getting nodeSubplan.c to cope, so I fixed it that way.
This has been broken for as long as we've had hashable SubPlans,
so back-patch to all supported branches.
Discussion: https://postgr.es/m/1549209182255-0.post@n3.nabble.com
M src/backend/executor/nodeSubplan.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/util/clauses.c
M src/include/nodes/execnodes.h
M src/include/optimizer/clauses.h
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
pg_dump: fix dependencies on FKs to partitioned tables
commit : b7cc21c57d738b41f6116f46d566b9949a433747
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 Aug 2020 17:33:31 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 14 Aug 2020 17:33:31 -0400
Parallel-restoring a foreign key that references a partitioned table
with several levels of partitions can fail:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 6684; 2606 29166 FK CONSTRAINT fk fk_a_fkey postgres
pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "pk"
Command was: ALTER TABLE fkpart3.fk
ADD CONSTRAINT fk_a_fkey FOREIGN KEY (a) REFERENCES fkpart3.pk(a);
This happens in parallel restore mode because some index partitions
aren't yet attached to the topmost partitioned index that the FK uses,
and so the index is still invalid. The current code marks the FK as
dependent on the first level of index-attach dump objects; the bug is
fixed by recursively marking the FK on their children.
Backpatch to 12, where FKs to partitioned tables were introduced.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/3170626.1594842723@sss.pgh.pa.us
Backpatch: 12-master
M src/bin/pg_dump/pg_dump.c
Fix postmaster's behavior during smart shutdown.
commit : 1c6066fbdb70a92b10b8616652d25c4434cd222e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 13:26:57 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 14 Aug 2020 13:26:57 -0400
Up to now, upon receipt of a SIGTERM ("smart shutdown" command), the
postmaster has immediately killed all "optional" background processes,
and subsequently refused to launch new ones while it's waiting for
foreground client processes to exit. No doubt this seemed like an OK
policy at some point; but it's a pretty bad one now, because it makes
for a seriously degraded environment for the remaining clients:
* Parallel queries are killed, and new ones fail to launch. (And our
parallel-query infrastructure utterly fails to deal with the case
in a reasonable way --- it just hangs waiting for workers that are
not going to arrive. There is more work needed in that area IMO.)
* Autovacuum ceases to function. We can tolerate that for awhile,
but if bulk-update queries continue to run in the surviving client
sessions, there's eventually going to be a mess. In the worst case
the system could reach a forced shutdown to prevent XID wraparound.
* The bgwriter and walwriter are also stopped immediately, likely
resulting in performance degradation.
Hence, let's rearrange things so that the only immediate change in
behavior is refusing to let in new normal connections. Once the last
normal connection is gone, shut everything down as though we'd received
a "fast" shutdown. To implement this, remove the PM_WAIT_BACKUP and
PM_WAIT_READONLY states, instead staying in PM_RUN or PM_HOT_STANDBY
while normal connections remain. A subsidiary state variable tracks
whether or not we're letting in new connections in those states.
This also allows having just one copy of the logic for killing child
processes in smart and fast shutdown modes. I moved that logic into
PostmasterStateMachine() by inventing a new state PM_STOP_BACKENDS.
Back-patch to 9.6 where parallel query was added. In principle
this'd be a good idea in 9.5 as well, but the risk/reward ratio
is not as good there, since lack of autovacuum is not a problem
during typical uses of smart shutdown.
Per report from Bharath Rupireddy.
Patch by me, reviewed by Thomas Munro
Discussion: https://postgr.es/m/CALj2ACXAZ5vKxT9P7P89D87i3MDO9bfS+_bjMHgnWJs8uwUOOw@mail.gmail.com
M doc/src/sgml/ref/pg_ctl-ref.sgml
M src/backend/postmaster/postmaster.c
M src/backend/utils/init/postinit.c
M src/include/libpq/libpq-be.h
Fix typo in test comment.
commit : cabec1dbdf405211c1a4d30f52a91e8de2cf7226
author : Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 14 Aug 2020 10:40:50 +0300
committer: Heikki Linnakangas <heikki.linnakangas@iki.fi>
date : Fri, 14 Aug 2020 10:40:50 +0300
M src/test/regress/expected/stats_ext.out
M src/test/regress/sql/stats_ext.sql
Doc: improve examples for json_populate_record() and related functions.
commit : 124772e3cac6d3bf6c1f6d32518209c6a3fdd3eb
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Aug 2020 20:00:39 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 13 Aug 2020 20:00:39 -0400
Make these examples self-contained by providing declarations of the
user-defined row types they rely on. There wasn't room to do this
in the old doc format, but now there is, and I think it makes the
examples a good bit less confusing.
M doc/src/sgml/func.sgml
Handle new HOT chains in index-build table scans
commit : 2f29fd4cb2522dfd64888892f0442a0ead8c6dd0
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Aug 2020 17:33:49 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 13 Aug 2020 17:33:49 -0400
When a table is scanned by heapam_index_build_range_scan (née
IndexBuildHeapScan) and the table lock being held allows concurrent data
changes, it is possible for new HOT chains to sprout in a page that were
unknown when the scan of a page happened. This leads to an error such
as
ERROR: failed to find parent tuple for heap-only tuple at (X,Y) in table "tbl"
because the root tuple was not present when we first obtained the list
of the page's root tuples. This can be fixed by re-obtaining the list
of root tuples, if we see that a heap-only tuple appears to point to a
non-existing root.
This was reported by Anastasia as occurring for BRIN summarization
(which exists since 9.5), but I think it could theoretically also happen
with CREATE INDEX CONCURRENTLY (much older) or REINDEX CONCURRENTLY
(very recent). It seems a happy coincidence that BRIN forces us to
backpatch this all the way to 9.5.
Reported-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Diagnosed-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Co-authored-by: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/602d8487-f0b2-5486-0088-0f372b2549fa@postgrespro.ru
Backpatch: 9.5 - master
M src/backend/access/heap/heapam_handler.c
M src/backend/access/heap/pruneheap.c
BRIN: Handle concurrent desummarization properly
commit : 8782ea2f36a2468e08c227c8d7907a5a1a9f5e32
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 12 Aug 2020 15:33:36 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 12 Aug 2020 15:33:36 -0400
If a page range is desummarized at just the right time concurrently with
an index walk, BRIN would raise an error indicating index corruption.
This is scary and unhelpful; silently returning that the page range is
not summarized is sufficient reaction.
This bug was introduced by commit 975ad4e602ff as additional protection
against a bug whose actual fix was elsewhere. Backpatch equally.
Reported-By: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Diagnosed-By: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/2588667e-d07d-7e10-74e2-7e1e46194491@postgrespro.ru
Backpatch: 9.5 - master
M src/backend/access/brin/brin_revmap.c
Stamp 13beta3.
commit : 1754a71986e806330ac3ab9e8125c902286b829d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Aug 2020 17:12:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Aug 2020 17:12:51 -0400
M configure
M configure.in
Document clashes between logical replication and untrusted users.
commit : b601f24c875d79e747eb8b50a4b1555ac22cf8f9
author : Noah Misch <noah@leadboat.com>
date : Mon, 10 Aug 2020 09:22:54 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 10 Aug 2020 09:22:54 -0700
Back-patch to v10, which introduced logical replication.
Security: CVE-2020-14349
M doc/src/sgml/logical-replication.sgml
Empty search_path in logical replication apply worker and walsender.
commit : 412c5c4010c0bec294f60a10cd56929680d3f95b
author : Noah Misch <noah@leadboat.com>
date : Mon, 10 Aug 2020 09:22:54 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 10 Aug 2020 09:22:54 -0700
This is like CVE-2018-1058 commit
582edc369cdbd348d68441fc50fa26a84afd0c1a. Today, a malicious user of a
publisher or subscriber database can invoke arbitrary SQL functions
under an identity running replication, often a superuser. This fix may
cause "does not exist" or "no schema has been selected to create in"
errors in a replication process. After upgrading, consider watching
server logs for these errors. Objects accruing schema qualification in
the wake of the earlier commit are unlikely to need further correction.
Back-patch to v10, which introduced logical replication.
Security: CVE-2020-14349
M src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
M src/backend/replication/logical/worker.c
M src/test/subscription/t/001_rep_changes.pl
Move connect.h from fe_utils to src/include/common.
commit : 41dae35532d40041297ee728eac1f4584af05570
author : Noah Misch <noah@leadboat.com>
date : Mon, 10 Aug 2020 09:22:54 -0700
committer: Noah Misch <noah@leadboat.com>
date : Mon, 10 Aug 2020 09:22:54 -0700
Any libpq client can use the header. Clients include backend components
postgres_fdw, dblink, and logical replication apply worker. Back-patch
to v10, because another fix needs this. In released branches, just copy
the header and keep the original.
M contrib/oid2name/oid2name.c
M contrib/vacuumlo/vacuumlo.c
M src/bin/pg_basebackup/streamutil.c
M src/bin/pg_dump/pg_backup_db.c
M src/bin/pg_dump/pg_dump.c
M src/bin/pg_dump/pg_dumpall.c
M src/bin/pg_rewind/libpq_fetch.c
M src/bin/pg_upgrade/server.c
M src/bin/scripts/common.c
M src/bin/scripts/reindexdb.c
M src/bin/scripts/vacuumdb.c
M src/fe_utils/cancel.c
R096 src/include/fe_utils/connect.h src/include/common/connect.h
M src/tools/findoidjoins/findoidjoins.c
Make contrib modules' installation scripts more secure.
commit : 98ca64899cec6a4bf3099481aff43b8777319c41
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Aug 2020 10:44:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 10 Aug 2020 10:44:42 -0400
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
M contrib/btree_gist/btree_gist–1.1–1.2.sql
M contrib/citext/citext–1.1–1.2.sql
M contrib/citext/citext–1.2–1.3.sql
M contrib/cube/cube–1.1–1.2.sql
M contrib/cube/cube–1.3–1.4.sql
M contrib/earthdistance/earthdistance–1.1.sql
M contrib/earthdistance/earthdistance.control
M contrib/hstore/hstore–1.1–1.2.sql
M contrib/hstore/hstore–1.3–1.4.sql
M contrib/hstore_plperl/hstore_plperl.control
M contrib/intagg/intagg–1.0–1.1.sql
M contrib/intarray/intarray–1.1–1.2.sql
M contrib/ltree/ltree–1.0–1.1.sql
M contrib/pg_trgm/pg_trgm–1.2–1.3.sql
M contrib/seg/seg–1.0–1.1.sql
M contrib/seg/seg–1.2–1.3.sql
M doc/src/sgml/earthdistance.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/hstore.sgml
M doc/src/sgml/ltree.sgml
M doc/src/sgml/ref/create_extension.sgml
M src/backend/commands/extension.c
M src/backend/commands/operatorcmds.c
M src/backend/commands/typecmds.c
Translation updates
commit : 378bd1ed6e4314a8b8b32c555f73524c2283b016
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 Aug 2020 15:15:50 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 10 Aug 2020 15:15:50 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 42620448109473e0d2271f0f0015d3647fbbfff6
M src/bin/initdb/po/uk.po
M src/bin/pg_archivecleanup/po/uk.po
M src/bin/pg_basebackup/po/es.po
M src/bin/pg_checksums/nls.mk
A src/bin/pg_checksums/po/uk.po
A src/bin/pg_checksums/po/zh_CN.po
M src/bin/pg_config/po/uk.po
M src/bin/pg_controldata/po/uk.po
M src/bin/pg_ctl/po/uk.po
M src/bin/pg_dump/po/de.po
M src/bin/pg_resetwal/nls.mk
A src/bin/pg_resetwal/po/uk.po
M src/bin/pg_rewind/nls.mk
A src/bin/pg_rewind/po/uk.po
M src/bin/pg_test_fsync/po/uk.po
M src/bin/pg_test_timing/po/uk.po
M src/bin/pg_verifybackup/nls.mk
A src/bin/pg_verifybackup/po/zh_CN.po
M src/bin/pg_waldump/nls.mk
A src/bin/pg_waldump/po/uk.po
M src/bin/pg_waldump/po/zh_CN.po
M src/bin/psql/po/uk.po
M src/interfaces/ecpg/ecpglib/po/uk.po
M src/interfaces/ecpg/preproc/po/uk.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/uk.po
M src/interfaces/libpq/po/zh_CN.po
M src/pl/plpgsql/src/po/uk.po
M src/pl/plpython/po/uk.po
M src/pl/tcl/po/uk.po
Doc: update v13 release notes for changes through today.
commit : bc635dd16388015c72e47a4b90c4c3ceecebb0cd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Aug 2020 16:59:53 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Aug 2020 16:59:53 -0400
M doc/src/sgml/release-13.sgml
Check for fseeko() failure in pg_dump's _tarAddFile().
commit : dd63a71ebe3937e0ec965248513fb71e45ee0ec8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Aug 2020 12:39:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 9 Aug 2020 12:39:08 -0400
Coverity pointed out, not unreasonably, that we checked fseeko's
result at every other call site but these. Failure to seek in the
temp file (note this is NOT pg_dump's output file) seems quite
unlikely, and even if it did happen the file length cross-check
further down would probably detect the problem. Still, that's a
poor excuse for not checking the result of a system call.
M src/bin/pg_dump/pg_backup_tar.c
walsnd: Don't set waiting_for_ping_response spuriously
commit : 900429d0c03668ac474770c01ba5911b15025dfb
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 8 Aug 2020 12:31:55 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Sat, 8 Aug 2020 12:31:55 -0400
Ashutosh Bapat noticed that when logical walsender needs to wait for
WAL, and it realizes that it must send a keepalive message to
walreceiver to update the sent-LSN, which *does not* request a reply
from walreceiver, it wrongly sets the flag that it's going to wait for
that reply. That means that any future would-be sender of feedback
messages ends up not sending a feedback message, because they all
believe that a reply is expected.
With built-in logical replication there's not much harm in this, because
WalReceiverMain will send a ping-back every wal_receiver_timeout/2
anyway; but with other logical replication systems (e.g. pglogical) it
can cause significant pain.
This problem was introduced in commit 41d5f8ad734, where the
request-reply flag was changed from true to false to WalSndKeepalive,
without at the same time removing the line that sets
waiting_for_ping_response.
Just removing that line would be a sufficient fix, but it seems better
to shift the responsibility of setting the flag to WalSndKeepalive
itself instead of requiring caller to do it; this is clearly less
error-prone.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Ashutosh Bapat <ashutosh.bapat@2ndquadrant.com>
Backpatch: 9.5 and up
Discussion: https://postgr.es/m/20200806225558.GA22401@alvherre.pgsql
M src/backend/replication/walsender.c
Add list of acknowledgments to release notes
commit : f1c3a41bd695c4ba97d50ffa4b2ab2a72068bd3b
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Aug 2020 20:38:55 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 7 Aug 2020 20:38:55 +0200
This contains all individuals mentioned in the commit messages during
PostgreSQL 13 development.
current through 79a3ab1e98d6b5952e29ad91e07c0e9fc777cc0b
M doc/src/sgml/release-13.sgml
Fix yet another issue with step generation in partition pruning.
commit : 79a3ab1e98d6b5952e29ad91e07c0e9fc777cc0b
author : Etsuro Fujita <efujita@postgresql.org>
date : Fri, 7 Aug 2020 14:45:01 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Fri, 7 Aug 2020 14:45:01 +0900
Commit 13838740f fixed some issues with step generation in partition
pruning, but there was yet another one: get_steps_using_prefix() assumes
that clauses in the passed-in prefix list are sorted in ascending order
of their partition key numbers, but the caller failed to ensure this for
range partitioning, which led to an assertion failure in debug builds.
Adjust the caller function to arrange the clauses in the prefix list in
the required order for range partitioning.
Back-patch to v11, like the previous commit.
Patch by me, reviewed by Amit Langote.
Discussion: https://postgr.es/m/CAPmGK16jkXiFG0YqMbU66wte-oJTfW6D1HaNvQf%3D%2B5o9%3Dm55wQ%40mail.gmail.com
M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
amcheck: Sanitize metapage's allequalimage field.
commit : be9bdab983cfc44db1d7f8c06df7d7a7cbcb8128
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 6 Aug 2020 15:25:47 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 6 Aug 2020 15:25:47 -0700
This will be helpful if it ever proves necessary to revoke an opclass's
support for deduplication.
Backpatch: 13-, where nbtree deduplication was introduced.
M contrib/amcheck/verify_nbtree.c
Fix bogus EXPLAIN output for Hash Aggregate
commit : 05dfb813245bf3b896b5317570a24a3d66e97a41
author : David Rowley <drowley@postgresql.org>
date : Fri, 7 Aug 2020 10:22:08 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 7 Aug 2020 10:22:08 +1200
9bdb300de modified the EXPLAIN output for Hash Aggregate to show details
from parallel workers. However, it neglected to consider that a given
parallel worker may not have assisted with the given Hash Aggregate. This
can occur when workers fail to start or during Parallel Append with
enable_partitionwise_join enabled when only a single worker is working on
a non-parallel aware sub-plan. It could also happen if a worker simply
wasn't fast enough to get any work done before other processes went and
finished all the work.
The bogus output came from the fact that ExplainOpenWorker() skipped
showing any details for non-initialized workers but show_hashagg_info()
did show details from the worker. This meant that the worker properties
that were shown were not properly attributed to the worker that they
belong to.
In passing, we also now don't show Hash Aggregate properties for the
leader process when it did not contribute any work to the Hash Aggregate.
This can occur either during Parallel Append when only a parallel worker
worked on a given sub plan or with parallel_leader_participation set to
off. This aims to make the behavior of Hash Aggregate's EXPLAIN output
more similar to Sort's.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20200805012105.GZ28072%40telsasoft.com
Backpatch-through: 13, where the original breakage was introduced
M src/backend/commands/explain.c
doc: clarify "state" table reference in tutorial
commit : d0aa57d0e9441b9fab5de5dbed0cb0afd4fa756d
author : Bruce Momjian <bruce@momjian.us>
date : Wed, 5 Aug 2020 17:12:10 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Wed, 5 Aug 2020 17:12:10 -0400
Reported-by: Vyacheslav Shablistyy
Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/advanced.sgml
Fix matching of sub-partitions when a partitioned plan is stale.
commit : c43a36fa8fff380072d9fc745b1e27baf1a4d3f8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 Aug 2020 15:38:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 5 Aug 2020 15:38:55 -0400
Since we no longer require AccessExclusiveLock to add a partition,
the executor may see that a partitioned table has more partitions
than the planner saw. ExecCreatePartitionPruneState's code for
matching up the partition lists in such cases was faulty, and would
misbehave if the planner had successfully pruned any partitions from
the query. (Thus, trouble would occur only if a partition addition
happens concurrently with a query that uses both static and dynamic
partition pruning.) This led to an Assert failure in debug builds,
and probably to crashes or query misbehavior in production builds.
To repair the bug, just explicitly skip zeroes in the plan's
relid_map[] list. I also made some cosmetic changes to make the code
more readable (IMO anyway). Also, convert the cross-checking Assert
to a regular test-and-elog, since it's now apparent that this logic
is more fragile than one would like.
Currently, there's no way to repeatably exercise this code, except
with manual use of a debugger to stop the backend between planning
and execution. Hence, no test case in this patch. We oughta do
something about that testability gap, but that's for another day.
Amit Langote and Tom Lane, per report from Justin Pryzby. Oversight
in commit 898e5e329; backpatch to v12 where that appeared.
Discussion: https://postgr.es/m/20200802181131.GA27754@telsasoft.com
M src/backend/executor/execPartition.c
Increase hard-wired timeout values in ecpg regression tests.
commit : 565f1690264b5773c23b86303645cf11a0893cff
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 4 Aug 2020 15:20:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 4 Aug 2020 15:20:31 -0400
A couple of test cases had connect_timeout=14, a value that seems
to have been plucked from a hat. While it's more than sufficient
for normal cases, slow/overloaded buildfarm machines can get a timeout
failure here, as per recent report from "sungazer". Increase to 180
seconds, which is in line with our typical timeouts elsewhere in
the regression tests.
Back-patch to 9.6; the code looks different in 9.5, and this doesn't
seem to be quite worth the effort to adapt to that.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sungazer&dt=2020-08-04%2007%3A12%3A22
M src/interfaces/ecpg/test/connect/test1.pgc
M src/interfaces/ecpg/test/connect/test5.pgc
M src/interfaces/ecpg/test/expected/connect-test1-minGW32.stderr
M src/interfaces/ecpg/test/expected/connect-test1.c
M src/interfaces/ecpg/test/expected/connect-test1.stderr
M src/interfaces/ecpg/test/expected/connect-test5.c
M src/interfaces/ecpg/test/expected/connect-test5.stderr
Make new SSL TAP test for channel_binding more robust
commit : 1a01595cc2ffb20ef9f2bc1a92d39728b65e3797
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Aug 2020 14:36:09 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 4 Aug 2020 14:36:09 +0900
The test would fail in an environment including a certificate file in
~/.postgresql/. bdd6e9b fixed a similar failure, and d6e612f introduced
the same problem again with a new test.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20200804.120033.31225582282178001.horikyota.ntt@gmail.com
Backpatch-through: 13
M src/test/ssl/t/002_scram.pl
doc: PG 13 relnotes: hash_mem_multiplier can restore old behav.
commit : e7a6cd5dcf24c6d4b04d036a4837c7af154c4b49
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 3 Aug 2020 17:01:42 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 3 Aug 2020 17:01:42 -0400
Document that hash_mem_multiplier can get query behavior closer to the
pre-PG 13 behavior of allowing hashing to use more memory.
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-Wzn3kwQm_pe6g2=ki+P7+ZRqH5GvFGn6SWfv_j7UUgcLdQ@mail.gmail.com
Backpatch-through: 13 only
M doc/src/sgml/release-13.sgml
Remove unnecessary "DISTINCT" in psql's queries for \dAc and \dAf.
commit : 72ca61101ad4076941f175b50cc86e6372023034
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Aug 2020 14:02:35 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Aug 2020 14:02:35 -0400
A moment's examination of these queries is sufficient to see that
they do not produce duplicate rows, unless perhaps there's
catalog corruption. Using DISTINCT anyway is inefficient and
confusing; moreover it sets a poor example for anyone who
refers to psql -E output to see how to query the catalogs.
M src/bin/psql/describe.c
Doc: fix obsolete info about allowed range of TZ offsets in timetz.
commit : 6d78f219e8afc57b6ca3765eb2dfa1b8fe095ddc
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Aug 2020 13:11:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Aug 2020 13:11:16 -0400
We've allowed UTC offsets up to +/- 15:59 since commit cd0ff9c0f, but
that commit forgot to fix the documentation about timetz.
Per bug #16571 from osdba.
Discussion: https://postgr.es/m/16571-eb7501598de78c8a@postgresql.org
M doc/src/sgml/datatype.sgml
Fix behavior of ecpg's "EXEC SQL elif name".
commit : 44cd434ec4a70d2dfbc460492fc0574d08440250
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Aug 2020 09:46:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 3 Aug 2020 09:46:12 -0400
This ought to work much like C's "#elif defined(name)"; but the code
implemented it in a way equivalent to endif followed by ifdef, so that
it didn't matter whether any previous branch of the IF construct had
succeeded. Fix that; add some test cases covering elif and nested IFs;
and improve the documentation, which also seemed a bit confused.
AFAICS the code has been like this since the feature was added in 1999
(commit b57b0e044). So while it's surely wrong, there might be code
out there relying on the current behavior. Hence, don't back-patch
into stable branches. It seems all right to fix it in v13 though.
Per report from Ashutosh Sharma. Reviewed by Ashutosh Sharma and
Michael Meskes.
Discussion: https://postgr.es/m/CAE9k0P=dQk9X0cU2tN49S7a9tv733-e1pVdpB1P-pWJ5PdTktg@mail.gmail.com
M doc/src/sgml/ecpg.sgml
M src/interfaces/ecpg/preproc/pgc.l
M src/interfaces/ecpg/test/expected/preproc-define.c
M src/interfaces/ecpg/test/expected/preproc-define.stderr
M src/interfaces/ecpg/test/preproc/define.pgc
Fix rare failure in LDAP tests.
commit : f5293fb09e7346bb663f2f5c63081e8aabe61a8e
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 3 Aug 2020 12:39:15 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 3 Aug 2020 12:39:15 +1200
Instead of writing a query to psql's stdin, use -c. This avoids a
failure where psql exits before we write, seen a few times on the build
farm. Thanks to Tom Lane for the suggestion.
Back-patch to 11, where the LDAP tests arrived.
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CA%2BhUKGLFmW%2BHQYPeKiwSp5sdFFHtFViCpw4Mh6yAgEx74r5-Cw%40mail.gmail.com
M src/test/ldap/t/001_auth.pl
Fix minor issues in psql's new \dAc and related commands.
commit : 719304a3043d9f60247df371f55236058a7f3caa
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 2 Aug 2020 17:00:26 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 2 Aug 2020 17:00:26 -0400
The type-name pattern in \dAc and \dAf was matched only to the actual
pg_type.typname string, which is fairly user-unfriendly in cases where
that is not what's shown to the user by format_type (compare "_int4"
and "integer[]"). Make this code match what \dT does, i.e. match the
pattern against either typname or format_type() output. Also fix its
broken handling of schema-name restrictions. (IOW, make these
processSQLNamePattern calls match \dT's.) While here, adjust
whitespace to make the query a little prettier in -E output, too.
Also improve some inaccuracies and shaky grammar in the related
documentation.
Noted while working on a patch for intarray's opclasses; I wondered
why I couldn't get a match to "integer*" for the input type name.
M doc/src/sgml/indices.sgml
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/describe.c
Use int64 instead of long in incremental sort code
commit : 22c105595fc736ae94ce3b806da16b0bd8e94fb8
author : David Rowley <drowley@postgresql.org>
date : Sun, 2 Aug 2020 14:23:57 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sun, 2 Aug 2020 14:23:57 +1200
Windows 64bit has 4-byte long values which is not suitable for tracking
disk space usage in the incremental sort code. Let's just make all these
fields int64s.
Author: James Coleman
Discussion: https://postgr.es/m/CAApHDvpky%2BUhof8mryPf5i%3D6e6fib2dxHqBrhp0Qhu0NeBhLJw%40mail.gmail.com
Backpatch-through: 13, where the incremental sort code was added
M src/backend/commands/explain.c
M src/include/nodes/execnodes.h
M src/include/utils/tuplesort.h
Restore lost amcheck TOAST test coverage.
commit : 725b43b9c3b3c25c60ac717c41047ad0dffb1312
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 31 Jul 2020 15:34:26 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 31 Jul 2020 15:34:26 -0700
Commit eba77534 fixed an amcheck false positive bug involving
inconsistencies in TOAST input state between table and index. A test
case was added that verified that such an inconsistency didn't result in
a spurious corruption related error.
Test coverage from the test was accidentally lost by commit 501e41dd,
which propagated ALTER TABLE ... SET STORAGE attstorage state to
indexes. This broke the test because the test specifically relied on
attstorage not being propagated. This artificially forced there to be
index tuples whose datums were equivalent to the datums in the heap
without the datums actually being bitwise equal.
Fix this by updating pg_attribute directly instead. Commit 501e41dd
made similar changes to a test_decoding TOAST-related test case which
made the same assumption, but overlooked the amcheck test case.
Backpatch: 11-, just like commit eba77534 (and commit 501e41dd).
M contrib/amcheck/expected/check_btree.out
M contrib/amcheck/sql/check_btree.sql
Fix oversight in ALTER TYPE: typmodin/typmodout must propagate to arrays.
commit : 5c439f189bf4bdbb0ff75b2043ca713d76019528
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jul 2020 17:11:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jul 2020 17:11:28 -0400
If a base type supports typmods, its array type does too, with the
same interpretation. Hence changes in pg_type.typmodin/typmodout
must be propagated to the array type.
While here, improve AlterTypeRecurse to not recurse to domains if
there is nothing we'd need to change.
Oversight in fe30e7ebf. Back-patch to v13 where that came in.
M src/backend/commands/typecmds.c
M src/test/regress/expected/create_type.out
M src/test/regress/sql/create_type.sql
Fix recently-introduced performance problem in ts_headline().
commit : 11dce63d6c3f676a9f1829eca96f085b6d935af0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jul 2020 11:43:12 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 31 Jul 2020 11:43:12 -0400
The new hlCover() algorithm that I introduced in commit c9b0c678d
turns out to potentially take O(N^2) or worse time on long documents,
if there are many occurrences of individual query words but few or no
substrings that actually satisfy the query. (One way to hit this
behavior is with a "common_word & rare_word" type of query.) This
seems unavoidable given the original goal of checking every substring
of the document, so we have to back off that idea. Fortunately, it
seems unlikely that anyone would really want headlines spanning all of
a long document, so we can avoid the worse-than-linear behavior by
imposing a maximum length of substring that we'll consider.
For now, just hard-wire that maximum length as a multiple of max_words
times max_fragments. Perhaps at some point somebody will argue for
exposing it as a ts_headline parameter, but I'm hesitant to make such
a feature addition in a back-patched bug fix.
I also noted that the hlFirstIndex() function I'd added in that
commit was unnecessarily stupid: it really only needs to check whether
a HeadlineWordEntry's item pointer is null or not. This wouldn't make
all that much difference in typical cases with queries having just
a few terms, but a cycle shaved is a cycle earned.
In addition, add a CHECK_FOR_INTERRUPTS call in TS_execute_recurse.
This ensures that hlCover's loop is cancellable if it manages to take
a long time, and it may protect some other TS_execute callers as well.
Back-patch to 9.6 as the previous commit was. I also chose to add the
CHECK_FOR_INTERRUPTS call to 9.5. The old hlCover() algorithm seems
to avoid the O(N^2) behavior, at least on the test case I tried, but
nonetheless it's not very quick on a long document.
Per report from Stephen Frost.
Discussion: https://postgr.es/m/20200724160535.GW12375@tamriel.snowman.net
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/tsvector_op.c
Doc: fix high availability solutions comparison.
commit : 0d9b64fe8d2d98f8f074334f86aaaedfb2b5a1e1
author : Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 31 Jul 2020 07:46:25 +0900
committer: Tatsuo Ishii <ishii@postgresql.org>
date : Fri, 31 Jul 2020 07:46:25 +0900
In "High Availability, Load Balancing, and Replication" chapter,
certain descriptions of Pgpool-II were not correct at this point. It
does not need conflict resolution. Also "Multiple-Server Parallel
Query Execution" is not supported anymore.
Discussion: https://postgr.es/m/20200726.230128.53842489850344110.t-ishii%40sraoss.co.jp
Author: Tatsuo Ishii
Reviewed-by: Bruce Momjian
Backpatch-through: 9.5
M doc/src/sgml/high-availability.sgml
Use pg_bitutils for HyperLogLog.
commit : 07cbcdd00c2465a844513f14b7a96232013d3129
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 30 Jul 2020 08:44:58 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 30 Jul 2020 08:44:58 -0700
Using pg_leftmost_one_post32() yields substantial performance benefits.
Backpatching to version 13 because HLL is used for HashAgg
improvements in 9878b643, which was also backpatched to 13.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzkGvDKVDo+0YvfvZ+1CE=iCi88DCOGFF3i1hTGGaxcKPw@mail.gmail.com
Backpatch-through: 13
M src/backend/lib/hyperloglog.c
doc: Mention index references in pg_inherits
commit : e710f3b8e7da43de2ad8113ce7c7933928092656
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 30 Jul 2020 15:48:52 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 30 Jul 2020 15:48:52 +0900
Partitioned indexes are also registered in pg_inherits, but the
description of this catalog did not reflect that.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/87k0ynj35y.fsf@wibble.ilmari.org
Backpatch-through: 11
M doc/src/sgml/catalogs.sgml
Add hash_mem_multiplier GUC.
commit : 78530c8e7a5abe0b646b0b46527f8799f831e1e1
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 29 Jul 2020 14:14:57 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 29 Jul 2020 14:14:57 -0700
Add a GUC that acts as a multiplier on work_mem. It gets applied when
sizing executor node hash tables that were previously size constrained
using work_mem alone.
The new GUC can be used to preferentially give hash-based nodes more
memory than the generic work_mem limit. It is intended to enable admin
tuning of the executor's memory usage. Overall system throughput and
system responsiveness can be improved by giving hash-based executor
nodes more memory (especially over sort-based alternatives, which are
often much less sensitive to being memory constrained).
The default value for hash_mem_multiplier is 1.0, which is also the
minimum valid value. This means that hash-based nodes continue to apply
work_mem in the traditional way by default.
hash_mem_multiplier is generally useful. However, it is being added now
due to concerns about hash aggregate performance stability for users
that upgrade to Postgres 13 (which added disk-based hash aggregation in
commit 1f39bce0). While the old hash aggregate behavior risked
out-of-memory errors, it is nevertheless likely that many users actually
benefited. Hash agg's previous indifference to work_mem during query
execution was not just faster; it also accidentally made aggregation
resilient to grouping estimate problems (at least in cases where this
didn't create destabilizing memory pressure).
hash_mem_multiplier can provide a certain kind of continuity with the
behavior of Postgres 12 hash aggregates in cases where the planner
incorrectly estimates that all groups (plus related allocations) will
fit in work_mem/hash_mem. This seems necessary because hash-based
aggregation is usually much slower when only a small fraction of all
groups can fit. Even when it isn't possible to totally avoid hash
aggregates that spill, giving hash aggregation more memory will reliably
improve performance (the same cannot be said for external sort
operations, which appear to be almost unaffected by memory availability
provided it's at least possible to get a single merge pass).
The PostgreSQL 13 release notes should advise users that increasing
hash_mem_multiplier can help with performance regressions associated
with hash aggregation. That can be taken care of by a later commit.
Author: Peter Geoghegan
Reviewed-By: Álvaro Herrera, Jeff Davis
Discussion: https://postgr.es/m/20200625203629.7m6yvut7eqblgmfo@alap3.anarazel.de
Discussion: https://postgr.es/m/CAH2-WzmD%2Bi1pG6rc1%2BCjc4V6EaFJ_qSuKCCHVnH%3DoruqD-zqow%40mail.gmail.com
Backpatch: 13-, where disk-based hash aggregation was introduced.
M doc/src/sgml/config.sgml
M doc/src/sgml/ref/postgres-ref.sgml
M doc/src/sgml/runtime.sgml
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeHash.c
M src/backend/executor/nodeHashjoin.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/optimizer/plan/subselect.c
M src/backend/optimizer/prep/prepunion.c
M src/backend/optimizer/util/pathnode.c
M src/backend/utils/adt/ri_triggers.c
M src/backend/utils/init/globals.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/executor/hashjoin.h
M src/include/executor/nodeHash.h
M src/include/miscadmin.h
HashAgg: use better cardinality estimate for recursive spilling.
commit : 3a232a3183d517743acf232794fadc07f0944220
author : Jeff Davis <jdavis@postgresql.org>
date : Tue, 28 Jul 2020 23:15:47 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Tue, 28 Jul 2020 23:15:47 -0700
Use HyperLogLog to estimate the group cardinality in a spilled
partition. This estimate is used to choose the number of partitions if
we recurse.
The previous behavior was to use the number of tuples in a spilled
partition as the estimate for the number of groups, which lead to
overpartitioning. That could cause the number of batches to be much
higher than expected (with each batch being very small), which made it
harder to interpret EXPLAIN ANALYZE results.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/a856635f9284bc36f7a77d02f47bbb6aaf7b59b3.camel@j-davis.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/include/executor/nodeAgg.h
Rename another "hash_mem" local variable.
commit : cdd7bd695bed552936e86b70ff1d234360bc5bea
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 28 Jul 2020 17:59:14 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 28 Jul 2020 17:59:14 -0700
Missed by my commit 564ce621.
Backpatch: 13-, where disk-based hash aggregation was introduced.
M src/backend/executor/nodeAgg.c
Correct obsolete UNION hash aggs comment.
commit : b6c15e71f33fe9aa7f38cc7bde26d420fbaaef5b
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 28 Jul 2020 17:14:06 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 28 Jul 2020 17:14:06 -0700
Oversight in commit 1f39bce0, which added disk-based hash aggregation.
Backpatch: 13-, where disk-based hash aggregation was introduced.
M src/backend/optimizer/prep/prepunion.c
Doc: Remove obsolete CREATE AGGREGATE note.
commit : e362f469c50f6e671285640cc2087345ab55a9b2
author : Peter Geoghegan <pg@bowt.ie>
date : Tue, 28 Jul 2020 16:58:59 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Tue, 28 Jul 2020 16:58:59 -0700
The planner is in fact willing to use hash aggregation when work_mem is
not set high enough for everything to fit in memory. This has been the
case since commit 1f39bce0, which added disk-based hash aggregation.
There are a few remaining cases in which hash aggregation is avoided as
a matter of policy when the planner surmises that spilling will be
necessary. For example, callers of choose_hashed_setop() still
conservatively avoid hash aggregation when spilling is anticipated.
That doesn't seem like a good enough reason to mention hash aggregation
in this context.
Backpatch: 13-, where disk-based hash aggregation was introduced.
M doc/src/sgml/ref/create_aggregate.sgml
Make EXPLAIN ANALYZE of HashAgg more similar to Hash Join
commit : a57c837e5cdf601d6ec05e5e10a40d01f1d2b84e
author : David Rowley <drowley@postgresql.org>
date : Wed, 29 Jul 2020 11:43:11 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 29 Jul 2020 11:43:11 +1200
There were various unnecessary differences between Hash Agg's EXPLAIN
ANALYZE output and Hash Join's. Here we modify the Hash Agg output so
that it's better aligned to Hash Join's.
The following changes have been made:
1. Start batches counter at 1 instead of 0.
2. Always display the "Batches" property, even when we didn't spill to
disk.
3. Use the text "Batches" instead of "HashAgg Batches" for text format.
4. Use the text "Memory Usage" instead of "Peak Memory Usage" for text
format.
5. Include "Batches" before "Memory Usage" in both text and non-text
formats.
In passing also modify the "Planned Partitions" property so that we show
it regardless of if the value is 0 or not for non-text EXPLAIN formats.
This was pointed out by Justin Pryzby and probably should have been part
of 40efbf870.
Reviewed-by: Justin Pryzby, Jeff Davis
Discussion: https://postgr.es/m/CAApHDvrshRnA6C0VFnu7Fb9TVvgGo80PUMm5+2DiaS1gEkPvtw@mail.gmail.com
Backpatch-through: 13, where HashAgg batching was introduced
M src/backend/commands/explain.c
M src/backend/executor/nodeAgg.c
Doc: Improve documentation for pg_jit_available()
commit : dc6f2fb4353508af27dde44bdf5cb14749ae73a1
author : David Rowley <drowley@postgresql.org>
date : Tue, 28 Jul 2020 22:52:43 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 28 Jul 2020 22:52:43 +1200
Per complaint from Scott Ribe. Based on wording suggestion from Tom Lane.
Discussion: https://postgr.es/m/1956E806-1468-4417-9A9D-235AE1D5FE1A@elevated-dev.com
Backpatch-through: 11, where pg_jit_available() was added
M doc/src/sgml/func.sgml
doc: Mention the rename of wal_keep_segments GUC in release note.
commit : 128fd0a65ae54d97896cb6409fbc56d5da6319f1
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 28 Jul 2020 11:23:02 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 28 Jul 2020 11:23:02 +0900
Commit f5dff45962 renamed wal_keep_segments to wal_keep_size.
This commit adds the mention to this change in the release note.
Author: Fujii Masao
Reviewed-by: David Steele
Discussion: https://postgr.es/m/dc4768f2-1eff-d2fc-35ba-6b2985b7cb6c@oss.nttdata.com
M doc/src/sgml/release-13.sgml
Fix some issues with step generation in partition pruning.
commit : cebe10a5f307635e187f424d0dce700666ae58fd
author : Etsuro Fujita <efujita@postgresql.org>
date : Tue, 28 Jul 2020 11:00:00 +0900
committer: Etsuro Fujita <efujita@postgresql.org>
date : Tue, 28 Jul 2020 11:00:00 +0900
In the case of range partitioning, get_steps_using_prefix() assumes that
the passed-in prefix list contains at least one clause for each of the
partition keys earlier than one specified in the passed-in
step_lastkeyno, but the caller (ie, gen_prune_steps_from_opexps())
didn't take it into account, which led to a server crash or incorrect
results when the list contained no clauses for such partition keys, as
reported in bug #16500 and #16501 from Kobayashi Hisanori. Update the
caller to call that function only when the list created there contains
at least one clause for each of the earlier partition keys in the case
of range partitioning.
While at it, fix some other issues:
* The list to pass to get_steps_using_prefix() is allowed to contain
multiple clauses for the same partition key, as described in the
comment for that function, but that function actually assumed that the
list contained just a single clause for each of middle partition keys,
which led to an assertion failure when the list contained multiple
clauses for such partition keys. Update that function to match the
comment.
* In the case of hash partitioning, partition keys are allowed to be
NULL, in which case the list to pass to get_steps_using_prefix()
contains no clauses for NULL partition keys, but that function treats
that case as like the case of range partitioning, which led to the
assertion failure. Update the assertion test to take into account
NULL partition keys in the case of hash partitioning.
* Fix a typo in a comment in get_steps_using_prefix_recurse().
* gen_partprune_steps() failed to detect self-contradiction from
strict-qual clauses and an IS NULL clause for the same partition key
in some cases, producing incorrect partition-pruning steps, which led
to incorrect results of partition pruning, but didn't cause any
user-visible problems fortunately, as the self-contradiction is
detected later in the query planning. Update that function to detect
the self-contradiction.
Per bug #16500 and #16501 from Kobayashi Hisanori. Patch by me, initial
diagnosis for the reported issue and review by Dmitry Dolgov.
Back-patch to v11, where partition pruning was introduced.
Discussion: https://postgr.es/m/16500-d1613f2a78e1e090%40postgresql.org
Discussion: https://postgr.es/m/16501-5234a9a0394f6754%40postgresql.org
M src/backend/partitioning/partprune.c
M src/test/regress/expected/partition_prune.out
M src/test/regress/sql/partition_prune.sql
Remove hashagg_avoid_disk_plan GUC.
commit : 5a6cc6ffa914775e0184207686c2216998126549
author : Peter Geoghegan <pg@bowt.ie>
date : Mon, 27 Jul 2020 17:53:17 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Mon, 27 Jul 2020 17:53:17 -0700
Note: This GUC was originally named enable_hashagg_disk when it appeared
in commit 1f39bce0, which added disk-based hash aggregation. It was
subsequently renamed in commit 92c58fd9.
Author: Peter Geoghegan
Reviewed-By: Jeff Davis, Álvaro Herrera
Discussion: https://postgr.es/m/9d9d1e1252a52ea1bad84ea40dbebfd54e672a0f.camel%40j-davis.com
Backpatch: 13-, where disk-based hash aggregation was introduced.
M doc/src/sgml/config.sgml
M doc/src/sgml/release-13.sgml
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc.c
M src/include/optimizer/cost.h
Fix corner case with 16kB-long decompression in pgcrypto, take 2
commit : 0caf1fc6e86a8985ef8b881a4dbb3488381ff976
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jul 2020 15:58:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jul 2020 15:58:54 +0900
A compressed stream may end with an empty packet. In this case
decompression finishes before reading the empty packet and the
remaining stream packet causes a failure in reading the following
data. This commit makes sure to consume such extra data, avoiding a
failure when decompression the data. This corner case was reproducible
easily with a data length of 16kB, and existed since e94dd6a. A cheap
regression test is added to cover this case based on a random,
incompressible string.
The first attempt of this patch has allowed to find an older failure
within the compression logic of pgcrypto, fixed by b9b6105. This
involved SLES 15 with z390 where a custom flavor of libz gets used.
Bonus thanks to Mark Wong for providing access to the specific
environment.
Reported-by: Frank Gagnepain
Author: Kyotaro Horiguchi, Michael Paquier
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/16476-692ef7b84e5fb893@postgresql.org
Backpatch-through: 9.5
M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql
Fix handling of structure for bytea data type in ECPG
commit : ed4a9dc9cf69eee06f4e655ef5d175c5efa87dd7
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jul 2020 10:29:08 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 27 Jul 2020 10:29:08 +0900
Some code paths dedicated to bytea used the structure for varchar. This
did not lead to any actual bugs, as bytea and varchar have the same
definition, but it could become a trap if one of these definitions
changes for a new feature or a bug fix.
Issue introduced by 050710b.
Author: Shenhao Wang
Reviewed-by: Vignesh C, Michael Paquier
Discussion: https://postgr.es/m/07ac7dee1efc44f99d7f53a074420177@G08CNEXMBPEKD06.g08.fujitsu.local
Backpatch-through: 12
M src/interfaces/ecpg/ecpglib/data.c
M src/interfaces/ecpg/ecpglib/descriptor.c
M src/interfaces/ecpg/ecpglib/execute.c
Fix LookupTupleHashEntryHash() pipeline-stall issue.
commit : 7f5f2249b27a46a4d91d6be5aff188ca67719fa7
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 26 Jul 2020 14:55:52 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 26 Jul 2020 14:55:52 -0700
Refactor hash lookups in nodeAgg.c to improve performance.
Author: Andres Freund and Jeff Davis
Discussion: https://postgr.es/m/20200612213715.op4ye4q7gktqvpuo%40alap3.anarazel.de
Backpatch-through: 13
M src/backend/executor/execGrouping.c
M src/backend/executor/nodeAgg.c
M src/backend/executor/nodeRecursiveunion.c
M src/backend/executor/nodeSetOp.c
M src/backend/executor/nodeSubplan.c
M src/include/executor/executor.h
Tweak behavior of pg_stat_activity.leader_pid
commit : 21b0055359f036e3ba22402d14536431dd39242e
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 26 Jul 2020 16:32:20 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 26 Jul 2020 16:32:20 +0900
The initial implementation of leader_pid in pg_stat_activity added by
b025f32 took the approach to strictly print what a PGPROC entry
includes. In short, if a backend has been involved in parallel query at
least once, leader_pid would remain set as long as the backend is alive.
For a parallel group leader, this means that the field would always be
set after it participated at least once in parallel query, and after
more discussions this could be confusing if using for example a
connection pooler.
This commit changes the data printed so as leader_pid becomes always
NULL for a parallel group leader, showing up a non-NULL value only for
the parallel workers, and actually as long as a parallel query is
running as workers are shut down once the query has completed.
This does not change the definition of any catalog, so no catalog bump
is needed. Per discussion with Justin Pryzby, Álvaro Herrera, Julien
Rouhaud and me.
Discussion: https://postgr.es/m/20200721035145.GB17300@paquier.xyz
Backpatch-through: 13
M doc/src/sgml/monitoring.sgml
M src/backend/utils/adt/pgstatfuncs.c
Fix buffer usage stats for nodes above Gather Merge.
commit : b15367ae39402eb4eb8736f9c38c607995c82bb2
author : Amit Kapila <akapila@postgresql.org>
date : Sat, 25 Jul 2020 10:31:19 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Sat, 25 Jul 2020 10:31:19 +0530
Commit 85c9d347 addressed a similar problem for Gather and Gather
Merge nodes but forgot to account for nodes above parallel nodes. This
still works for nodes above Gather node because we shut down the workers
for Gather node as soon as there are no more tuples. We can do a similar
thing for Gather Merge as well but it seems better to account for stats
during nodes shutdown after completing the execution.
Reported-by: Stéphane Lorek, Jehan-Guillaume de Rorthais
Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Reviewed-by: Amit Kapila
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/20200718160206.584532a2@firost
M src/backend/executor/execProcnode.c
Replace TS_execute's TS_EXEC_CALC_NOT flag with TS_EXEC_SKIP_NOT.
commit : 70eca6a9a6df679a86f30442194cc6b858b82000
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 24 Jul 2020 15:43:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 24 Jul 2020 15:43:56 -0400
It's fairly silly that ignoring NOT subexpressions is TS_execute's
default behavior. It's wrong on its face and it encourages errors
of omission. Moreover, the only two remaining callers that aren't
specifying CALC_NOT are in ts_headline calculations, and it's very
arguable that those are bugs: if you've specified "!foo" in your
query, why would you want to get a headline that includes "foo"?
Hence, rip that out and change the default behavior to be to calculate
NOT accurately. As a concession to the slim chance that there is still
somebody somewhere who needs the incorrect behavior, provide a new
SKIP_NOT flag to explicitly request that.
Back-patch into v13, mainly because it seems better to change this
at the same time as the previous commit's rejiggering of TS_execute
related APIs. Any outside callers affected by this change are
probably also affected by that one.
Discussion: https://postgr.es/m/CALT9ZEE-aLotzBg-pOp2GFTesGWVYzXA3=mZKzRDa_OKnLF7Mg@mail.gmail.com
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector_op.c
M src/include/tsearch/ts_utils.h
Fix assorted bugs by changing TS_execute's callback API to ternary logic.
commit : 92fe6895d66da93a3c920089cfbbe4eb2db2145e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 24 Jul 2020 15:26:51 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 24 Jul 2020 15:26:51 -0400
Text search sometimes failed to find valid matches, for instance
'!crew:A'::tsquery might fail to locate 'crew:1B'::tsvector during
an index search. The root of the issue is that TS_execute's callback
functions were not changed to use ternary (yes/no/maybe) reporting
when we made the search logic itself do so. It's somewhat annoying
to break that API, but on the other hand we now see that any code
using plain boolean logic is almost certainly broken since the
addition of phrase search. There seem to be very few outside callers
of this code anyway, so we'll just break them intentionally to get
them to adapt.
This allows removal of tsginidx.c's private re-implementation of
TS_execute, since that's now entirely duplicative. It's also no
longer necessary to avoid use of CALC_NOT in tsgistidx.c, since
the underlying callbacks can now do something reasonable.
Back-patch into v13. We can't change this in stable branches,
but it seems not quite too late to fix it in v13.
Tom Lane and Pavel Borisov
Discussion: https://postgr.es/m/CALT9ZEE-aLotzBg-pOp2GFTesGWVYzXA3=mZKzRDa_OKnLF7Mg@mail.gmail.com
M src/backend/tsearch/wparser_def.c
M src/backend/utils/adt/tsginidx.c
M src/backend/utils/adt/tsgistidx.c
M src/backend/utils/adt/tsrank.c
M src/backend/utils/adt/tsvector_op.c
M src/include/tsearch/ts_utils.h
M src/test/regress/expected/tsearch.out
M src/test/regress/expected/tstypes.out
M src/test/regress/sql/tsearch.sql
M src/test/regress/sql/tstypes.sql
Fix ancient violation of zlib's API spec.
commit : 7dab4569d19341720316d139141f1c867ef438bf
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jul 2020 17:19:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 23 Jul 2020 17:19:37 -0400
contrib/pgcrypto mishandled the case where deflate() does not consume
all of the offered input on the first try. It reset the next_in pointer
to the start of the input instead of leaving it alone, causing the wrong
data to be fed to the next deflate() call.
This has been broken since pgcrypto was committed. The reason for the
lack of complaints seems to be that it's fairly hard to get stock zlib
to not consume all the input, so long as the output buffer is big enough
(which it normally would be in pgcrypto's usage; AFAICT the input is
always going to be packetized into packets no larger than ZIP_OUT_BUF).
However, IBM's zlibNX implementation for AIX evidently will do it
in some cases.
I did not add a test case for this, because I couldn't find one that
would fail with stock zlib. When we put back the test case for
bug #16476, that will cover the zlibNX situation well enough.
While here, write deflate()'s second argument as Z_NO_FLUSH per its
API spec, instead of hard-wiring the value zero.
Per buildfarm results and subsequent investigation.
Discussion: https://postgr.es/m/16476-692ef7b84e5fb893@postgresql.org
M contrib/pgcrypto/pgp-compress.c
doc: Document that ssl_ciphers does not affect TLS 1.3
commit : dbd03482c626e7c5bb92dde983c4b9de6f623253
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Jul 2020 17:13:00 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 23 Jul 2020 17:13:00 +0200
TLS 1.3 uses a different way of specifying ciphers and a different
OpenSSL API. PostgreSQL currently does not support setting those
ciphers. For now, just document this. In the future, support for
this might be added somehow.
Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
M doc/src/sgml/config.sgml
Fix error message.
commit : 6b366190d54a2cfc57782e11c62f05899e17b6ae
author : Thomas Munro <tmunro@postgresql.org>
date : Thu, 23 Jul 2020 21:10:49 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Thu, 23 Jul 2020 21:10:49 +1200
Remove extra space. Back-patch to all releases, like commit 7897e3bb.
Author: Lu, Chenyang <lucy.fnst@cn.fujitsu.com>
Discussion: https://postgr.es/m/795d03c6129844d3803e7eea48f5af0d%40G08CNEXMBPEKD04.g08.fujitsu.local
M src/backend/storage/file/buffile.c
Revert "Fix corner case with PGP decompression in pgcrypto"
commit : 9f5162ef43c52396c4365b9cabfdffb4e4bf716d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 23 Jul 2020 08:29:14 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 23 Jul 2020 08:29:14 +0900
This reverts commit 9e10898, after finding out that buildfarm members
running SLES 15 on z390 complain on the compression and decompression
logic of the new test: pipistrelles, barbthroat and steamerduck.
Those hosts are visibly using hardware-specific changes to improve zlib
performance, requiring more investigation.
Thanks to Tom Lane for the discussion.
Discussion: https://postgr.es/m/20200722093749.GA2564@paquier.xyz
Backpatch-through: 9.5
M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql
Fix corner case with PGP decompression in pgcrypto
commit : 35e142202b14fa12f8edbd7e58b33c39d3c03c62
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Jul 2020 14:52:36 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 22 Jul 2020 14:52:36 +0900
A compressed stream may end with an empty packet, and PGP decompression
finished before reading this empty packet in the remaining stream. This
caused a failure in pgcrypto, handling this case as corrupted data.
This commit makes sure to consume such extra data, avoiding a failure
when decompression the entire stream. This corner case was reproducible
with a data length of 16kB, and existed since its introduction in
e94dd6a. A cheap regression test is added to cover this case.
Thanks to Jeff Janes for the extra investigation.
Reported-by: Frank Gagnepain
Author: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/16476-692ef7b84e5fb893@postgresql.org
Backpatch-through: 9.5
M contrib/pgcrypto/expected/pgp-compression.out
M contrib/pgcrypto/pgp-compress.c
M contrib/pgcrypto/sql/pgp-compression.sql
neqjoinsel must now pass through collation to eqjoinsel.
commit : cc4dd2a7af13b4759cd76074a932c8cf24e32bb2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Jul 2020 19:40:44 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Jul 2020 19:40:44 -0400
Since commit 044c99bc5, eqjoinsel passes the passed-in collation
to any operators it invokes. However, neqjoinsel failed to pass
on whatever collation it got, so that if we invoked a
collation-dependent operator via that code path, we'd get "could not
determine which collation to use for string comparison" or the like.
Per report from Justin Pryzby. Back-patch to v12, like the previous
commit.
Discussion: https://postgr.es/m/20200721191606.GL5748@telsasoft.com
M src/backend/utils/adt/selfuncs.c
Minor glossary tweaks
commit : ac25e7b039d5eacb8eb8fcc856a7ba4f14136b50
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jul 2020 13:08:16 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 21 Jul 2020 13:08:16 -0400
Add "(process)" qualifier to two terms, remove self-reference in one
term.
Author: Jürgen Purtz <juergen@purtz.de>
Discussion: https://postgr.es/m/95f90a5d-7692-701d-2c0c-0c88eb5cea7d@purtz.de
M doc/src/sgml/glossary.sgml
Assert that we don't insert nulls into attnotnull catalog columns.
commit : bca409e5b160f81ccd980bef2aeb32f8b731b0fd
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Jul 2020 12:38:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 21 Jul 2020 12:38:08 -0400
The executor checks for this error, and so does the bootstrap catalog
loader, but we never checked for it in retail catalog manipulations.
The folly of that has now been exposed, so let's add assertions
checking it. Checking in CatalogTupleInsert[WithInfo] and
CatalogTupleUpdate[WithInfo] should be enough to cover this.
Back-patch to v10; the aforesaid functions didn't exist before that,
and it didn't seem worth adapting the patch to the oldest branches.
But given the risk of JIT crashes, I think we certainly need this
as far back as v11.
Pre-v13, we have to explicitly exclude pg_subscription.subslotname
and pg_subscription_rel.srsublsn from the checks, since they are
mismarked. (Even if we change our mind about applying BKI_FORCE_NULL
in the branch tips, it doesn't seem wise to have assertions that
would fire in existing databases.)
Discussion: https://postgr.es/m/298837.1595196283@sss.pgh.pa.us
M doc/src/sgml/bki.sgml
M src/backend/catalog/indexing.c
Correctly mark pg_subscription_rel.srsublsn as nullable.
commit : e5372b48b94f17dcce5d8f3b26d55b0f182e4306
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jul 2020 14:55:56 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jul 2020 14:55:56 -0400
The code has always set this column to NULL when it's not valid,
but the catalog header's description failed to reflect that,
as did the SGML docs, as did some of the code. To prevent future
coding errors of the same ilk, let's hide the field from C code
as though it were variable-length (which, in a sense, it is).
As with commit 72eab84a5, we can only fix this cleanly in HEAD
and v13; the problem extends further back but we'll need some
klugery in the released branches.
Discussion: https://postgr.es/m/367660.1595202498@sss.pgh.pa.us
M doc/src/sgml/catalogs.sgml
M src/backend/catalog/pg_subscription.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription_rel.h
Fix construction of updated-columns bitmap in logical replication.
commit : 2f1f189cf8806a02987dfa1759257b154162fac2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jul 2020 13:40:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 20 Jul 2020 13:40:16 -0400
Commit b9c130a1f failed to apply the publisher-to-subscriber column
mapping while checking which columns were updated. Perhaps less
significantly, it didn't exclude dropped columns either. This could
result in an incorrect updated-columns bitmap and thus wrong decisions
about whether to fire column-specific triggers on the subscriber while
applying updates. In HEAD (since commit 9de77b545), it could also
result in accesses off the end of the colstatus array, as detected by
buildfarm member skink. Fix the logic, and adjust 003_constraints.pl
so that the problem is exposed in unpatched code.
In HEAD, also add some assertions to check that we don't access off
the ends of these newly variable-sized arrays.
Back-patch to v10, as b9c130a1f was.
Discussion: https://postgr.es/m/CAH2-Wz=79hKQ4++c5A060RYbjTHgiYTHz=fw6mptCtgghH2gJA@mail.gmail.com
M src/backend/replication/logical/worker.c
M src/test/subscription/t/003_constraints.pl
Rename wal_keep_segments to wal_keep_size.
commit : f5dff45962ec0a0daad443e45811d6c426be1237
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 20 Jul 2020 13:30:18 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 20 Jul 2020 13:30:18 +0900
max_slot_wal_keep_size that was added in v13 and wal_keep_segments are
the GUC parameters to specify how much WAL files to retain for
the standby servers. While max_slot_wal_keep_size accepts the number of
bytes of WAL files, wal_keep_segments accepts the number of WAL files.
This difference of setting units between those similar parameters could
be confusing to users.
To alleviate this situation, this commit renames wal_keep_segments to
wal_keep_size, and make users specify the WAL size in it instead of
the number of WAL files.
There was also the idea to rename max_slot_wal_keep_size to
max_slot_wal_keep_segments, in the discussion. But we have been moving
away from measuring in segments, for example, checkpoint_segments was
replaced by max_wal_size. So we concluded to rename wal_keep_segments
to wal_keep_size.
Back-patch to v13 where max_slot_wal_keep_size was added.
Author: Fujii Masao
Reviewed-by: Álvaro Herrera, Kyotaro Horiguchi, David Steele
Discussion: https://postgr.es/m/574b4ea3-e0f9-b175-ead2-ebea7faea855@oss.nttdata.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/wal.sgml
M src/backend/access/transam/xlog.c
M src/backend/replication/slotfuncs.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/bin/pg_rewind/t/RewindTest.pm
M src/include/access/xlog.h
M src/test/recovery/t/019_replslot_limit.pl
Fix minor typo in nodeIncrementalSort.c.
commit : 4a1ae21750cbf23d8317d565c55ac7bce46bf0f6
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 20 Jul 2020 07:54:04 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 20 Jul 2020 07:54:04 +0530
Author: Vignesh C
Reviewed-by: James Coleman
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CALDaNm0WjZqRvdeL59ZfYH0o4mLbKQ23jm-bnjXcFzgpANx55g@mail.gmail.com
M src/backend/executor/nodeIncrementalSort.c
Correctly mark pg_subscription.subslotname as nullable.
commit : 914d2383ae91918b359311a90d23c0d5862781ac
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Jul 2020 12:37:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 19 Jul 2020 12:37:23 -0400
Due to the layout of this catalog, subslotname has to be explicitly
marked BKI_FORCE_NULL, else initdb will default to the assumption
that it's non-nullable. Since, in fact, CREATE/ALTER SUBSCRIPTION
will store null values there, the existing marking is just wrong,
and has been since this catalog was invented.
We haven't noticed because not much in the system actually depends
on attnotnull being truthful. However, JIT'ed tuple deconstruction
does depend on that in some cases, allowing crashes or wrong answers
in queries that inspect pg_subscription. Commit 9de77b545 quite
accidentally exposed this on the buildfarm members that force JIT
activation.
Back-patch to v13. The problem goes further back, but we cannot
force initdb in released branches, so some klugier solution will
be needed there. Before working on that, push this simple fix
to try to get the buildfarm back to green.
Discussion: https://postgr.es/m/4118109.1595096139@sss.pgh.pa.us
M doc/src/sgml/catalogs.sgml
M src/include/catalog/catversion.h
M src/include/catalog/pg_subscription.h
M src/test/regress/expected/subscription.out
M src/test/regress/sql/subscription.sql
doc: Refresh more URLs in the docs
commit : f2b65519e17d6de4cd95dfe1570ab1aca187b24d
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 18 Jul 2020 22:43:41 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 18 Jul 2020 22:43:41 +0900
This updates some URLs that are redirections, mostly to an equivalent
using https. One URL referring to generalized partial indexes was
outdated.
Author: Kyotaro Horiguchi
Discussion: https://postgr.es/m/20200717.121308.1369606287593685396.horikyota.ntt@gmail.com
Backpatch-through: 9.5
M doc/src/sgml/acronyms.sgml
M doc/src/sgml/biblio.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/cube.sgml
M doc/src/sgml/dfunc.sgml
M doc/src/sgml/earthdistance.sgml
M doc/src/sgml/external-projects.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/install-windows.sgml
M doc/src/sgml/intro.sgml
M doc/src/sgml/isn.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/nls.sgml
M doc/src/sgml/pgcrypto.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/pltcl.sgml
M doc/src/sgml/seg.sgml
M doc/src/sgml/textsearch.sgml
doc: Fix description of \copy for psql
commit : 580d8ac9b1ef2803f39b1e23d9da68fff9109c42
author : Michael Paquier <michael@paquier.xyz>
date : Sat, 18 Jul 2020 10:42:46 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sat, 18 Jul 2020 10:42:46 +0900
The WHERE clause introduced by 31f3817 was not described. While on it,
split the grammar of \copy FROM and TO into two distinct parts for
clarity as they support different set of options.
Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm3zWr=OmxeNqOqfT=uZTSdam_j-gkX94CL8eTNfgUtf6A@mail.gmail.com
Backpatch-through: 12
M doc/src/sgml/ref/psql-ref.sgml
Cope with data-offset-less archive files during out-of-order restores.
commit : 71e8e66f783c143b04d381566d7a2a61e8d7598f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jul 2020 13:03:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jul 2020 13:03:50 -0400
pg_dump produces custom-format archive files that lack data offsets
when it is unable to seek its output. Up to now that's been a hazard
for pg_restore. But if pg_restore is able to seek in the archive
file, there is no reason to throw up our hands when asked to restore
data blocks out of order. Instead, whenever we are searching for a
data block, record the locations of the blocks we passed over (that
is, fill in the missing data-offset fields in our in-memory copy of
the TOC data). Then, when we hit a case that requires going
backwards, we can just seek back.
Also track the furthest point that we've searched to, and seek back
to there when beginning a search for a new data block. This avoids
possible O(N^2) time consumption, by ensuring that each data block
is examined at most twice. (On Unix systems, that's at most twice
per parallel-restore job; but since Windows uses threads here, the
threads can share block location knowledge, reducing the amount of
duplicated work.)
We can also improve the code a bit by using fseeko() to skip over
data blocks during the search.
This is all of some use even in simple restores, but it's really
significant for parallel pg_restore. In that case, we require
seekability of the input already, and we will very probably need
to do out-of-order restores.
Back-patch to v12, as this fixes a regression introduced by commit
548e50976. Before that, parallel restore avoided requesting
out-of-order restores, so it would work on a data-offset-less
archive. Now it will again.
Ideally this patch would include some test coverage, but there are
other open bugs that need to be fixed before we can extend our
coverage of parallel restore very much. Plan to revisit that later.
David Gilman and Tom Lane; reviewed by Justin Pryzby
Discussion: https://postgr.es/m/CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com
M doc/src/sgml/ref/pg_restore.sgml
M src/bin/pg_dump/pg_backup_custom.c
Remove manual tracking of file position in pg_dump/pg_backup_custom.c.
commit : 447cf2f8e9dcf9fd89c935f2daee13326a91630a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jul 2020 12:14:28 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jul 2020 12:14:28 -0400
We do not really need to track the file position by hand. We were
already relying on ftello() whenever the archive file is seekable,
while if it's not seekable we don't need the file position info
anyway because we're not going to be able to re-write the TOC.
Moreover, that tracking was buggy since it failed to account for
the effects of fseeko(). Somewhat remarkably, that seems not to
have made for any live bugs up to now. We could fix the oversights,
but it seems better to just get rid of the whole error-prone mess.
In itself this is merely code cleanup. However, it's necessary
infrastructure for an upcoming bug-fix patch (because that code
*does* need valid file position after fseeko). The bug fix
needs to go back as far as v12; hence, back-patch that far.
Discussion: https://postgr.es/m/CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com
M src/bin/pg_dump/pg_backup_custom.c
Avoid CREATE INDEX unique index deduplication.
commit : 49eb96852b02069600a4f6997dfb388fc37a2778
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 17 Jul 2020 09:50:46 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 17 Jul 2020 09:50:46 -0700
There is no advantage to attempting deduplication for a unique index
during CREATE INDEX, since there cannot possibly be any duplicates.
Doing so wastes cycles due to unnecessary copying. Make sure that we
avoid it consistently.
We already avoided unique index deduplication in the case where there
were some spool2 tuples to merge. That didn't account for the fact that
spool2 is removed early/unset in the common case where it has no tuples
that need to be merged (i.e. it failed to account for the "spool2 turns
out to be unnecessary" optimization in _bt_spools_heapscan()).
Oversight in commit 0d861bbb, which added nbtree deduplication
Backpatch: 13-, where nbtree deduplication was introduced.
M src/backend/access/nbtree/nbtsort.c
Ensure that distributed timezone abbreviation files are plain ASCII.
commit : a220e345c87d5224d810584a7c4ef29ea7a6c1f1
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jul 2020 11:03:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 17 Jul 2020 11:03:55 -0400
We had two occurrences of "Mitteleuropäische Zeit" in Europe.txt,
though the corresponding entries in Default were spelled
"Mitteleuropaeische Zeit". Standardize on the latter spelling to
avoid questions of which encoding to use.
While here, correct a couple of other trivial inconsistencies between
the Default file and the supposedly-matching entries in the *.txt
files, as exposed by some checking with comm(1). Also, add BDST to
the Europe.txt file; it previously was only listed in Default.
None of this has any direct functional effect.
Per complaint from Christoph Berg. As usual for timezone data patches,
apply to all branches.
Discussion: https://postgr.es/m/20200716100743.GE3534683@msg.df7cb.de
M src/timezone/tznames/Antarctica.txt
M src/timezone/tznames/Australia.txt
M src/timezone/tznames/Default
M src/timezone/tznames/Europe.txt
Fix whitespace
commit : 6bab40bf605665fc01b84a434127cbaec3b3d1de
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jul 2020 15:16:13 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jul 2020 15:16:13 +0200
M src/backend/utils/adt/genfile.c
Resolve gratuitous tabs in SQL file
commit : e7240ccecd3001892880fd076a7ffd107b533afc
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jul 2020 15:07:54 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 17 Jul 2020 15:07:54 +0200
M src/backend/catalog/system_views.sql
Fix signal handler setup for SIGHUP in the apply launcher process.
commit : 35647ea9d214c8745922cb757424f11d791ed733
author : Amit Kapila <akapila@postgresql.org>
date : Fri, 17 Jul 2020 08:43:06 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Fri, 17 Jul 2020 08:43:06 +0530
Commit 1e53fe0e70 has unified the usage of the config-file reload flag by
using the same signal handler function for the SIGHUP signal at many places
in the code. By mistake, it used the wrong SIGNAL in apply launcher
process for the SIGHUP signal handler function.
Author: Bharath Rupireddy
Reviewed-by: Dilip Kumar
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CALj2ACVzHCRnS20bOiEHaLtP5PVBENZQn4khdsSJQgOv_GM-LA@mail.gmail.com
M src/backend/replication/logical/launcher.c
Switch pg_test_fsync to use binary mode on Windows
commit : beebbb39d932289b73fa7cab4037895675615a8d
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 16 Jul 2020 15:52:54 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 16 Jul 2020 15:52:54 +0900
pg_test_fsync has always opened files using the text mode on Windows, as
this is the default mode used if not enforced by _setmode().
This fixes a failure when running pg_test_fsync down to 12 because
O_DSYNC and the text mode are not able to work together nicely. We
fixed the handling of O_DSYNC in 12~ for the tool by switching to the
concurrent-safe version of fopen() in src/port/ with 0ba06e0. And
40cfe86, by enforcing the text mode for compatibility reasons if O_TEXT
or O_BINARY are not specified by the caller, broke pg_test_fsync. For
all versions, this avoids any translation overhead, and pg_test_fsync
should test binary writes, so it is a gain in all cases.
Note that O_DSYNC is still not handled correctly in ~11, leading to
pg_test_fsync to show insanely high numbers for open_datasync() (using
this property it is easy to notice that the binary mode is much
faster). This would require a backpatch of 0ba06e0 and 40cfe86, which
could potentially break existing applications, so this is left out.
There are no TAP tests for this tool yet, so I have checked all builds
manually using MSVC. We could invent a new option to run a single
transaction instead of using a duration of 1s to make the tests a
maximum short, but this is left as future work.
Thanks to Bruce Momjian for the discussion.
Reported-by: Jeff Janes
Author: Michael Paquier
Discussion: https://postgr.es/m/16526-279ded30a230d275@postgresql.org
Backpatch-through: 9.5
M src/bin/pg_test_fsync/pg_test_fsync.c
doc: Fix typo
commit : 6b5ca893f737184a8113aee33c6c10b1c3bbcd39
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jul 2020 21:01:29 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 15 Jul 2020 21:01:29 +0200
M doc/src/sgml/release-13.sgml
Fix handling of missing files when using pg_rewind with online source
commit : 5f89bb4cf0109bdb36eb8f78943f5b0f141c614a
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 15 Jul 2020 15:17:32 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 15 Jul 2020 15:17:32 +0900
When working with an online source cluster, pg_rewind gets a list of all
the files in the source data directory using a WITH RECURSIVE query,
returning a NULL result for a file's metadata if it gets removed between
the moment it is listed in a directory and the moment its metadata is
obtained with pg_stat_file() (say a recycled WAL segment). The query
result was processed in such a way that for each tuple we checked only
that the first file's metadata was NULL. This could have two
consequences, both resulting in a failure of the rewind:
- If the first tuple referred to a removed file, all files from the
source would be ignored.
- Any file actually missing would not be considered as such.
While on it, rework slightly the code so as no values are saved if we
know that a file is going to be skipped.
Issue introduced by b36805f, so backpatch down to 9.5.
Author: Justin Pryzby, Michael Paquier
Reviewed-by: Daniel Gustafsson, Masahiko Sawada
Discussion: https://postgr.es/m/20200713061010.GC23581@telsasoft.com
Backpatch-through: 9.5
M src/bin/pg_rewind/libpq_fetch.c
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
commit : e38705b5c7635644a70cf7c9eadb7a90bb8ea13b
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Jul 2020 18:56:49 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 14 Jul 2020 18:56:49 -0400
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
M src/backend/optimizer/path/indxpath.c
M src/backend/optimizer/util/pathnode.c
M src/test/regress/expected/partition_join.out
M src/test/regress/sql/partition_join.sql
Fix timing issue with ALTER TABLE's validate constraint
commit : b827304291aff8019cdd0cee68219fe43f064380
author : David Rowley <drowley@postgresql.org>
date : Tue, 14 Jul 2020 16:57:41 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 14 Jul 2020 16:57:41 +1200
An ALTER TABLE to validate a foreign key in which another subcommand
already caused a pending table rewrite could fail due to ALTER TABLE
attempting to validate the foreign key before the actual table rewrite
takes place. This situation could result in an error such as:
ERROR: could not read block 0 in file "base/nnnnn/nnnnn": read only 0 of 8192 bytes
The failure here was due to the SPI call which validates the foreign key
trying to access an index which is yet to be rebuilt.
Similarly, we also incorrectly tried to validate CHECK constraints before
the heap had been rewritten.
The fix for both is to delay constraint validation until phase 3, after
the table has been rewritten. For CHECK constraints this means a slight
behavioral change. Previously ALTER TABLE VALIDATE CONSTRAINT on
inheritance tables would be validated from the bottom up. This was
different from the order of evaluation when a new CHECK constraint was
added. The changes made here aligns the VALIDATE CONSTRAINT evaluation
order for inheritance tables to be the same as ADD CONSTRAINT, which is
generally top-down.
Reported-by: Nazli Ugur Koyluoglu, using SQLancer
Discussion: https://postgr.es/m/CAApHDvp%3DZXv8wiRyk_0rWr00skhGkt8vXDrHJYXRMft3TjkxCA%40mail.gmail.com
Backpatch-through: 9.5 (all supported versions)
M src/backend/commands/tablecmds.c
M src/test/regress/expected/alter_table.out
M src/test/regress/sql/alter_table.sql
Fix comments related to table AMs
commit : 9678c08184a82deafac9297b9af0fc5cb07ab347
author : Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jul 2020 13:17:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Tue, 14 Jul 2020 13:17:31 +0900
Incorrect function names were referenced. As this fixes some portions
of tableam.h, that is mentioned in the docs as something to look at when
implementing a table AM, backpatch down to 12 where this has been
introduced.
Author: Hironobu Suzuki
Discussion: https://postgr.es/m/8fe6d672-28dd-3f1d-7aed-ac2f6d599d3f@interdb.jp
Backpatch-through: 12
M src/backend/access/heap/heapam.c
M src/include/access/tableam.h
Cope with lateral references in the quals of a subquery RTE.
commit : 0734dbc45034540fec1d98b15c9f173ad1d4af02
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Jul 2020 20:38:20 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Jul 2020 20:38:20 -0400
The qual pushdown logic assumed that all Vars in a restriction clause
must be Vars referencing subquery outputs; but since we introduced
LATERAL, it's possible for such a Var to be a lateral reference instead.
This led to an assertion failure in debug builds. In a non-debug
build, there might be no ill effects (if qual_is_pushdown_safe decided
the qual was unsafe anyway), or we could get failures later due to
construction of an invalid plan. I've not gone to much length to
characterize the possible failures, but at least segfaults in the
executor have been observed.
Given that this has been busted since 9.3 and it took this long for
anybody to notice, I judge that the case isn't worth going to great
lengths to optimize. Hence, fix by just teaching qual_is_pushdown_safe
that such quals are unsafe to push down, matching the previous behavior
when it accidentally didn't fail.
Per report from Tom Ellis. Back-patch to all supported branches.
Discussion: https://postgr.es/m/20200713175124.GQ8220@cloudinit-builder
M src/backend/optimizer/path/allpaths.c
M src/test/regress/expected/subselect.out
M src/test/regress/sql/subselect.sql
Fix uninitialized value in segno calculation
commit : 794e8e32bb5ae1b38a90cbae2a88895633797599
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 13 Jul 2020 13:49:50 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Mon, 13 Jul 2020 13:49:50 -0400
Remove previous hack in KeepLogSeg that added a case to deal with a
(badly represented) invalid segment number. This was added for the sake
of GetWALAvailability. But it's not needed if in that function we
initialize the segment number to be retreated to the currently being
written segment, so do that instead.
Per valgrind-running buildfarm member skink, and some sparc64 animals.
Discussion: https://postgr.es/m/1724648.1594230917@sss.pgh.pa.us
M src/backend/access/transam/xlog.c
Fix bugs in libpq's management of GSS encryption state.
commit : 8e6f134a9a8db52cbd2818ce8a60b9e5cdadfc8f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Jul 2020 11:57:55 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 13 Jul 2020 11:57:55 -0400
GSS-related resources should be cleaned up in pqDropConnection,
not freePGconn, else the wrong things happen when resetting
a connection or trying to switch to a different server.
It's also critical to reset conn->gssenc there.
During connection setup, initialize conn->try_gss at the correct
place, else switching to a different server won't work right.
Remove now-redundant cleanup of GSS resources around one (and, for
some reason, only one) pqDropConnection call in connectDBStart.
Per report from Kyotaro Horiguchi that psql would freeze up,
rather than successfully resetting a GSS-encrypted connection
after a server restart.
This is YA oversight in commit b0b39f72b, so back-patch to v12.
Discussion: https://postgr.es/m/20200710.173803.435804731896516388.horikyota.ntt@gmail.com
M src/interfaces/libpq/fe-connect.c
Improvements to psql \dAo and \dAp commands
commit : ae290059e1aa5bc2272a0345184bcf05407f69a4
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 11 Jul 2020 14:14:49 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 11 Jul 2020 14:14:49 +0300
* Strategy number and purpose are essential information for opfamily operator.
So, show those columns in non-verbose output.
* "Left/right arg type" \dAp column names are confusing, because those type
don't necessary match to function arguments. Rename them to "Registered
left/right type".
* Replace manual assembling of operator/procedure names with casts to
regoperator/regprocedure.
* Add schema-qualification for pg_catalog functions and tables.
Reported-by: Peter Eisentraut, Tom Lane
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/2edc7b27-031f-b2b6-0db2-864241c91cb9%402ndquadrant.com
Backpatch-through: 13
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/test/regress/expected/psql.out
M src/test/regress/sql/psql.sql
HashAgg: before spilling tuples, set unneeded columns to NULL.
commit : d8a7ce245095e3a70a2ad738c17be95593f68996
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 12 Jul 2020 17:48:49 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 12 Jul 2020 17:48:49 -0700
This is a replacement for 4cad2534. Instead of projecting all tuples
going into a HashAgg, only remove unnecessary attributes when actually
spilling. This avoids the regression for the in-memory case.
Discussion: https://postgr.es/m/a2fb7dfeb4f50aa0a123e42151ee3013933cb802.camel%40j-davis.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
M src/include/nodes/execnodes.h
Revert "Use CP_SMALL_TLIST for hash aggregate"
commit : 926ecf83c0bab7e985eaf5727bb69cdf8ce6b067
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 12 Jul 2020 16:46:19 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 12 Jul 2020 16:46:19 -0700
This reverts commit 4cad2534da6d17067d98cf04be2dfc1bda8f2cd0 due to a
performance regression. It will be replaced by a new approach in an
upcoming commit.
Reported-by: Andres Freund
Discussion: https://postgr.es/m/20200614181418.mx4bvljmfkkhoqzl@alap3.anarazel.de
Backpatch-through: 13
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/optimizer/plan/createplan.c
Revert "Track statistics for spilling of changes from ReorderBuffer".
commit : b074813d48bcd2a7e224b56a3aff6db9df745237
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 13 Jul 2020 08:27:40 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 13 Jul 2020 08:27:40 +0530
The stats with this commit was available only for WALSenders, however,
users might want to see for backends doing logical decoding via SQL API.
Then, users might want to reset and access these stats across server
restart which was not possible with the current patch.
List of commits reverted:
caa3c4242c Don't call elog() while holding spinlock.
e641b2a995 Doc: Update the documentation for spilled transaction
statistics.
5883f5fe27 Fix unportable printf format introduced in commit 9290ad198.
9290ad198b Track statistics for spilling of changes from ReorderBuffer.
Additionaly, remove the release notes entry for this feature.
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CA+fd4k5_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/release-13.sgml
M src/backend/catalog/system_views.sql
M src/backend/replication/logical/reorderbuffer.c
M src/backend/replication/walsender.c
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/include/replication/reorderbuffer.h
M src/include/replication/walsender_private.h
M src/test/regress/expected/rules.out
Avoid trying to restore table ACLs and per-column ACLs in parallel.
commit : bc9aaac1a188cac11b1ebb04047de3db71257785
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Jul 2020 13:36:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 11 Jul 2020 13:36:50 -0400
Parallel pg_restore has always supposed that ACL items for different
objects are independent and can be restored in parallel without
conflicts. However, there is one case where this fails: because
REVOKE on a table is defined to also revoke the privilege(s) at
column level, we can't restore per-column ACLs till after we restore
any table-level privileges on their table. Failure to honor this
restriction can lead to "tuple concurrently updated" errors during
parallel restore, or even to the per-column ACLs silently disappearing
because the table-level REVOKE is executed afterwards.
To fix, add a dependency from each column-level ACL item to its table's
ACL item, if there is one. Note that this doesn't fix the hazard
for pre-existing archive files, only for ones made with a corrected
pg_dump. Given that the bug's been there quite awhile without
field reports, I think this is acceptable.
This requires changing the API of pg_dump's dumpACL() function.
To keep its argument list from getting even longer, I removed the
"CatalogId objCatId" argument, which has been unused for ages.
Per report from Justin Pryzby. Back-patch to all supported branches.
Discussion: https://postgr.es/m/20200706050129.GW4107@telsasoft.com
M src/bin/pg_dump/common.c
M src/bin/pg_dump/pg_backup.h
M src/bin/pg_dump/pg_dump.c
Forbid numeric NaN in jsonpath
commit : 89a0b1a7ca0af36818ed7076c12ac00bcf4f007d
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 11 Jul 2020 03:21:00 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 11 Jul 2020 03:21:00 +0300
SQL standard doesn't define numeric Inf or NaN values. It appears even more
ridiculous to support then in jsonpath assuming JSON doesn't support these
values as well. This commit forbids returning NaN from .double(), which was
previously allowed. NaN can't be result of inner-jsonpath computation over
non-NaNs. So, we can not expect NaN in the jsonpath output.
Reported-by: Tom Lane
Discussion: https://postgr.es/m/203949.1591879542%40sss.pgh.pa.us
Author: Alexander Korotkov
Reviewed-by: Tom Lane
Backpatch-through: 12
M src/backend/utils/adt/jsonb_util.c
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
Improve error reporting for jsonpath .double() method
commit : b9a04a9bc6653183ed23532145325694fbc46002
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 11 Jul 2020 03:20:46 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 11 Jul 2020 03:20:46 +0300
When jsonpath .double() method detects that numeric or string can't be
converted to double precision, it throws an error. This commit makes these
errors explicitly express the reason of failure.
Discussion: https://postgr.es/m/CAPpHfdtqJtiSXkP7tOXez18NxhLUH_-75bL8%3DOce4Ki%2Bbv7V6Q%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane
Backpatch-through: 12
M src/backend/utils/adt/jsonpath_exec.c
M src/test/regress/expected/jsonb_jsonpath.out
M src/test/regress/sql/jsonb_jsonpath.sql
Doc: update or remove dead external links.
commit : 763a0b63a25c13ac940ce2c64b37b8446ccbf895
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 10 Jul 2020 13:16:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 10 Jul 2020 13:16:00 -0400
Re-point comp.ai.genetic FAQ link to a more stable address.
Remove stale links to AIX documentation; we don't really need to
tell AIX users how to use their systems.
Remove stale links to HP documentation about SSL. We've had to
update those twice before, making it increasingly obvious that
HP does not intend them to be stable landing points. They're
not particularly authoritative, either. (This change effectively
reverts bbd3bdba3.)
Daniel Gustafsson and Álvaro Herrera, per a gripe from
Kyotaro Horiguchi. Back-patch, since these links are
just as dead in the back branches.
Discussion: https://postgr.es/m/20200709.161226.204639179120026914.horikyota.ntt@gmail.com
M doc/src/sgml/geqo.sgml
M doc/src/sgml/installation.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/runtime.sgml
Log the location field before any backtrace
commit : 8ff4d1277b8660de85e4a7d796ccc1b64187d80f
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 10 Jul 2020 08:27:00 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 10 Jul 2020 08:27:00 +0200
This order makes more sense because the location is effectively at the
lowest level of the backtrace.
Discussion: https://www.postgresql.org/message-id/flat/90f5fa04-c410-a54e-9449-aa3749fb7972%402ndquadrant.com
M src/backend/utils/error/elog.c
Remove WARNING message from brin_desummarize_range
commit : c3a79e71929a6b5cdd2416ab4976dab59736c937
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jul 2020 20:13:25 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Thu, 9 Jul 2020 20:13:25 -0400
This message was being emitted on the grounds that only crashed
summarization could cause it, but in reality even an aborted vacuum
could do it ... which makes it way too noisy, particularly since it
shows up in regression tests and makes them die.
Reported by Tom Lane.
Discussion: https://postgr.es/m/489091.1593534251@sss.pgh.pa.us
M src/backend/access/brin/brin_revmap.c
Tighten up Windows CRLF conversion in our TAP test scripts.
commit : 17b87b3049fa7e3ddc68bf9daaffa3b01d7b8be2
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Jul 2020 17:38:52 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Jul 2020 17:38:52 -0400
Back-patch commits 91bdf499b and ffb4cee43, so that all branches
agree on when and how to do Windows CRLF conversion.
This should close the referenced thread. Thanks to Andrew Dunstan
for discussion/review.
Discussion: https://postgr.es/m/412ae8da-76bb-640f-039a-f3513499e53d@gmx.net
M src/bin/pg_rewind/t/RewindTest.pm
M src/test/perl/PostgresNode.pm
M src/test/perl/TestLib.pm
Fix pg_current_logfile() to not emit a carriage return on Windows.
commit : 601d419b2b5def62a1867169e67c7ef876f0b886
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Jul 2020 16:02:23 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 9 Jul 2020 16:02:23 -0400
Due to not having our signals straight about CRLF vs. LF line
termination, the output of pg_current_logfile() included a trailing
\r on Windows. To fix, force the file descriptor it uses into text
mode.
While here, move a couple of local variable declarations to make
the function's logic clearer.
In v12 and v13, also back-patch the test added by 1c4e88e2f so that
this function has some test coverage. However, the 004_logrotate.pl
test script doesn't exist before v12, and it didn't seem worth adding
to older branches just for this.
Per report from Thomas Kellerer. Back-patch to v10 where this
function was added.
Discussion: https://postgr.es/m/412ae8da-76bb-640f-039a-f3513499e53d@gmx.net
M src/backend/utils/adt/misc.c
M src/bin/pg_ctl/t/004_logrotate.pl
doc: Correct the description about the length of pg_stat_activity.query.
commit : 331da659bedbad2c4d649d70cccf9d87ca1a0b7e
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 9 Jul 2020 13:31:33 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 9 Jul 2020 13:31:33 +0900
pg_stat_activity.query text is truncated at 1024 bytes. But previously
the document described that it's truncated at 1024 characters.
This was not accurate when considering multibyte characters.
Back-patch to v10 where this inaccurate description was added.
Author: Atsushi Torikoshi
Reviewed-by: Daniel Gustafsson, Fujii Masao
Discussion: https://postgr.es/m/cd5b49a5a14e887542f5f569c1c6bde2@oss.nttdata.com
M doc/src/sgml/monitoring.sgml
Fix whitespace in HashAgg EXPLAIN ANALYZE
commit : 285da44a69ddcbe8aa955b5f863e02121f41c189
author : David Rowley <drowley@postgresql.org>
date : Thu, 9 Jul 2020 10:07:00 +1200
committer: David Rowley <drowley@postgresql.org>
date : Thu, 9 Jul 2020 10:07:00 +1200
The Sort node does not put a space between the number of kilobytes and
the "kB" of memory or disk space used, but HashAgg does. Here we align
HashAgg to do the same as Sort. Sort has been displaying this
information for longer than HashAgg, so it makes sense to align HashAgg
to Sort rather than the other way around.
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20200708163021.GW4107@telsasoft.com
Backpatch-through: 13, where the hashagg started showing these details
M src/backend/commands/explain.c
Fix incorrect variable datatype.
commit : a2b94693beb1d052f2b4935384dd6f7f47143669
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 8 Jul 2020 21:24:34 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 8 Jul 2020 21:24:34 +0900
Since slot_keep_segs indicates the number of WAL segments not LSN,
its datatype should not be XLogRecPtr.
Back-patch to v13 where this issue was added.
Reported-by: Atsushi Torikoshi
Author: Atsushi Torikoshi, tweaked by Fujii Masao
Discussion: https://postgr.es/m/ebd0d674f3e050222238a960cac5251a@oss.nttdata.com
M src/backend/access/transam/xlog.c
doc: Fix inconsistencies in GIN, BRIN and SP-GiST for optional opclass methods
commit : ea5737889f0586c2d46738bc52b97b86369f03e2
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Jul 2020 10:42:15 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 8 Jul 2020 10:42:15 +0900
The GIN and SP-GiST parts were out-of-sync since the changes of 14903f2,
and the BRIN part was wrong since its introduction in 15cb2bd.
Author: Guillaume Lelarge
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CAECtzeXKvEPEr967h0PRYRi39uTmdEms=oUtc_PWGjZRNN1prw@mail.gmail.com
Backpatch-through: 13
M doc/src/sgml/brin.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/spgist.sgml
Morph pg_replication_slots.min_safe_lsn to safe_wal_size
commit : c54b5891f415df36809de1aeb97e4574d5456d69
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jul 2020 13:08:00 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Tue, 7 Jul 2020 13:08:00 -0400
The previous definition of the column was almost universally disliked,
so provide this updated definition which is more useful for monitoring
purposes: a large positive value is good, while zero or a negative value
means danger. This should be operationally more convenient.
Backpatch to 13, where the new column to pg_replication_slots (and the
feature it represents) were added.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reported-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Discussion: https://postgr.es/m/9ddfbf8c-2f67-904d-44ed-cf8bc5916228@oss.nttdata.com
M doc/src/sgml/catalogs.sgml
M src/backend/access/transam/xlog.c
M src/backend/catalog/system_views.sql
M src/backend/replication/slotfuncs.c
M src/include/access/xlog_internal.h
M src/include/catalog/catversion.h
M src/include/catalog/pg_proc.dat
M src/test/recovery/t/019_replslot_limit.pl
M src/test/regress/expected/rules.out
doc: Add note about possible performance overhead by enabling track_planning.
commit : da6b6ff95bcaadc109ab248471527a2511e853d5
author : Fujii Masao <fujii@postgresql.org>
date : Mon, 6 Jul 2020 14:27:09 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Mon, 6 Jul 2020 14:27:09 +0900
Enabling pg_stat_statements.track_plaanning may incur a noticeable
performance penalty, especially when a fewer kinds of queries are executed
on many concurrent connections. This commit documents this note.
Back-patch to v13 where pg_stat_statements.track_plaanning was added.
Suggested-by: Pavel Stehule
Author: Fujii Masao
Reviewed-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRC9Jxa8r5i0TNBWLb8mzuaYzEoLq3QOvip0jVpHPOLbVA@mail.gmail.com
M doc/src/sgml/pgstatstatements.sgml
Remove extra whitespace in comments atop ReorderBufferCheckMemoryLimit.
commit : e163f3a2b1d987f83e291e86969ed4a91ded6abb
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 6 Jul 2020 08:36:58 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 6 Jul 2020 08:36:58 +0530
Backpatch-through: 13, where it was introduced
M src/backend/replication/logical/reorderbuffer.c
Remove unused function parameter in end_parallel_vacuum.
commit : f92c24ec9f61b3502007e2a9a6de4c236844254d
author : Amit Kapila <akapila@postgresql.org>
date : Mon, 6 Jul 2020 08:24:12 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Mon, 6 Jul 2020 08:24:12 +0530
Author: Vignesh C
Reviewed-by: Sawada Masahiko
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CALDaNm3Ppt71NafGY5mk3V2i3Q+mm93pVibDq-0NpW7WU67Jcg@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
doc: Spell checking
commit : ffb23488af5e6776935c46370465dcc1704e7540
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jul 2020 15:37:57 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jul 2020 15:37:57 +0200
M contrib/pg_stat_statements/pg_stat_statements.c
M doc/src/sgml/backup-manifest.sgml
M doc/src/sgml/extend.sgml
M doc/src/sgml/fdwhandler.sgml
M doc/src/sgml/glossary.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/create_table.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/ref/pgbench.sgml
doc: Fix incorrect reference to textout in plpgsql examples
commit : 45f165b18b72abb1e4579a3cca0862a4e5cb8b6b
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 5 Jul 2020 19:36:12 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 5 Jul 2020 19:36:12 +0900
This error has survived for 22 years, and has been introduced by
da63386.
Reported-by: Erwin Brandstetter
Discussion: https://postgr.es/m/CAGHENJ57wogGOvGXo5LgWYcqswxafLck8ELqHDR+zrkTPgs_OQ@mail.gmail.com
Backpatch-through: 9.5
M doc/src/sgml/plpgsql.sgml
Rename enable_incrementalsort for clarity
commit : 94e454cddfbae5e32ae7bb70fedd24f243cd486a
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jul 2020 11:41:52 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 5 Jul 2020 11:41:52 +0200
Author: James Coleman <jtc331@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/df652910-e985-9547-152c-9d4357dc3979%402ndquadrant.com
M doc/src/sgml/config.sgml
M doc/src/sgml/release-13.sgml
M src/backend/optimizer/path/allpaths.c
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc.c
M src/backend/utils/misc/postgresql.conf.sample
M src/include/optimizer/cost.h
M src/test/regress/expected/incremental_sort.out
M src/test/regress/expected/partition_aggregate.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/incremental_sort.sql
M src/test/regress/sql/partition_aggregate.sql
Fix "ignoring return value" complaints from commit 96d1f423f9
commit : c536da177cbbc9e30de17a0a445b53d79a5bbe7f
author : Joe Conway <mail@joeconway.com>
date : Sat, 4 Jul 2020 13:47:07 -0400
committer: Joe Conway <mail@joeconway.com>
date : Sat, 4 Jul 2020 13:47:07 -0400
The cfbot and some BF animals are complaining about the previous
read_binary_file commit because of ignoring return value of ‘fread’.
So let's make everyone happy by testing the return value even though
not strictly needed.
Reported by Justin Pryzby, and suggested patch by Tom Lane. Backpatched
to v11 same as the previous commit.
Reported-By: Justin Pryzby
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com
Backpatch-through: 11
M src/backend/utils/adt/genfile.c
Read until EOF vice stat-reported size in read_binary_file
commit : 0025c3a2c295459002711e0b37e48e3b067a83ba
author : Joe Conway <mail@joeconway.com>
date : Sat, 4 Jul 2020 06:28:21 -0400
committer: Joe Conway <mail@joeconway.com>
date : Sat, 4 Jul 2020 06:28:21 -0400
read_binary_file(), used by SQL functions pg_read_file() and friends,
uses stat to determine file length to read, when not passed an explicit
length as an argument. This is problematic, for example, if the file
being read is a virtual file with a stat-reported length of zero.
Arrange to read until EOF, or StringInfo data string lenth limit, is
reached instead.
Original complaint and patch by me, with significant review, corrections,
advice, and code optimizations by Tom Lane. Backpatched to v11. Prior to
that only paths relative to the data and log dirs were allowed for files,
so no "zero length" files were reachable anyway.
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/flat/969b8d82-5bb2-5fa8-4eb1-f0e685c5d736%40joeconway.com
Backpatch-through: 11
M contrib/adminpack/expected/adminpack.out
M src/backend/utils/adt/genfile.c
Clamp total-tuples estimates for foreign tables to ensure planner sanity.
commit : 9233624b128b41fd410712a7223821878f1943b0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Jul 2020 19:01:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Jul 2020 19:01:21 -0400
After running GetForeignRelSize for a foreign table, adjust rel->tuples
to be at least as large as rel->rows. This prevents bizarre behavior
in estimate_num_groups() and perhaps other places, especially in the
scenario where rel->tuples is zero because pg_class.reltuples is
(suggesting that ANALYZE has never been run for the table). As things
stood, we'd end up estimating one group out of any GROUP BY on such a
table, whereas the default group-count estimate is more likely to result
in a sane plan.
Also, clarify in the documentation that GetForeignRelSize has the option
to override the rel->tuples value if it has a better idea of what to use
than what is in pg_class.reltuples.
Per report from Jeff Janes. Back-patch to all supported branches.
Patch by me; thanks to Etsuro Fujita for review
Discussion: https://postgr.es/m/CAMkU=1xNo9cnan+Npxgz0eK7394xmjmKg-QEm8wYG9P5-CcaqQ@mail.gmail.com
M doc/src/sgml/fdwhandler.sgml
M src/backend/optimizer/path/allpaths.c
Fix temporary tablespaces for shared filesets some more.
commit : cfe89f5e6b7874e89dac7d9511b1894a7d033870
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Jul 2020 17:01:34 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 3 Jul 2020 17:01:34 -0400
Commit ecd9e9f0b fixed the problem in the wrong place, causing unwanted
side-effects on the behavior of GetNextTempTableSpace(). Instead,
let's make SharedFileSetInit() responsible for subbing in the value
of MyDatabaseTableSpace when the default tablespace is called for.
The convention about what is in the tempTableSpaces[] array is
evidently insufficiently documented, so try to improve that.
It also looks like SharedFileSetInit() is doing the wrong thing in the
case where temp_tablespaces is empty. It was hard-wiring use of the
pg_default tablespace, but it seems like using MyDatabaseTableSpace
is more consistent with what happens for other temp files.
Back-patch the reversion of PrepareTempTablespaces()'s behavior to
9.5, as ecd9e9f0b was. The changes in SharedFileSetInit() go back
to v11 where that was introduced. (Note there is net zero code change
before v11 from these two patch sets, so nothing to release-note.)
Magnus Hagander and Tom Lane
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
M src/backend/commands/tablespace.c
M src/backend/storage/file/fd.c
M src/backend/storage/file/sharedfileset.c
Fix temporary tablespaces for shared filesets
commit : 1d94c3965450417ebb0e0fd73ea636df823feeed
author : Magnus Hagander <magnus@hagander.net>
date : Fri, 3 Jul 2020 15:09:06 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Fri, 3 Jul 2020 15:09:06 +0200
A likely copy/paste error in 98e8b480532 from back in 2004 would
cause temp tablespace to be reset to InvalidOid if temp_tablespaces
was set to the same value as the primary tablespace in the database.
This would cause shared filesets (such as for parallel hash joins)
to ignore them, putting the temporary files in the default tablespace
instead of the configured one. The bug is in the old code, but it
appears to have been exposed only once we had shared filesets.
Reviewed-By: Daniel Gustafsson
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
Backpatch-through: 9.5
M src/backend/commands/tablespace.c
doc: Correct description of restart_lsn in pg_replication_slots
commit : 95a604eaebd145729d9f8c936b37703a212f27fd
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 3 Jul 2020 12:08:35 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 3 Jul 2020 12:08:35 +0900
Previously the document explained that restart_lsn indicates the LSN of
oldest WAL won't be automatically removed during checkpoints. But
since v13 this was no longer true thanks to max_slot_wal_keep_size.
Back-patch to v13 where max_slot_wal_keep_size was added.
Author: Fujii Masao
Discussion: https://postgr.es/m/6497f1e9-3148-c5da-7e49-b2fddad9a42f@oss.nttdata.com
M doc/src/sgml/catalogs.sgml
Change default of pg_stat_statements.track_planning to off.
commit : 8d459762b10372e48845a49b305f4e1e165fe173
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 3 Jul 2020 11:35:22 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 3 Jul 2020 11:35:22 +0900
Since v13 pg_stat_statements is allowed to track the planning time of
statements when track_planning option is enabled. Its default was on.
But this feature could cause more terrible spinlock contentions in
pg_stat_statements. As a result of this, Robins Tharakan reported that
v13 beta1 showed ~45% performance drop at high DB connection counts
(when compared with v12.3) during fully-cached SELECT-only test using
pgbench.
To avoid this performance regression by the default setting,
this commit changes default of pg_stat_statements.track_planning to off.
Back-patch to v13 where pg_stat_statements.track_planning was introduced.
Reported-by: Robins Tharakan
Author: Fujii Masao
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/2895b53b033c47ccb22972b589050dd9@EX13D05UWC001.ant.amazon.com
M contrib/pg_stat_statements/expected/pg_stat_statements.out
M contrib/pg_stat_statements/pg_stat_statements.c
M contrib/pg_stat_statements/sql/pg_stat_statements.sql
M doc/src/sgml/pgstatstatements.sgml
Improve vacuum error context handling.
commit : 83fa48c8cd26c9a8171a85e786bb6ae1c5b04139
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 1 Jul 2020 08:06:00 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 1 Jul 2020 08:06:00 +0530
Use separate functions to save and restore error context information as
that made code easier to understand. Also, make it clear that the index
information required for error context is sane.
Author: Andres Freund, Justin Pryzby, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CAA4eK1LWo+v1OWu=Sky27GTGSCuOmr7iaURNbc5xz6jO+SaPeA@mail.gmail.com
M src/backend/access/heap/vacuumlazy.c
M src/tools/pgindent/typedefs.list
Fix removal of files generated by TAP tests for SSL
commit : 48d50ee9aff9be0817a175418e100b7d7fa55a0f
author : Michael Paquier <michael@paquier.xyz>
date : Wed, 1 Jul 2020 10:47:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Wed, 1 Jul 2020 10:47:29 +0900
001_ssltests.pl and 002_scram.pl both generated an extra file for a
client key used in the tests that were not removed. In Debian, this
causes repeated builds to fail.
The code refactoring done in 4dc6355 broke the cleanup done in
001_ssltests.pl, and the new tests added in 002_scram.pl via d6e612f
forgot the removal of one file. While on it, fix a second issue
introduced in 002_scram.pl where we use the same file name in 001 and
002 for the temporary client key whose permissions are changed in the
test, as using the same file name in both tests could cause failures
with parallel jobs of src/test/ssl/ if one test removes a file still
needed by the second test.
Reported-by: Felix Lechner
Author: Daniel Gustafsson, Felix Lechner
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CAFHYt543sjX=Cm_aEeoejStyP47C+Y3+Wh6WbirLXsgUMaw7iw@mail.gmail.com
Backpatch-through: 13
M src/test/ssl/t/001_ssltests.pl
M src/test/ssl/t/002_scram.pl
Further adjustments to Hashagg EXPLAIN ANALYZE output
commit : d73e9a57bf5bd977d9bf36bc07c77a1acf45e35b
author : David Rowley <drowley@postgresql.org>
date : Wed, 1 Jul 2020 12:16:42 +1200
committer: David Rowley <drowley@postgresql.org>
date : Wed, 1 Jul 2020 12:16:42 +1200
The "Disk Usage" and "HashAgg Batches" properties in the EXPLAIN ANALYZE
output for HashAgg were previously only shown if the number of batches
was greater than 0. Here we change this so that these properties are
always shown for EXPLAIN ANALYZE formats other than "text". The idea here
is that since the HashAgg could have spilled to disk if there had been
more data or groups to aggregate, then it's relevant that we're clear in
the EXPLAIN ANALYZE output when no spilling occurred in this particular
execution of the given plan.
For the "text" EXPLAIN format, we still hide these properties when no
spilling occurs. This EXPLAIN format is designed to be easy for humans
to read. To maintain the readability we have a higher threshold for which
properties we display for this format.
Discussion: https://postgr.es/m/CAApHDvo_dmNozQQTmN-2jGp1vT%3Ddxx7Q0vd%2BMvD1cGpv2HU%3DSg%40mail.gmail.com
Backpatch-through: 13, where the hashagg spilling code was added.
M src/backend/commands/explain.c
Fix ecpg crash with bytea and cursor variables.
commit : 70dc45e8cb76e0c612648ccefc433b7fb2b16c2b
author : Michael Meskes <meskes@postgresql.org>
date : Tue, 30 Jun 2020 17:31:08 +0200
committer: Michael Meskes <meskes@postgresql.org>
date : Tue, 30 Jun 2020 17:31:08 +0200
Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
M src/interfaces/ecpg/preproc/ecpg.header
M src/interfaces/ecpg/test/expected/sql-bytea.c
M src/interfaces/ecpg/test/expected/sql-bytea.stderr
M src/interfaces/ecpg/test/expected/sql-bytea.stdout
M src/interfaces/ecpg/test/sql/bytea.pgc
doc: clarify that storage parameter values are optional
commit : 0bddb3a95995008ed116858ddde9a89e01659dae
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Jun 2020 12:26:51 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Jun 2020 12:26:51 -0400
In a few cases, the documented syntax specified storage parameter values
as required.
Reported-by: galiev_mr@taximaxim.ru
Discussion: https://postgr.es/m/159283163235.684.4482737698910467437@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/ref/alter_index.sgml
M doc/src/sgml/ref/alter_materialized_view.sgml
M doc/src/sgml/ref/alter_table.sgml
M doc/src/sgml/ref/create_index.sgml
doc: change pg_upgrade wal_level to be not minimal
commit : c7ff80ffaa933d26298ce2d4eb7bd90d56c16668
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Jun 2020 11:55:53 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 30 Jun 2020 11:55:53 -0400
Previously it was specified to be only replica.
Discussion: https://postgr.es/m/20200618180058.GK7349@momjian.us
Backpatch-through: 9.5
M doc/src/sgml/ref/pgupgrade.sgml
Remove support for timezone "posixrules" file.
commit : 21aac2ff96e37c75cc6814b86d4b55172090413c
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 29 Jun 2020 18:55:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 29 Jun 2020 18:55:01 -0400
The IANA tzcode library has a feature to read a time zone file named
"posixrules" and apply the daylight-savings transition dates and times
therein, when it is given a POSIX-style time zone specification that
lacks an explicit transition rule. However, there's a problem with
that code: it doesn't work for dates past the Y2038 time_t rollover.
(Effectively, all times beyond that point are treated as standard
time.) The IANA crew regard this feature as legacy, so their plan is
to remove it not fix it. The time frame in which that will happen
is unclear, but presumably it'll happen well before 2038.
Moreover, effective with the next IANA data update (probably this
fall), the recommended default will be to not install a "posixrules"
file in the first place. The time frame in which tzdata packagers
might adopt that suggestion is likewise unclear, but at least some
platforms will probably do it in the next year or so. While we could
ignore that recommendation so far as PG-supplied tzdata trees are
concerned, builds using --with-system-tzdata will be subject to
whatever the platform's tzdata packager decides to do.
Thus, whether or not we do anything, some increasing fraction of
Postgres users will be exposed to the behavior observed when there
is no "posixrules" file; and if we do nothing, we'll have essentially
no control over the timing of that change.
The best thing to do to ameliorate the uncertainty seems to be to
proactively remove the posixrules-reading feature. If we do that in
a scheduled release then at least we can release-note the behavioral
change, rather than having users be surprised by it after a routine
tzdata update.
The change in question is fairly minor anyway: to be affected,
you have to be using a POSIX-style timezone spec, it has to not
have an explicit rule, and it has to not be one of the four traditional
continental-USA zone names (EST5EDT, CST6CDT, MST7MDT, or PST8PDT),
as those are special-cased. Since the default "posixrules" file
provides USA DST rules, the number of people who are likely to find
such a zone spec useful is probably quite small. Moreover, the
fallback behavior with no explicit rule and no "posixrules" file is to
apply current USA rules, so the only thing that really breaks is the
DST transitions in years before 2007 (and you get the countervailing
fix that transitions after 2038 will be applied).
Now, some installations might have replaced the "posixrules" file,
allowing e.g. EU rules to be applied to a POSIX-style timezone spec.
That won't work anymore. But it's not exactly clear why this solution
would be preferable to using a regular named zone. In any case, given
the Y2038 issue, we need to be pushing users to stop depending on this.
Back-patch into v13; it hasn't been released yet, so it seems OK to
change its behavior. (Personally I think we ought to back-patch
further, but I've been outvoted.)
Discussion: https://postgr.es/m/1390.1562258309@sss.pgh.pa.us
Discussion: https://postgr.es/m/20200621211855.6211-1-eggert@cs.ucla.edu
M doc/src/sgml/datetime.sgml
M src/timezone/Makefile
M src/timezone/README
M src/timezone/localtime.c
M src/tools/msvc/Install.pm
Fix documentation of "must be vacuumed within" warning.
commit : b86be844a40c439e44ea6fc974df37b7c2c9c832
author : Noah Misch <noah@leadboat.com>
date : Sat, 27 Jun 2020 22:05:04 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sat, 27 Jun 2020 22:05:04 -0700
Warnings start 10M transactions before xidStopLimit, which is 11M
transactions before wraparound. The sample WARNING output showed a
value greater than 11M, and its HINT message predated commit
25ec228ef760eb91c094cc3b6dea7257cc22ffb5. Hence, the sample was
impossible. Back-patch to 9.5 (all supported versions).
M doc/src/sgml/maintenance.sgml
Fix list of SSL error codes for older OpenSSL versions.
commit : e5f63db995514473f7b3421bc80f8e7715cd6d35
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 27 Jun 2020 13:26:17 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 27 Jun 2020 13:26:17 -0400
Apparently 1.0.1 lacks SSL_R_VERSION_TOO_HIGH and
SSL_R_VERSION_TOO_LOW. Per buildfarm.
M src/backend/libpq/be-secure-openssl.c
M src/interfaces/libpq/fe-secure-openssl.c
Add hints about protocol-version-related SSL connection failures.
commit : e2bcd99be18c67fea575a9789ebafd650e6e1076
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 27 Jun 2020 12:47:58 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 27 Jun 2020 12:47:58 -0400
OpenSSL's native reports about problems related to protocol version
restrictions are pretty opaque and inconsistent. When we get an
SSL error that is plausibly due to this, emit a hint message that
includes the range of SSL protocol versions we (think we) are
allowing. This should at least get the user thinking in the right
direction to resolve the problem, even if the hint isn't totally
accurate, which it might not be for assorted reasons.
Back-patch to v13 where we increased the default minimum protocol
version, thereby increasing the risk of this class of failure.
Patch by me, reviewed by Daniel Gustafsson
Discussion: https://postgr.es/m/a9408304-4381-a5af-d259-e55d349ae4ce@2ndquadrant.com
M src/backend/libpq/be-secure-openssl.c
M src/include/common/openssl.h
M src/interfaces/libpq/fe-secure-openssl.c
Change libpq's default ssl_min_protocol_version to TLSv1.2.
commit : 16412c78403e8ebcb06e34ac1eb74ff8dd299495
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 27 Jun 2020 12:20:33 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 27 Jun 2020 12:20:33 -0400
When we initially created this parameter, in commit ff8ca5fad, we left
the default as "allow any protocol version" on grounds of backwards
compatibility. However, that's inconsistent with the backend's default
since b1abfec82; protocol versions prior to 1.2 are not considered very
secure; and OpenSSL has had TLSv1.2 support since 2012, so the number
of PG servers that need a lesser minimum is probably quite small.
On top of those things, it emerges that some popular distros (including
Debian and RHEL) set MinProtocol=TLSv1.2 in openssl.cnf. Thus, far
from having "allow any protocol version" behavior in practice, what
we actually have as things stand is a platform-dependent lower limit.
So, change our minds and set the min version to TLSv1.2. Anybody
wanting to connect with a new libpq to a pre-2012 server can either
set ssl_min_protocol_version=TLSv1 or accept the fallback to non-SSL.
Back-patch to v13 where the aforementioned patches appeared.
Patch by me, reviewed by Daniel Gustafsson
Discussion: https://postgr.es/m/a9408304-4381-a5af-d259-e55d349ae4ce@2ndquadrant.com
M doc/src/sgml/libpq.sgml
M src/interfaces/libpq/fe-connect.c
Persist slot invalidation correctly
commit : 3b4b541777f0b85df7626623ef78df0ea48ca5dc
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Jun 2020 20:41:29 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 26 Jun 2020 20:41:29 -0400
We failed to save slot to disk after invalidating it, so the state was
lost in case of server restart or crash. Fix by marking it dirty and
flushing.
Also, if the slot is known invalidated we don't need to reason about the
LSN at all -- it's known invalidated. Only test the LSN if the slot is
known not invalidated.
Author: Fujii Masao <masao.fujii@oss.nttdata.com>
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/17a69cfe-f1c1-a416-ee25-ae15427c69eb@oss.nttdata.com
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
doc: PG 13 relnotes; remove FOREIGN keyword item and clarify
commit : 1f601b14e3a7c5ca035cfb59575462004a8c3125
author : Bruce Momjian <bruce@momjian.us>
date : Fri, 26 Jun 2020 18:24:12 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Fri, 26 Jun 2020 18:24:12 -0400
Clarify --include-foreign-data option addition.
Reported-by: Masahiko Sawada, Alvaro Herrera
Discussion: https://postgr.es/m/CA+fd4k62hYtce8VrEMGm6Y+1c24QBgCksXvOaH5kE8PbY+68sA@mail.gmail.com
Backpatch-through: 13 only
M doc/src/sgml/release-13.sgml
Doc: explain that "timestamp - timestamp" applies justify_hours().
commit : 098868b57687ef9c5e3cd9dff469594c6a1c6d10
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Jun 2020 13:54:01 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 26 Jun 2020 13:54:01 -0400
Back-patch to v13; before that, there's not really space for this
kind of detail.
Discussion: https://postgr.es/m/c1696f68-fa8d-7759-6a9c-eb293ab1bbc9@gmx.net
M doc/src/sgml/func.sgml
doc: mention trigger helper functions in CREATE TRIGGER docs
commit : 08671057e025b48136d4eed5477f287ffce217b0
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 25 Jun 2020 18:33:28 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 25 Jun 2020 18:33:28 -0400
Reported-by: petermpallesen@gmail.com
Discussion: https://postgr.es/m/159195294959.673.5752624528747900508@wrigleys.postgresql.org
Backpatch-through: 9.5
M doc/src/sgml/ref/create_trigger.sgml
docs: clarify that CREATE DATABASE does not copy db permissions
commit : 563ed36d5b4819066e13f5272bf1a02cf5dac0bf
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 25 Jun 2020 18:22:44 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 25 Jun 2020 18:22:44 -0400
That is, those database permissions set by GRANT.
Diagnosed-by: Joseph Nahmias
Discussion: https://postgr.es/m/20200614072613.GA21852@nahmias.net
Backpatch-through: 9.5
M doc/src/sgml/ref/create_database.sgml
Fix misuse of table_index_fetch_tuple_check().
commit : 8c2010f12344ed8834c6f63406a78e5843ebec69
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 25 Jun 2020 10:55:26 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 25 Jun 2020 10:55:26 -0700
Commit 0d861bbb, which added deduplication to nbtree, had
_bt_check_unique() pass a TID to table_index_fetch_tuple_check() that
isn't safe to mutate. table_index_fetch_tuple_check()'s tid argument is
modified when the TID in question is not the latest visible tuple in a
hot chain, though this wasn't documented.
To fix, go back to using a local copy of the TID in _bt_check_unique(),
and update comments above table_index_fetch_tuple_check().
Backpatch: 13-, where B-Tree deduplication was introduced.
M src/backend/access/nbtree/nbtinsert.c
M src/backend/access/table/tableam.c
M src/include/access/tableam.h
Doc: correct nitpicky mistakes in array_position/array_positions examples.
commit : 185c6bc4aef1201b7d0f2c4e9c8893c4a663dfd4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Jun 2020 13:28:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 25 Jun 2020 13:28:30 -0400
Daniel Gustafsson and Erik Rijkers, per report from nick@cleaton
Discussion: https://postgr.es/m/159275646273.679.16940709892308114570@wrigleys.postgresql.org
M doc/src/sgml/array.sgml
Remove erroneous assertion from pg_copy_logical_replication_slot().
commit : 126c8fcec790652dd0cb755fdeedf2c02c8d8079
author : Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Jun 2020 11:13:13 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Thu, 25 Jun 2020 11:13:13 +0900
If restart_lsn of logical replication slot gets behind more than
max_slot_wal_keep_size from the current LSN, the logical replication slot
would be invalidated and its restart_lsn is reset to an invalid LSN.
If this logical replication slot with an invalid restart_lsn was specified as
the source slot in pg_copy_logical_replication_slot(), the function caused
the assertion failure unexpectedly.
This assertion was added because restart_lsn should not be invalid before.
But in v13, it can be invalid thanks to max_slot_wal_keep_size. So since this
assertion is no longer useful, this commit removes it.
This commit also changes the errcode in the error message that
pg_copy_logical_replication_slot() emits when the slot with an invalid
restart_lsn is specified, to more appropriate one.
Back-patch to v13 where max_slot_wal_keep_size was added and
the assertion was no longer valid.
Author: Fujii Masao
Reviewed-by: Alvaro Herrera, Kyotaro Horiguchi
Discussion: https://postgr.es/m/f91de4fb-a7ab-b90e-8132-74796e049d51@oss.nttdata.com
M src/backend/replication/slotfuncs.c
Fix compiler warning induced by commit d8b15eeb8.
commit : 086bef8ac8b3635e7af94ac41e92dfc016b87e90
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 24 Jun 2020 15:47:30 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 24 Jun 2020 15:47:30 -0400
I forgot that INT64_FORMAT can't be used with sscanf on Windows.
Use the same trick of sscanf'ing into a temp variable as we do in
some other places in zic.c.
The upstream IANA code avoids the portability problem by relying on
<inttypes.h>'s SCNdFAST64 macro. Once we're requiring C99 in all
branches, we should do likewise and drop this set of diffs from
upstream. For now, though, a hack seems fine, since we do not
actually care about leapseconds anyway.
Discussion: https://postgr.es/m/4e5d1a5b-143e-e70e-a99d-a3b01c1ae7c3@2ndquadrant.com
M src/timezone/zic.c
Adjust max_slot_wal_keep_size behavior per review
commit : 6f7a862bed3a49283c74c0adf207172002e3e03c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 24 Jun 2020 14:23:39 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 24 Jun 2020 14:23:39 -0400
In pg_replication_slot, change output from normal/reserved/lost to
reserved/extended/unreserved/ lost, which better expresses the possible
states particularly near the time where segments are no longer safe but
checkpoint has not run yet.
Under the new definition, reserved means the slot is consuming WAL
that's still under the normal WAL size constraints; extended means it's
consuming WAL that's being protected by wal_keep_segments or the slot
itself, whose size is below max_slot_wal_keep_size; unreserved means the
WAL is no longer safe, but checkpoint has not yet removed those files.
Such as slot is in imminent danger, but can still continue for a little
while and may catch up to the reserved WAL space.
Also, there were some bugs in the calculations used to report the
status; fixed those.
Backpatch to 13.
Reported-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20200616.120236.1809496990963386593.horikyota.ntt@gmail.com
M doc/src/sgml/catalogs.sgml
M src/backend/access/transam/xlog.c
M src/backend/replication/slotfuncs.c
M src/include/access/xlog.h
M src/test/recovery/t/019_replslot_limit.pl
Save slot's restart_lsn when invalidated due to size
commit : 12e52ba5a76e56aacdfbbb269e6b45c53d80c477
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 24 Jun 2020 14:15:17 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 24 Jun 2020 14:15:17 -0400
We put it aside as invalidated_at, which let us show "lost" in
pg_replication slot. Prior to this change, the state value was reported
as NULL.
Backpatch to 13.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20200617.101707.1735599255100002667.horikyota.ntt@gmail.com
Discussion: https://postgr.es/m/20200407.120905.1507671100168805403.horikyota.ntt@gmail.com
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/include/access/xlog.h
M src/include/replication/slot.h
M src/test/recovery/t/019_replslot_limit.pl
Add parens to ConvertToXSegs macro
commit : 411493d701e2f97e778dc1ff14fb7169eea2e94c
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 24 Jun 2020 14:00:37 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Wed, 24 Jun 2020 14:00:37 -0400
The current definition is dangerous. No bugs exist in our code at
present, but backpatch to 11 nonetheless where it was introduced.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
M src/backend/access/transam/xlog.c
Stamp 13beta2.
commit : bc4d7817e0cbd26998ebaa682772bf6bc579c302
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Jun 2020 17:16:25 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Jun 2020 17:16:25 -0400
M configure
M configure.in
Doc fixup for hashagg_avoid_disk_plan GUC.
commit : d33f33539d7f90d024a1dcb73b74c15b07349be8
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 22 Jun 2020 12:14:55 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 22 Jun 2020 12:14:55 -0700
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20200620220402.GZ17995@telsasoft.com
Backport-through: 13
M doc/src/sgml/config.sgml
Undo double-quoting of index names in non-text EXPLAIN output formats.
commit : 57f8b9913b912f2bdfe24b73d44b9713e328ee2e
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Jun 2020 11:46:41 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Mon, 22 Jun 2020 11:46:41 -0400
explain_get_index_name() applied quote_identifier() to the index name.
This is fine for text output, but the non-text output formats all have
their own quoting conventions and would much rather start from the
actual index name. For example in JSON you'd get something like
"Index Name": "\"My Index\"",
which is surely not desirable, especially when the same does not
happen for table names. Hence, move the responsibility for applying
quoting out to the callers, where it can go into already-existing
special code paths for text format.
This changes the API spec for users of explain_get_index_name_hook:
before, they were supposed to apply quote_identifier() if necessary,
now they should not. Research suggests that the only publicly
available user of the hook is hypopg, and it actually forgot to
apply quoting anyway, so it's fine. (In any case, there's no
behavioral change for the output of a hook as seen in non-text
EXPLAIN formats, so this won't break any case that programs should
be relying on.)
Digging in the commit logs, it appears that quoting was included in
explain_get_index_name's duties when commit 604ffd280 invented it;
and that was fine at the time because we only had text output format.
This should have been rethought when non-text formats were invented,
but it wasn't.
This is a fairly clear bug for users of non-text EXPLAIN formats,
so back-patch to all supported branches.
Per bug #16502 from Maciek Sakrejda. Patch by me (based on
investigation by Euler Taveira); thanks to Julien Rouhaud for review.
Discussion: https://postgr.es/m/16502-57bd1c9f913ed1d1@postgresql.org
M src/backend/commands/explain.c
Translation updates
commit : 793e5ad3cb7f8a8345881c7057618228546de3c6
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 22 Jun 2020 14:08:30 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 22 Jun 2020 14:08:30 +0200
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 434134899af310153f7511ccaa3f376e4c817e66
M src/backend/po/de.po
M src/backend/po/sv.po
M src/bin/initdb/po/es.po
M src/bin/pg_dump/po/es.po
M src/bin/pg_rewind/po/sv.po
M src/bin/psql/po/de.po
M src/bin/psql/po/sv.po
M src/bin/scripts/po/de.po
M src/bin/scripts/po/sv.po
M src/interfaces/ecpg/preproc/po/es.po
M src/interfaces/libpq/po/de.po
M src/interfaces/libpq/po/sv.po
M src/pl/plpgsql/src/po/de.po
M src/pl/plpgsql/src/po/sv.po
Language fixes for docs related to opclass options
commit : 70004a2a0c52e05f4aa67541fb165715a3981204
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 21 Jun 2020 04:48:03 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 21 Jun 2020 04:48:03 +0300
Discussion: https://postgr.es/m/20200620232145.GB17995%40telsasoft.com
Author: Justin Pryzby
Backpatch-through: 13
M doc/src/sgml/brin.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/xindex.sgml
Doc: Tweak description of B-Tree duplicate tuples.
commit : f7e4989d1c65c376ca4aba2d39dc81cd1eaefe67
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 20 Jun 2020 17:34:06 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 20 Jun 2020 17:34:06 -0700
Defining duplicates as "close by" to each other was unclear. Simplify
the definition.
Backpatch: 13-, where deduplication was introduced (by commit 0d861bbb)
M doc/src/sgml/btree.sgml
Minor corrections to docs related to opclass options
commit : b56d91ebd2bef20f9adbcc61c1279083a91bdf3e
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 21 Jun 2020 00:35:42 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sun, 21 Jun 2020 00:35:42 +0300
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzmwhYbxuoL0WjTLaiCxW3gj6qadeNpBhWAo_KZsE5-FGw%40mail.gmail.com
M doc/src/sgml/btree.sgml
M doc/src/sgml/spgist.sgml
Fix masking of SP-GiST pages during xlog consistency check
commit : 39aafc88c4b4ac281df8b2c2b8be72d4e4d99e9f
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 20 Jun 2020 17:34:51 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 20 Jun 2020 17:34:51 +0300
spg_mask() didn't take into account that pd_lower equal to SizeOfPageHeaderData
is still valid value. This commit fixes that. Backpatch to 11, where
spg_mask() pg_lower check was introduced.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/20200615131405.GM52676%40paquier.xyz
Backpatch-through: 11
M src/backend/access/spgist/spgxlog.c
Add documentation for opclass options
commit : e6c6f427e356e3706ce2f0ae7e7e94e5501bbc13
author : Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 20 Jun 2020 13:34:54 +0300
committer: Alexander Korotkov <akorotkov@postgresql.org>
date : Sat, 20 Jun 2020 13:34:54 +0300
911e7020770 added opclass options and adjusted documentation for each
particular affected opclass. However, documentation for extendability was
not adjusted. This commit adjusts documentation for interfaces of index AMs
and opclasses.
Discussion: https://postgr.es/m/CAH2-WzmQnW6%2Bz5F9AW%2BSz%2BzEcEvXofTwh_A9J3%3D_WA-FBP0wYg%40mail.gmail.com
Author: Alexander Korotkov
Reported-by: Peter Geoghegan
Reviewed-by: Peter Geoghegan
M doc/src/sgml/brin.sgml
M doc/src/sgml/btree.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/indexam.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/xindex.sgml
Ensure write failure reports no-disk-space
commit : e74559c9763049ff4d3edd26817bad58c13113a1
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 19 Jun 2020 16:46:07 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 19 Jun 2020 16:46:07 -0400
A few places calling fwrite and gzwrite were not setting errno to ENOSPC
when reporting errors, as is customary; this led to some failures being
reported as
"could not write file: Success"
which makes us look silly. Make a few of these places in pg_dump and
pg_basebackup use our customary pattern.
Backpatch-to: 9.5
Author: Justin Pryzby <pryzby@telsasoft.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/20200611153753.GU14879@telsasoft.com
M src/bin/pg_basebackup/pg_basebackup.c
M src/bin/pg_dump/pg_backup_directory.c
Future-proof regression tests against possibly-missing posixrules file.
commit : 577dcf890cdb2621cf21ded1a2b6c96c40441f3d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Jun 2020 13:55:21 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 19 Jun 2020 13:55:21 -0400
The IANA time zone folk have deprecated use of a "posixrules" file in
the tz database. While for now it's our choice whether to keep
supplying one in our own builds, installations built with
--with-system-tzdata will soon be needing to cope with that file not
being present, at least on some platforms.
This causes a problem for the horology test, which expected the
nonstandard POSIX zone spec "CST7CDT" to apply pre-2007 US daylight
savings rules. That does happen if the posixrules file supplies such
information, but otherwise the test produces undesired results.
To fix, add an explicit transition date rule that matches 2005 practice.
(We could alternatively have switched the test to use some real time
zone, but it seems useful to have coverage of this type of zone spec.)
While at it, update a documentation example that also relied on
"CST7CDT"; use a real-world zone name instead. Also, document why
the zone names EST5EDT, CST6CDT, MST7MDT, PST8PDT aren't subject to
similar failures when "posixrules" is missing.
Back-patch to all supported branches, since the hazard is the same
for all.
Discussion: https://postgr.es/m/1665379.1592581287@sss.pgh.pa.us
M doc/src/sgml/datetime.sgml
M doc/src/sgml/func.sgml
M src/test/regress/expected/horology.out
M src/test/regress/sql/horology.sql
Adjust some glossary terms
commit : 91a890bd7fef1cd8bfe3c8832eea114290f16b02
author : Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 19 Jun 2020 12:55:43 -0400
committer: Alvaro Herrera <alvherre@alvh.no-ip.org>
date : Fri, 19 Jun 2020 12:55:43 -0400
Mostly in response to Jürgen Purtz critique of previous definitions,
though I added many other changes.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Jürgen Purtz <juergen@purtz.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Erik Rijkers <er@xs4all.nl>
Discussion: https://postgr.es/m/c1e06008-2132-30f4-9b38-877e8683d418@purtz.de
M doc/src/sgml/glossary.sgml
Fix deduplication "single value" strategy bug.
commit : dedb92d4a3adc6b5165a619383739ab05d24b24d
author : Peter Geoghegan <pg@bowt.ie>
date : Fri, 19 Jun 2020 08:57:23 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Fri, 19 Jun 2020 08:57:23 -0700
It was possible for deduplication's single value strategy to mistakenly
believe that a very small duplicate tuple counts as one of the six large
tuples that it aims to leave behind after the page finally splits. This
could cause slightly suboptimal space utilization with very low
cardinality indexes, though only under fairly narrow conditions.
To fix, be particular about what kind of tuple counts as a
maxpostingsize-capped tuple. This avoids confusion in the event of a
small tuple that gets "wedged" between two large tuples, where all
tuples on the page are duplicates of the same value.
Discussion: https://postgr.es/m/CAH2-Wz=Y+sgSFc-O3LpiZX-POx2bC+okec2KafERHuzdVa7-rQ@mail.gmail.com
Backpatch: 13-, where deduplication was introduced (by commit 0d861bbb)
M src/backend/access/nbtree/nbtdedup.c
M src/backend/access/nbtree/nbtsort.c
M src/backend/access/nbtree/nbtxlog.c
M src/include/access/nbtree.h
Fix issues in invalidation of obsolete replication slots.
commit : 08aa3151e7308556130c644c237fa4b20dfd6eba
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Jun 2020 17:15:52 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 19 Jun 2020 17:15:52 +0900
This commit fixes the following issues.
1. There is the case where the slot is dropped while trying to invalidate it.
InvalidateObsoleteReplicationSlots() did not handle this case, and
which could cause checkpoint to fail.
2. InvalidateObsoleteReplicationSlots() could emit the same log message
multiple times unnecessary. It should be logged only once.
3. When marking the slot as used, we always searched the target slot from
all the replication slots even if we already found it. This could cause
useless waste of cycles.
Back-patch to v13 where these issues were added as a part of
max_slot_wal_keep_size code.
Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Alvaro Herrera
Discussion: https://postgr.es/m/66c05b67-3396-042c-1b41-bfa6c3ddcf82@oss.nttdata.com
M src/backend/replication/slot.c
Fix EXPLAIN ANALYZE for parallel HashAgg plans
commit : bdee4af8e07648008fe522fc5a562db453be5ad7
author : David Rowley <drowley@postgresql.org>
date : Fri, 19 Jun 2020 17:25:07 +1200
committer: David Rowley <drowley@postgresql.org>
date : Fri, 19 Jun 2020 17:25:07 +1200
Since 1f39bce02, HashAgg nodes have had the ability to spill to disk when
memory consumption exceeds work_mem. That commit added new properties to
EXPLAIN ANALYZE to show the maximum memory usage and disk usage, however,
it didn't quite go as far as showing that information for parallel
workers. Since workers may have experienced something very different from
the main process, we should show this information per worker, as is done
in Sort.
Reviewed-by: Justin Pryzby
Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/CAApHDvpEKbfZa18mM1TD7qV6PG+w97pwCWq5tVD0dX7e11gRJw@mail.gmail.com
Backpatch-through: 13, where the hashagg spilling code was added.
M src/backend/commands/explain.c
M src/backend/executor/execParallel.c
M src/backend/executor/nodeAgg.c
M src/include/executor/nodeAgg.h
M src/include/nodes/execnodes.h
Fix deadlock danger when atomic ops are done under spinlock.
commit : 5fffa8fce37b981e1a5bb79affce9a856e021265
author : Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 16:50:37 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 16:50:37 -0700
This was a danger only for --disable-spinlocks in combination with
atomic operations unsupported by the current platform.
While atomics.c was careful to signal that a separate semaphore ought
to be used when spinlock emulation is active, spin.c didn't actually
implement that mechanism. That's my (Andres') fault, it seems to have
gotten lost during the development of the atomic operations support.
Fix that issue and add test for nesting atomic operations inside a
spinlock.
Author: Andres Freund
Discussion: https://postgr.es/m/20200605023302.g6v3ydozy5txifji@alap3.anarazel.de
Backpatch: 9.5-
M src/backend/storage/lmgr/spin.c
M src/test/regress/regress.c
Add basic spinlock tests to regression tests.
commit : 59225dcefef278415aef64c3b96f84616b95661e
author : Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 16:36:51 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 16:36:51 -0700
As s_lock_test, the already existing test for spinlocks, isn't run in
an automated fashion (and doesn't test a normal backend environment),
adding tests that are run as part of a normal regression run is a good
idea. Particularly in light of several recent and upcoming spinlock
related fixes.
Currently the new tests are run as part of the pre-existing
test_atomic_ops() test. That perhaps can be quibbled about, but for
now seems ok.
The only operations that s_lock_test tests but the new tests don't are
the detection of a stuck spinlock and S_LOCK_FREE (which is otherwise
unused, not implemented on all platforms, and will be removed).
This currently contains a test for more than INT_MAX spinlocks (only
run with --disable-spinlocks), to ensure the recent commit fixing a
bug with more than INT_MAX spinlock initializations is correct. That
test is somewhat slow, so we might want to disable it after a few
days.
It might be worth retiring s_lock_test after this. The added coverage
of a stuck spinlock probably isn't worth the added complexity?
Author: Andres Freund
Discussion: https://postgr.es/m/20200606023103.avzrctgv7476xj7i@alap3.anarazel.de
M src/test/regress/regress.c
Doc: document POSIX-style time zone specifications in full.
commit : c10dc2d11791cc18ceea78caa94eb4b651090259
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Jun 2020 16:27:18 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 18 Jun 2020 16:27:18 -0400
We'd glossed over most of this complexity for years, but it's hard
to avoid writing it all down now, so that we can explain what happens
when there's no "posixrules" file in the IANA time zone database.
That was at best a tiny minority situation till now, but it's likely
to become quite common in the future, so we'd better explain it.
Nonetheless, we don't really encourage people to use POSIX zone specs;
picking a named zone is almost always what you really want, unless
perhaps you're stuck with an out-of-date zone database. Therefore,
let's shove all this detail into an appendix.
Patch by me; thanks to Robert Haas for help with some awkward wording.
Discussion: https://postgr.es/m/1390.1562258309@sss.pgh.pa.us
M doc/src/sgml/datatype.sgml
M doc/src/sgml/datetime.sgml
Fix oldest xmin and LSN computation across repslots after advancing
commit : 43e70addf5a65f1b99c286f82e2e4970b0c2fda7
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 18 Jun 2020 16:35:29 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 18 Jun 2020 16:35:29 +0900
Advancing a replication slot did not recompute the oldest xmin and LSN
values across replication slots, preventing resource removal like
segments not recycled at checkpoint time. The original commit that
introduced the slot advancing in 9c7d06d never did the update of those
oldest values, and b0afdca removed this code.
This commit adds a TAP test to check segment recycling with advancing
for physical slots, enforcing an extra segment switch before advancing
to check if the segment gets correctly recycled after a checkpoint.
Reported-by: Andres Freund
Reviewed-by: Alexey Kondratov, Kyptaro Horiguchi
Discussion: https://postgr.es/m/20200609171904.kpltxxvjzislidks@alap3.anarazel.de
Backpatch-through: 11
M src/backend/replication/slotfuncs.c
M src/test/recovery/t/001_stream_rep.pl
doc: Fix formatting typo
commit : f2236d087eb8df9f15c016c02c92aa2bed7c2889
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 18 Jun 2020 03:22:26 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 18 Jun 2020 03:22:26 +0200
M doc/src/sgml/release-13.sgml
Sync our copy of the timezone library with IANA release tzcode2020a.
commit : 484a57643e02b7df2bb9085603772b33511c6668
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Jun 2020 18:29:29 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 17 Jun 2020 18:29:29 -0400
This absorbs a leap-second-related bug fix in localtime.c, and
teaches zic to handle an expiration marker in the leapseconds file.
Neither are of any interest to us (for the foreseeable future
anyway), but we need to stay more or less in sync with upstream.
Also adjust some over-eager changes in the README from commit 957338418.
I have no intention of making changes that require C99 in this code,
until such time as all the live back branches require C99. Otherwise
back-patching will get too exciting.
For the same reason, absorb assorted whitespace and other cosmetic
changes from HEAD into the back branches; mostly this reflects use of
improved versions of pgindent.
All in all then, quite a boring update. But I figured I'd get it
done while I was looking at this code.
M src/timezone/README
M src/timezone/localtime.c
M src/timezone/zic.c
Fix nbtree.h dedup state comment.
commit : 6b296102920323f360d53457ecda5179284cca8c
author : Peter Geoghegan <pg@bowt.ie>
date : Wed, 17 Jun 2020 15:23:54 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Wed, 17 Jun 2020 15:23:54 -0700
Oversight in commit 0d861bbb.
M src/include/access/nbtree.h
spinlock emulation: Fix bug when more than INT_MAX spinlocks are initialized.
commit : 276bdc93924afb2bd793627f49a9e7edd4172b63
author : Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 15:25:49 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 15:25:49 -0700
Once the counter goes negative we ended up with spinlocks that errored
out on first use (due to check in tas_sema).
Author: Andres Freund
Reviewed-By: Robert Haas
Discussion: https://postgr.es/m/20200606023103.avzrctgv7476xj7i@alap3.anarazel.de
Backpatch: 9.5-
M src/backend/storage/lmgr/spin.c
Avoid potential spinlock in a signal handler as part of global barriers.
commit : 09bff91b316e90bf7f523593c1e8000c772cbe52
author : Andres Freund <andres@anarazel.de>
date : Mon, 15 Jun 2020 18:23:10 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 15 Jun 2020 18:23:10 -0700
On platforms without support for 64bit atomic operations where we also
cannot rely on 64bit reads to have single copy atomicity, such atomics
are implemented using a spinlock based fallback. That means it's not
safe to even read such atomics from within a signal handler (since the
signal handler might run when the spinlock already is held).
To avoid this issue defer global barrier processing out of the signal
handler. Instead of checking local / shared barrier generation to
determine whether to set ProcSignalBarrierPending, introduce
PROCSIGNAL_BARRIER and always set ProcSignalBarrierPending when
receiving such a signal. Additionally avoid redundant work in
ProcessProcSignalBarrier if ProcSignalBarrierPending is unnecessarily.
Also do a small amount of other polishing.
Author: Andres Freund
Reviewed-By: Robert Haas
Discussion: https://postgr.es/m/20200609193723.eu5ilsjxwdpyxhgz@alap3.anarazel.de
Backpatch: 13-, where the code was introduced.
M src/backend/storage/ipc/procsignal.c
M src/include/storage/procsignal.h
Doc: fix copy-and-pasteo in ecpg docs.
commit : 7f932f77c7eb068772355c76cfa48bc5d0260e2f
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Jun 2020 16:41:11 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 16 Jun 2020 16:41:11 -0400
The synopsis for PGTYPESinterval_free() used the wrong name.
Discussion: https://postgr.es/m/159231203030.679.3061023914894071953@wrigleys.postgresql.org
M doc/src/sgml/ecpg.sgml
Fix file reference in nls.mk
commit : 9c25a873d631036a92036394863eba1f4a3f3cd5
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 16 Jun 2020 17:25:20 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 16 Jun 2020 17:25:20 +0200
Broken by move of fe_archive.c to fe_utils.
M src/bin/pg_rewind/nls.mk
Fix buffile.c error handling.
commit : 3e0b08c404b2a7d799db78eb942a01534ac7926b
author : Thomas Munro <tmunro@postgresql.org>
date : Tue, 16 Jun 2020 13:50:56 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Tue, 16 Jun 2020 13:50:56 +1200
Convert buffile.c error handling to use ereport. This fixes cases where
I/O errors were indistinguishable from EOF or not reported. Also remove
"%m" from error messages where errno would be bogus. While we're
modifying those strings, add block numbers and short read byte counts
where appropriate.
Back-patch to all supported releases.
Reported-by: Amit Khandekar <amitdkhan.pg@gmail.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Ibrar Ahmed <ibrar.ahmad@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKGJE04G%3D8TLK0DLypT_27D9dR8F1RQgNp0jK6qR0tZGWOw%40mail.gmail.com
M src/backend/access/gist/gistbuildbuffers.c
M src/backend/executor/nodeHashjoin.c
M src/backend/replication/backup_manifest.c
M src/backend/storage/file/buffile.c
M src/backend/utils/sort/logtape.c
M src/backend/utils/sort/sharedtuplestore.c
M src/backend/utils/sort/tuplestore.c
pg_upgrade: set vacuum_defer_cleanup_age to zero
commit : a2c72851a898170ee1f2e7c21c1bf9086dec2d5c
author : Bruce Momjian <bruce@momjian.us>
date : Mon, 15 Jun 2020 20:59:40 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Mon, 15 Jun 2020 20:59:40 -0400
Non-zero vacuum_defer_cleanup_age values cause pg_upgrade freezing of
the system catalogs to be incomplete, or do nothing. This will cause
the upgrade to fail in confusing ways.
Reported-by: Laurenz Albe
Discussion: https://postgr.es/m/7d6f6c22ba05ce0c526e9e8b7bfa8105e7da45e6.camel@cybertec.at
Backpatch-through: 9.5
M src/bin/pg_upgrade/server.c
Doc: Add references for SI and SSI.
commit : 4701efa9f741aa0ca38cfe922dfcaef1749b5b02
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Jun 2020 11:33:13 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 15 Jun 2020 11:33:13 +1200
Our documentation failed to point out that REPEATABLE READ is really
snapshot isolation, which might be important to some users. Point to
the standard reference paper for this complicated topic.
Likewise, add a reference to the VLDB paper about PostgreSQL SSI, for
technical information about our SSI implementation and how it compares
to S2PL.
While here, add a note about catalog access using a lower isolation
level, per recent user complaint.
Back-patch to all releases.
Reported-by: Kyle Kingsbury <aphyr@jepsen.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Reviewed-by: Tatsuo Ishii <ishii@sraoss.co.jp>
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443%40jepsen.io
Discussion: https://postgr.es/m/16454-9408996bb1750faf%40postgresql.org
M doc/src/sgml/biblio.sgml
M doc/src/sgml/mvcc.sgml
Fix behavior of float aggregates for single Inf or NaN inputs.
commit : 33dd9bb3b0a88981f18a10d89720b4e40d8876ba
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Jun 2020 13:43:24 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sat, 13 Jun 2020 13:43:24 -0400
When there is just one non-null input value, and it is infinity or NaN,
aggregates such as stddev_pop and covar_pop should produce a NaN
result, because the calculation is not well-defined. They used to do
so, but since we adopted Youngs-Cramer aggregation in commit e954a727f,
they produced zero instead. That's an oversight, so fix it. Add tests
exercising these edge cases.
Affected aggregates are
var_pop(double precision)
stddev_pop(double precision)
var_pop(real)
stddev_pop(real)
regr_sxx(double precision,double precision)
regr_syy(double precision,double precision)
regr_sxy(double precision,double precision)
regr_r2(double precision,double precision)
regr_slope(double precision,double precision)
regr_intercept(double precision,double precision)
covar_pop(double precision,double precision)
corr(double precision,double precision)
Back-patch to v12 where the behavior change was accidentally introduced.
Report and patch by me; thanks to Dean Rasheed for review.
Discussion: https://postgr.es/m/353062.1591898766@sss.pgh.pa.us
M src/backend/utils/adt/float.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Silence _bt_check_unique compiler warning.
commit : e745bcc00149fe3c35ba1123800e0beb948e3678
author : Peter Geoghegan <pg@bowt.ie>
date : Sat, 13 Jun 2020 09:33:31 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Sat, 13 Jun 2020 09:33:31 -0700
Reported-By: Tom Lane
Discussion: https://postgr.es/m/841649.1592065060@sss.pgh.pa.us
M src/backend/access/nbtree/nbtinsert.c
Add missing extern keyword for a couple of numutils functions
commit : 095f2d95c92704747d84d499a33b527af42bb08e
author : David Rowley <drowley@postgresql.org>
date : Sat, 13 Jun 2020 11:28:12 +1200
committer: David Rowley <drowley@postgresql.org>
date : Sat, 13 Jun 2020 11:28:12 +1200
In passing, also remove a few surplus empty lines from pg_ltoa and
pg_ulltoa_n in numutils.c
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y2ou3xuh.fsf@news-spur.riddles.org.uk
Backpatch-through: 13, where these changes were introduced
M src/backend/utils/adt/numutils.c
M src/include/utils/builtins.h
Improve comments for [Heap]CheckForSerializableConflictOut().
commit : 44eff28410598ef86d3d8bd812439aabf19f7ee0
author : Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Jun 2020 10:44:32 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Fri, 12 Jun 2020 10:44:32 +1200
Rewrite the documentation of these functions, in light of recent bug fix
commit 5940ffb2.
Back-patch to 13 where the check-for-conflict-out code was split up into
AM-specific and generic parts, and new documentation was added that now
looked wrong.
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443%40jepsen.io
M src/backend/access/heap/heapam.c
M src/backend/storage/lmgr/predicate.c
doc: remove xreflabels used in PG 13 relnotes
commit : db680fd82ede99d9a5224a1d316a64d763be1acc
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 11 Jun 2020 18:27:59 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 11 Jun 2020 18:27:59 -0400
xreflabels were removed in a previous commit
Discussion: https://postgr.es/m/8315c0ca-7758-8823-fcb6-f37f9413e6b6@2ndquadrant.com
Backpatch-through: 13 only
M doc/src/sgml/release-13.sgml
doc: remove xreflabels from commits 75fcdd2ae2 and 85af628da5
commit : c04612040165582f60cbcfe8ca1771598c9f3a05
author : Bruce Momjian <bruce@momjian.us>
date : Thu, 11 Jun 2020 18:25:46 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Thu, 11 Jun 2020 18:25:46 -0400
xreflabels prevent references to the chapter numbers of sections id's.
It should only be used in specific cases.
Discussion: https://postgr.es/m/8315c0ca-7758-8823-fcb6-f37f9413e6b6@2ndquadrant.com
Backpatch-through: 9.5
M doc/src/sgml/ecpg.sgml
M doc/src/sgml/geqo.sgml
M doc/src/sgml/gin.sgml
M doc/src/sgml/gist.sgml
M doc/src/sgml/libpq.sgml
M doc/src/sgml/maintenance.sgml
M doc/src/sgml/plperl.sgml
M doc/src/sgml/plpgsql.sgml
M doc/src/sgml/plpython.sgml
M doc/src/sgml/spgist.sgml
M doc/src/sgml/spi.sgml
M doc/src/sgml/storage.sgml
M doc/src/sgml/vacuumlo.sgml
Fix doc build, broken by 13e0fa7a.
commit : 6fbfa4eb244eab47d67fba4258c4af777729f119
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 11 Jun 2020 14:44:31 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 11 Jun 2020 14:44:31 -0700
Reported-by: Peter Geoghegan
M doc/src/sgml/release-13.sgml
Fix mishandling of NaN counts in numeric_[avg_]combine.
commit : ee788ba99011c9d1e8f6f352acc0b0d19350fff6
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Jun 2020 17:38:42 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 11 Jun 2020 17:38:42 -0400
When merging two NumericAggStates, the code missed adding the new
state's NaNcount unless its N was also nonzero; since those counts
are independent, this is wrong.
This would only have visible effect if some partial aggregate scans
found only NaNs while earlier ones found only non-NaNs; then we could
end up falsely deciding that there were no NaNs and fail to return a
NaN final result as expected. That's pretty improbable, so it's no
surprise this hasn't been reported from the field. Still, it's a bug.
I didn't try to produce a regression test that would show the bug,
but I did notice that these functions weren't being reached at all
in our regression tests, so I improved the tests to at least
exercise them. With these additions, I see pretty complete code
coverage on the aggregation-related functions in numeric.c.
Back-patch to 9.6 where this code was introduced. (I only added
the improved test case as far back as v10, though, since the
relevant part of aggregates.sql isn't there at all in 9.6.)
M src/backend/utils/adt/numeric.c
M src/test/regress/expected/aggregates.out
M src/test/regress/sql/aggregates.sql
Rework HashAgg GUCs.
commit : 13e0fa7ae50cd0e91158877dba37098492b234e8
author : Jeff Davis <jdavis@postgresql.org>
date : Thu, 11 Jun 2020 11:58:16 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Thu, 11 Jun 2020 11:58:16 -0700
Eliminate enable_groupingsets_hash_disk, which was primarily useful
for testing grouping sets that use HashAgg and spill. Instead, hack
the table stats to convince the planner to choose hashed aggregation
for grouping sets that will spill to disk. Suggested by Melanie
Plageman.
Rename enable_hashagg_disk to hashagg_avoid_disk_plan, and invert the
meaning of on/off. The new name indicates more strongly that it only
affects the planner. Also, the word "avoid" is less definite, which
should avoid surprises when HashAgg still needs to use the
disk. Change suggested by Justin Pryzby, though I chose a different
GUC name.
Discussion: https://postgr.es/m/CAAKRu_aisiENMsPM2gC4oUY1hHG3yrCwY-fXUg22C6_MJUwQdA%40mail.gmail.com
Discussion: https://postgr.es/m/20200610021544.GA14879@telsasoft.com
Backpatch-through: 13
M doc/src/sgml/config.sgml
M src/backend/optimizer/path/costsize.c
M src/backend/optimizer/plan/planner.c
M src/backend/utils/misc/guc.c
M src/include/optimizer/cost.h
M src/test/regress/expected/aggregates.out
M src/test/regress/expected/groupingsets.out
M src/test/regress/expected/sysviews.out
M src/test/regress/sql/aggregates.sql
M src/test/regress/sql/groupingsets.sql
Avoid update conflict out serialization anomalies.
commit : 6df7105e5d50460623421d00f24aaa46b66fa570
author : Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Jun 2020 10:09:45 -0700
committer: Peter Geoghegan <pg@bowt.ie>
date : Thu, 11 Jun 2020 10:09:45 -0700
SSI's HeapCheckForSerializableConflictOut() test failed to correctly
handle conditions involving a concurrently inserted tuple which is later
concurrently updated by a separate transaction . A SELECT statement
that called HeapCheckForSerializableConflictOut() could end up using the
same XID (updater's XID) for both the original tuple, and the successor
tuple, missing the XID of the xact that created the original tuple
entirely. This only happened when neither tuple from the chain was
visible to the transaction's MVCC snapshot.
The observable symptoms of this bug were subtle. A pair of transactions
could commit, with the later transaction failing to observe the effects
of the earlier transaction (because of the confusion created by the
update to the non-visible row). This bug dates all the way back to
commit dafaa3ef, which added SSI.
To fix, make sure that we check the xmin of concurrently inserted tuples
that happen to also have been updated concurrently.
Author: Peter Geoghegan
Reported-By: Kyle Kingsbury
Reviewed-By: Thomas Munro
Discussion: https://postgr.es/m/db7b729d-0226-d162-a126-8a8ab2dc4443@jepsen.io
Backpatch: All supported versions
M src/backend/access/heap/heapam.c
A src/test/isolation/expected/update-conflict-out.out
M src/test/isolation/isolation_schedule
A src/test/isolation/specs/update-conflict-out.spec
Fix typos.
commit : c4d5706db298f5a02ffd321c4605a7f8746b5428
author : Amit Kapila <akapila@postgresql.org>
date : Thu, 11 Jun 2020 14:10:43 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Thu, 11 Jun 2020 14:10:43 +0530
Reported-by: John Naylor
Author: John Naylor
Backpatch-through: 9.5
Discussion: https://postgr.es/m/CACPNZCtRuvs6G+EYqejhVJgBq2AKeZdXRVJsbX4syhO9gn5SNQ@mail.gmail.com
M src/test/regress/input/largeobject.source
M src/test/regress/output/largeobject.source
M src/test/regress/output/largeobject_1.source
Move frontend-side archive APIs from src/common/ to src/fe_utils/
commit : 8d8b89266ca0328d78df319bacd1e809631f2acc
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Jun 2020 15:48:56 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 11 Jun 2020 15:48:56 +0900
fe_archive.c was compiled only for the frontend in src/common/, but as
it will never share anything with the backend, it makes most sense to
move this file to src/fe_utils/.
Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/e9766d71-8655-ac86-bdf6-77e0e7169977@2ndquadrant.com
Backpatch-through: 13
M src/bin/pg_rewind/parsexlog.c
M src/common/Makefile
M src/fe_utils/Makefile
R094 src/common/fe_archive.c src/fe_utils/archive.c
R091 src/include/common/fe_archive.h src/include/fe_utils/archive.h
M src/tools/msvc/Mkvcbuild.pm
Update description of parameter password_encryption
commit : d6d3f8bc8433f00a43eaf936e75c757bfd743702
author : Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Jun 2020 11:57:41 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Wed, 10 Jun 2020 11:57:41 +0200
The previous description string still described the pre-PostgreSQL
10 (pre eb61136dc75a76caef8460fa939244d8593100f2) behavior of
selecting between encrypted and unencrypted, but it is now choosing
between encryption algorithms.
M src/backend/utils/misc/guc.c
Fix ReorderBuffer memory overflow check.
commit : 16a8d5cf0337724affc4bbb3e8ba07f748f00898
author : Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Jun 2020 10:20:10 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Wed, 10 Jun 2020 10:20:10 +0530
Commit cec2edfa78 introduced logical_decoding_work_mem to limit
ReorderBuffer memory usage. We spill the changes once the memory occupied
by changes exceeds logical_decoding_work_mem. There was an assumption
in the code that by evicting the largest (sub)transaction we will come
under the memory limit as the selected transaction will be at least as
large as the most recent change (which caused us to go over the memory
limit). However, that is not true because a user can reduce the
logical_decoding_work_mem to a smaller value before the most recent
change.
We fix it by allowing to evict the transactions until we reach under the
memory limit.
Reported-by: Fujii Masao
Author: Amit Kapila
Reviewed-by: Fujii Masao
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/2b7ba291-22e0-a187-d167-9e5309a3458d@oss.nttdata.com
M src/backend/replication/logical/reorderbuffer.c
Spelling adjustments
commit : a5202889b4c78e8ffcdd8be35d59f0a7aa644f64
author : Peter Eisentraut <peter@eisentraut.org>
date : Tue, 9 Jun 2020 10:41:41 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Tue, 9 Jun 2020 10:41:41 +0200
similar to 0fd2a79a637f9f96b9830524823df0454e962f96
M contrib/pg_trgm/trgm_regexp.c
M src/backend/commands/explain.c
M src/backend/optimizer/plan/planner.c
M src/pl/plpgsql/src/expected/plpgsql_control.out
M src/pl/plpgsql/src/pl_exec.c
M src/pl/plpgsql/src/pl_gram.y
M src/pl/plpgsql/src/plpgsql.h
Fix invalid function references in a few comments
commit : 4d655f154565662cbd11f1ce5c0e6a90cd6a8f56
author : David Rowley <drowley@postgresql.org>
date : Tue, 9 Jun 2020 18:43:58 +1200
committer: David Rowley <drowley@postgresql.org>
date : Tue, 9 Jun 2020 18:43:58 +1200
These appear to have been forgotten when the functions were renamed in
1fd687a03.
Backpatch-through: 13, where the functions were renamed
M src/backend/utils/adt/numutils.c
Repair unstable regression test.
commit : 6df8fb391b0b98efc917b5cc43de77cba785864a
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 9 Jun 2020 01:17:59 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Tue, 9 Jun 2020 01:17:59 -0400
Commit 0c882e52a tried to force table atest12 to have more-accurate-
than-default statistics; but transiently setting default_statistics_target
isn't enough for that, because autovacuum could come along and overwrite
the stats later. This evidently explains some intermittent buildfarm
failures we've seen since then. Repair by disabling autovac on this table.
Thanks to David Rowley for correctly diagnosing the cause.
Discussion: https://postgr.es/m/CA+hUKG+OUkQSOUTg=qo=S=fWa_tbm99i7rB7mfbHz1SYm4v-jQ@mail.gmail.com
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Fix HashAgg regression from choosing too many initial buckets.
commit : 2174d40117f62099c7b11a2d43d163b7b9271d39
author : Jeff Davis <jdavis@postgresql.org>
date : Mon, 8 Jun 2020 20:59:45 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Mon, 8 Jun 2020 20:59:45 -0700
Diagnosis by Andres.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRDLVakD5Aagt3yZeEQeTeEWaS3YE5h8XC3Q3qJ6TYkc2Q%40mail.gmail.com
Backpatch-through: 13
M src/backend/executor/nodeAgg.c
Avoid need for valgrind suppressions for pg_atomic_init_u64 on some platforms.
commit : de4a25989611d960360d0513d00b970d3b6c52c7
author : Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 19:52:19 -0700
committer: Andres Freund <andres@anarazel.de>
date : Mon, 8 Jun 2020 19:52:19 -0700
Previously we used pg_atomic_write_64_impl inside
pg_atomic_init_u64. That works correctly, but on platforms without
64bit single copy atomicity it could trigger spurious valgrind errors
about uninitialized memory, because we use compare_and_swap for atomic
writes on such platforms.
I previously suppressed one instance of this problem (6c878edc1df),
but as Tom reports that wasn't enough. As the atomic variable cannot
yet be concurrently accessible during initialization, it seems better
to have pg_atomic_init_64_impl set the value directly.
Change pg_atomic_init_u32_impl for symmetry.
Reported-By: Tom Lane
Author: Andres Freund
Discussion: https://postgr.es/m/1714601.1591503815@sss.pgh.pa.us
Backpatch: 9.5-
M src/include/port/atomics/generic.h
M src/tools/valgrind.supp
Fix locking bugs that could corrupt pg_control.
commit : acefa2cca6a2c2b9c0fd9f25d0c003595bed689e
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 8 Jun 2020 13:57:24 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 8 Jun 2020 13:57:24 +1200
The redo routines for XLOG_CHECKPOINT_{ONLINE,SHUTDOWN} must acquire
ControlFileLock before modifying ControlFile->checkPointCopy, or the
checkpointer could write out a control file with a bad checksum.
Likewise, XLogReportParameters() must acquire ControlFileLock before
modifying ControlFile and calling UpdateControlFile().
Back-patch to all supported releases.
Author: Nathan Bossart <bossartn@amazon.com>
Author: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/70BF24D6-DC51-443F-B55A-95735803842A%40amazon.com
M src/backend/access/transam/xlog.c
Doc: Update example symptom of systemd misconfiguration.
commit : a1c940cc58828b81cd72e04dd264fbc65e46f0de
author : Thomas Munro <tmunro@postgresql.org>
date : Mon, 8 Jun 2020 13:20:46 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Mon, 8 Jun 2020 13:20:46 +1200
In PostgreSQL 10, we stopped using System V semaphores on Linux
systems. Update the example we give of an error message from a
misconfigured system to show what people are most likely to see these
days.
Back-patch to 10, where PREFERRED_SEMAPHORES=UNNAMED_POSIX arrived.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGLmJUSwybaPQv39rB8ABpqJq84im2UjZvyUY4feYhpWMw%40mail.gmail.com
M doc/src/sgml/runtime.sgml
Fix crash in WAL sender when starting physical replication
commit : 10ffe0fa72ed895a3c18aef2d3950b480e810e13
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 8 Jun 2020 10:12:31 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 8 Jun 2020 10:12:31 +0900
Since database connections can be used with WAL senders in 9.4, it is
possible to use physical replication. This commit fixes a crash when
starting physical replication with a WAL sender using a database
connection, caused by the refactoring done in 850196b.
There have been discussions about forbidding the use of physical
replication in a database connection, but this is left for later,
taking care only of the crash new to 13.
While on it, add a test to check for a failure when attempting logical
replication if the WAL sender does not have a database connection. This
part is extracted from a larger patch by Kyotaro Horiguchi.
Reported-by: Vladimir Sitnikov
Author: Michael Paquier, Kyotaro Horiguchi
Reviewed-by: Kyotaro Horiguchi, Álvaro Herrera
Discussion: https://postgr.es/m/CAB=Je-GOWMj1PTPkeUhjqQp-4W3=nW-pXe2Hjax6rJFffB5_Aw@mail.gmail.com
Backpatch-through: 13
M src/backend/access/transam/xlogreader.c
M src/backend/replication/walsender.c
M src/include/access/xlogreader.h
M src/test/recovery/t/006_logical_decoding.pl
MSVC: Avoid warning when testing a TAP suite without PROVE_FLAGS.
commit : 9b5f85fb0a3e27040bc72451893d2dc35bb5d8bd
author : Noah Misch <noah@leadboat.com>
date : Sun, 7 Jun 2020 16:27:13 -0700
committer: Noah Misch <noah@leadboat.com>
date : Sun, 7 Jun 2020 16:27:13 -0700
Commit 7be5d8df1f74b78620167d3abf32ee607e728919 surfaced the logic
error, which had no functional implications, by adding "use warnings".
The buildfarm always customizes PROVE_FLAGS, so the warning did not
appear there. Back-patch to 9.5 (all supported versions).
M src/tools/msvc/vcregress.pl
pgindent run prior to branching v13.
commit : b5d69b7c22ee4c44b8bb99cfa0466ffaf3b5fab9
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Jun 2020 16:57:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Jun 2020 16:57:08 -0400
pgperltidy and reformat-dat-files too, though those didn't
find anything to change.
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/optimizer/plan/createplan.c
M src/backend/postmaster/autovacuum.c
M src/backend/utils/sort/logtape.c
M src/include/access/tableam.h
M src/tools/pgindent/typedefs.list
Try to read data from the socket in pqSendSome's write_failed paths.
commit : 7247e243a803044a79a2828ced51b05765e049a0
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Jun 2020 13:44:13 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Jun 2020 13:44:13 -0400
Even when we've concluded that we have a hard write failure on the
socket, we should continue to try to read data. This gives us an
opportunity to collect any final error message that the backend might
have sent before closing the connection; moreover it is the job of
pqReadData not pqSendSome to close the socket once EOF is detected.
Due to an oversight in 1f39a1c06, pqSendSome failed to try to collect
data in the case where we'd already set write_failed. The problem was
masked for ordinary query operations (which really only make one write
attempt anyway), but COPY to the server would continue to send data
indefinitely after a mid-COPY connection loss.
Hence, add pqReadData calls into the paths where pqSendSome drops data
because of write_failed. If we've lost the connection, this will
eventually result in closing the socket and setting CONNECTION_BAD,
which will cause PQputline and siblings to report failure, allowing
the application to terminate the COPY sooner. (Basically this restores
what happened before 1f39a1c06.)
There are related issues that this does not solve; for example, if the
backend sends an error but doesn't drop the connection, we did and
still will keep pumping COPY data as long as the application sends it.
Fixing that will require application-visible behavior changes though,
and anyway it's an ancient behavior that we've had few complaints about.
For now I'm just trying to fix the regression from 1f39a1c06.
Per a complaint from Andres Freund. Back-patch into v12 where
1f39a1c06 came in.
Discussion: https://postgr.es/m/20200603201242.ofvm4jztpqytwfye@alap3.anarazel.de
M src/interfaces/libpq/fe-misc.c
Rethink definition of cancel.c's CancelRequested flag.
commit : 92f33bb7afd373ed562e23077c14831944d1b0d4
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Jun 2020 13:07:31 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Sun, 7 Jun 2020 13:07:31 -0400
As it stands, this flag is only set when we've successfully sent a
cancel request, not if we get SIGINT and then fail to send a cancel.
However, for almost all callers, that's the Wrong Thing: we'd prefer
to abort processing after control-C even if no cancel could be sent.
As an example, since commit 1d468b9ad "pgbench -i" fails to give up
sending COPY data even after control-C, if the postmaster has been
stopped, which is clearly not what the code intends and not what anyone
would want. (The fact that it keeps going at all is the fault of a
separate bug in libpq, but not letting CancelRequested become set is
clearly not what we want here.)
The sole exception, as far as I can find, is that scripts_parallel.c's
ParallelSlotsGetIdle tries to consume a query result after issuing a
cancel, which of course might not terminate quickly if no cancel
happened. But that behavior was poorly thought out too. No user of
ParallelSlotsGetIdle tries to continue processing after a cancel,
so there is really no point in trying to clear the connection's state.
Moreover this has the same defect as for other users of cancel.c,
that if the cancel request fails for some reason then we end up with
control-C being completely ignored. (On top of that, select_loop failed
to distinguish clearly between SIGINT and other reasons for select(2)
failing, which means that it's possible that the existing code would
think that a cancel has been sent when it hasn't.)
Hence, redefine CancelRequested as simply meaning that SIGINT was
received. We could add a second flag with the other meaning, but
in the absence of any compelling argument why such a flag is needed,
I think it would just offer an opportunity for future callers to
get it wrong. Also remove the consumeQueryResult call in
ParallelSlotsGetIdle's failure exit. In passing, simplify the
API of select_loop.
It would now be possible to re-unify psql's cancel_pressed with
CancelRequested, partly undoing 5d43c3c54. But I'm not really
convinced that that's worth the trouble, so I left psql alone,
other than fixing a misleading comment.
This code is new in v13 (cf a4fd3aa71), so no need for back-patch.
Per investigation of a complaint from Andres Freund.
Discussion: https://postgr.es/m/20200603201242.ofvm4jztpqytwfye@alap3.anarazel.de
M src/bin/psql/common.c
M src/bin/scripts/scripts_parallel.c
M src/fe_utils/cancel.c
Fix platform-specific performance regression in logtape.c.
commit : 1fbb6c93df30801f83c6804ab7befde3cdefe677
author : Jeff Davis <jdavis@postgresql.org>
date : Sun, 7 Jun 2020 09:14:24 -0700
committer: Jeff Davis <jdavis@postgresql.org>
date : Sun, 7 Jun 2020 09:14:24 -0700
Commit 24d85952 made a change that indirectly caused a performance
regression by triggering a change in the way GCC optimizes memcpy() on
some platforms.
The behavior seemed to contradict a GCC document, so I filed a report:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95556
This patch implements a narrow workaround which eliminates the
regression I observed. The workaround is benign enough that it seems
unlikely to cause a different regression on another platform.
Discussion: https://postgr.es/m/99b2eab335c1592c925d8143979c8e9e81e1575f.camel@j-davis.com
M src/backend/utils/sort/logtape.c
psql: Format \? output a little better
commit : aa7927698acb813283d21aa6a47a67cd3c5a8b0c
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 16:12:05 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 16:12:05 +0200
M src/bin/psql/help.c
Fix message translatability
commit : 1e8ada0c8a448891971faf71f48125439ee07023
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 15:11:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 15:11:51 +0200
Two parts of the same message shouldn't be split across two function
calls.
M src/bin/psql/help.c
Spelling adjustments
commit : 0fd2a79a637f9f96b9830524823df0454e962f96
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 15:06:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 15:06:51 +0200
M doc/src/sgml/config.sgml
M src/backend/commands/async.c
M src/backend/commands/vacuum.c
M src/backend/jit/llvm/llvmjit_expr.c
M src/backend/port/win32/socket.c
M src/backend/port/win32/timer.c
M src/backend/postmaster/autovacuum.c
M src/backend/postmaster/checkpointer.c
M src/backend/postmaster/pgstat.c
M src/backend/postmaster/postmaster.c
M src/backend/replication/logical/origin.c
M src/backend/replication/logical/reorderbuffer.c
M src/backend/storage/ipc/latch.c
M src/backend/storage/ipc/procsignal.c
M src/backend/storage/ipc/shm_mq.c
M src/backend/storage/ipc/signalfuncs.c
M src/backend/storage/ipc/standby.c
M src/backend/storage/lmgr/lock.c
M src/backend/tcop/postgres.c
M src/backend/utils/cache/relfilenodemap.c
M src/include/access/tableam.h
M src/include/access/xact.h
M src/include/replication/slot.h
M src/include/storage/condition_variable.h
M src/include/storage/procsignal.h
M src/test/modules/test_shm_mq/setup.c
doc: Fix man page whitespace issues
commit : a02b8bdd9878ae1d1ead87aabb673d60432500ea
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 14:54:28 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 14:54:28 +0200
Whitespace between tags is significant, and in some cases it creates
extra vertical space in man pages. The fix is either to remove some
newlines or in some cases to reword slightly to avoid the awkward
markup layout.
M doc/src/sgml/ref/alter_collation.sgml
M doc/src/sgml/ref/alter_subscription.sgml
M doc/src/sgml/ref/alter_type.sgml
M doc/src/sgml/ref/alter_view.sgml
M doc/src/sgml/ref/create_extension.sgml
M doc/src/sgml/ref/create_language.sgml
M doc/src/sgml/ref/create_publication.sgml
M doc/src/sgml/ref/create_subscription.sgml
M doc/src/sgml/ref/create_view.sgml
M doc/src/sgml/ref/drop_function.sgml
M doc/src/sgml/ref/drop_procedure.sgml
M doc/src/sgml/ref/drop_routine.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/ref/pgbench.sgml
M doc/src/sgml/ref/pgupgrade.sgml
M doc/src/sgml/ref/psql-ref.sgml
M doc/src/sgml/ref/select.sgml
M doc/src/sgml/spi.sgml
Formatting and punctuation improvements in postgresql.conf.sample
commit : f4c88ce1a20e8e944d74cb964926781d6ca4cb18
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 14:35:12 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 14:35:12 +0200
M src/backend/utils/misc/postgresql.conf.sample
doc: Move options on man pages into more alphabetical order
commit : b25da866152347109943f998b66b1a320a9de3e0
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 14:07:33 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 14:07:33 +0200
M doc/src/sgml/ref/dropdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dumpall.sgml
M doc/src/sgml/ref/pg_rewind.sgml
M doc/src/sgml/ref/pgbench.sgml
doc: Fix up spacing around verbatim DocBook elements
commit : 9ac0a26210901a5869fd7ea83ab1c59489c1aeef
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:34:37 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:34:37 +0200
M doc/src/sgml/hstore.sgml
M doc/src/sgml/release-13.sgml
doc: Language review
commit : 4c6f70cd33ac395dea1acca7dabf4cb8556235e7
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:27:57 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:27:57 +0200
M doc/src/sgml/libpq.sgml
doc: Trim trailing whitespace
commit : b79cb8a919c2614c81ae7578b863b7f582a9baf2
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:24:40 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:24:40 +0200
M doc/src/sgml/ref/drop_database.sgml
M doc/src/sgml/ref/pg_checksums.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/release-13.sgml
doc: Clean up title case use
commit : b3c2412e70f2be25ac70f7e9b2f12dbe4efd2a8b
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:18:36 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:18:36 +0200
M doc/src/sgml/features.sgml
doc: Remove line breaks after <title>
commit : ab5b55505ec4bf08a9f93810e1bfada93bc63bb5
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:10:18 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 13:10:18 +0200
This creates unnecessary rendering problem risks, and it's
inconsistent and gets copied around.
M doc/src/sgml/features.sgml
M doc/src/sgml/ref/createdb.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/ref/pg_dump.sgml
M doc/src/sgml/ref/pg_receivewal.sgml
M doc/src/sgml/ref/pg_verifybackup.sgml
M doc/src/sgml/xplang.sgml
Doc: Clean up references to obsolete OS versions.
commit : c8be915aa9fcc4c0cba563ddbb2e5af7a2dadd12
author : Thomas Munro <tmunro@postgresql.org>
date : Sun, 7 Jun 2020 21:36:43 +1200
committer: Thomas Munro <tmunro@postgresql.org>
date : Sun, 7 Jun 2020 21:36:43 +1200
Remove obsolete instructions for old operating system versions, and
update the text to reflect the defaults on modern systems.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/CA%2BhUKGLmJUSwybaPQv39rB8ABpqJq84im2UjZvyUY4feYhpWMw%40mail.gmail.com
M doc/src/sgml/runtime.sgml
doc: Fix incorrect link target
commit : c14a98032b17d514a195e4e76073ebc98a6521be
author : Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 11:16:51 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sun, 7 Jun 2020 11:16:51 +0200
M doc/src/sgml/func.sgml
Add missing source files to nls.mk
commit : 35b527428d6110dd0de585223a4783fe996a0020
author : Peter Eisentraut <peter@eisentraut.org>
date : Sat, 6 Jun 2020 19:56:21 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Sat, 6 Jun 2020 19:56:21 +0200
M src/bin/pg_basebackup/nls.mk
M src/bin/pg_rewind/nls.mk
M src/bin/psql/nls.mk
M src/bin/scripts/nls.mk
Fix reference to wrong view in release notes
commit : 6e2f11b631b712d691aecdbbcaa7a75b391c1e98
author : Magnus Hagander <magnus@hagander.net>
date : Sat, 6 Jun 2020 15:35:42 +0200
committer: Magnus Hagander <magnus@hagander.net>
date : Sat, 6 Jun 2020 15:35:42 +0200
The estimate of total backup size effects the view
pg_stat_progress_basebackup, not pg_stat_progress_analyze.
M doc/src/sgml/release-13.sgml
Refresh function name in CRC-associated Valgrind suppressions.
commit : 26056b3ba84d6cb51eea5d6c83fefae19919a56b
author : Noah Misch <noah@leadboat.com>
date : Fri, 5 Jun 2020 20:10:53 -0700
committer: Noah Misch <noah@leadboat.com>
date : Fri, 5 Jun 2020 20:10:53 -0700
Back-patch to 9.5, where commit 4f700bcd20c087f60346cb8aefd0e269be8e2157
first appeared.
Reviewed by Tom Lane. Reported by Andrew Dunstan.
Discussion: https://postgr.es/m/4dfabec2-a3ad-0546-2d62-f816c97edd0c@2ndQuadrant.com
M src/tools/valgrind.supp
Doc: remove annotations about multi-row output of set-returning functions.
commit : ec5d6fc4ae8c75391d99993cd030a8733733747d
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Jun 2020 18:04:37 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Jun 2020 18:04:37 -0400
I thought this added clarity, or at least was consistent with the way
these entries looked before v13 ... but apparently I'm in the minority.
Discussion: https://postgr.es/m/CAFj8pRAXuetiHUfs73zjsJD6B78FWcUsBS-j23sdCMFXkgx5Fg@mail.gmail.com
M doc/src/sgml/func.sgml
Improve ineq_histogram_selectivity's behavior for non-default orderings.
commit : 0c882e52a8660114234a0c4a29db919bb727e552
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Jun 2020 16:55:16 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Jun 2020 16:55:16 -0400
ineq_histogram_selectivity() can be invoked in situations where the
ordering we care about is not that of the column's histogram. We could
be considering some other collation, or even more drastically, the
query operator might not agree at all with what was used to construct
the histogram. (We'll get here for anything using scalarineqsel-based
estimators, so that's quite likely to happen for extension operators.)
Up to now we just ignored this issue and assumed we were dealing with
an operator/collation whose sort order exactly matches the histogram,
possibly resulting in junk estimates if the binary search gets confused.
It's past time to improve that, since the use of nondefault collations
is increasing. What we can do is verify that the given operator and
collation match what's recorded in pg_statistic, and use the existing
code only if so. When they don't match, instead execute the operator
against each histogram entry, and take the fraction of successes as our
selectivity estimate. This gives an estimate that is probably good to
about 1/histogram_size, with no assumptions about ordering. (The quality
of the estimate is likely to degrade near the ends of the value range,
since the two orderings probably don't agree on what is an extremal value;
but this is surely going to be more reliable than what we did before.)
At some point we might further improve matters by storing more than one
histogram calculated according to different orderings. But this code
would still be good fallback logic when no matches exist, so that is
not an argument for not doing this.
While here, also improve get_variable_range() to deal more honestly
with non-default collations.
This isn't back-patchable, because it requires adding another argument
to ineq_histogram_selectivity, and because it might have significant
impact on the estimation results for extension operators relying on
scalarineqsel --- mostly for the better, one hopes, but in any case
destabilizing plan choices in back branches is best avoided.
Per investigation of a report from James Lucas.
Discussion: https://postgr.es/m/CAAFmbbOvfi=wMM=3qRsPunBSLb8BFREno2oOzSBS=mzfLPKABw@mail.gmail.com
M src/backend/utils/adt/like_support.c
M src/backend/utils/adt/selfuncs.c
M src/backend/utils/cache/lsyscache.c
M src/include/utils/lsyscache.h
M src/include/utils/selfuncs.h
M src/test/regress/expected/privileges.out
M src/test/regress/sql/privileges.sql
Add unlikely() to CHECK_FOR_INTERRUPTS()
commit : 87fb04af1e705b615ac01feba958f841ea4a71a6
author : Joe Conway <mail@joeconway.com>
date : Fri, 5 Jun 2020 16:49:25 -0400
committer: Joe Conway <mail@joeconway.com>
date : Fri, 5 Jun 2020 16:49:25 -0400
Add the unlikely() branch hint macro to CHECK_FOR_INTERRUPTS().
Backpatch to REL_10_STABLE where we first started using unlikely().
Discussion: https://www.postgresql.org/message-id/flat/8692553c-7fe8-17d9-cbc1-7cddb758f4c6%40joeconway.com
M src/include/miscadmin.h
Use query collation, not column's collation, while examining statistics.
commit : 044c99bc567ac5d44dff0af7aebb81737dc36a69
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Jun 2020 16:18:50 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Fri, 5 Jun 2020 16:18:50 -0400
Commit 5e0928005 changed the planner so that, instead of blindly using
DEFAULT_COLLATION_OID when invoking operators for selectivity estimation,
it would use the collation of the column whose statistics we're
considering. This was recognized as still being not quite the right
thing, but it seemed like a good incremental improvement. However,
shortly thereafter we introduced nondeterministic collations, and that
creates cases where operators can fail if they're passed the wrong
collation. We don't want planning to fail in cases where the query itself
would work, so this means that we *must* use the query's collation when
invoking operators for estimation purposes.
The only real problem this creates is in ineq_histogram_selectivity, where
the binary search might produce a garbage answer if we perform comparisons
using a different collation than the column's histogram is ordered with.
However, when the query's collation is significantly different from the
column's default collation, the estimate we previously generated would be
pretty irrelevant anyway; so it's not clear that this will result in
noticeably worse estimates in practice. (A follow-on patch will improve
this situation in HEAD, but it seems too invasive for back-patch.)
The patch requires changing the signatures of mcv_selectivity and allied
functions, which are exported and very possibly are used by extensions.
In HEAD, I just did that, but an API/ABI break of this sort isn't
acceptable in stable branches. Therefore, in v12 the patch introduces
"mcv_selectivity_ext" and so on, with signatures matching HEAD, and makes
the old functions into wrappers that assume DEFAULT_COLLATION_OID should
be used. That does not match the prior behavior, but it should avoid risk
of failure in most cases. (In practice, I think most extension datatypes
aren't collation-aware, so the change probably doesn't matter to them.)
Per report from James Lucas. Back-patch to v12 where the problem was
introduced.
Discussion: https://postgr.es/m/CAAFmbbOvfi=wMM=3qRsPunBSLb8BFREno2oOzSBS=mzfLPKABw@mail.gmail.com
M contrib/ltree/ltree_op.c
M src/backend/utils/adt/like_support.c
M src/backend/utils/adt/network_selfuncs.c
M src/backend/utils/adt/selfuncs.c
M src/include/utils/selfuncs.h
OpenSSL 3.0.0 compatibility in tests
commit : f0d2c65f17cab8cfaf4d39f7f8e2254824cd4092
author : Peter Eisentraut <peter@eisentraut.org>
date : Fri, 5 Jun 2020 11:18:11 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Fri, 5 Jun 2020 11:18:11 +0200
DES has been deprecated in OpenSSL 3.0.0 which makes loading keys
encrypted with DES fail with "fetch failed". Solve by changing the
cipher used to aes256 which has been supported since 1.0.1 (and is
more realistic to use anyways).
Note that the minimum supported OpenSSL version is 1.0.1 as of
7b283d0e1d1d79bf1c962d790c94d2a53f3bb38a, so this does not introduce
any new version requirements.
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/flat/FEF81714-D479-4512-839B-C769D2605F8A%40yesql.se
M src/test/ssl/Makefile
M src/test/ssl/ssl/server-password.key
Preserve pg_index.indisreplident across REINDEX CONCURRENTLY
commit : 1127f0e392c757fc4fbbeffd7d0202bb02670e9c
author : Michael Paquier <michael@paquier.xyz>
date : Fri, 5 Jun 2020 10:26:02 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Fri, 5 Jun 2020 10:26:02 +0900
If the flag value is lost, logical decoding would work the same way as
REPLICA IDENTITY NOTHING, meaning that no old tuple values would be
included in the changes anymore produced by logical decoding.
Author: Michael Paquier
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/20200603065340.GK89559@paquier.xyz
Backpatch-through: 12
M src/backend/catalog/index.c
M src/test/regress/expected/create_index.out
M src/test/regress/sql/create_index.sql
Reject "23:59:60.nnn" in datetime input.
commit : a9632830bb05dc98ae24017cafc652e4a66d44a8
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 4 Jun 2020 16:42:08 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Thu, 4 Jun 2020 16:42:08 -0400
It's intentional that we don't allow values greater than 24 hours,
while we do allow "24:00:00" as well as "23:59:60" as inputs.
However, the range check was miscoded in such a way that it would
accept "23:59:60.nnn" with a nonzero fraction. For time or timetz,
the stored result would then be greater than "24:00:00" which would
fail dump/reload, not to mention possibly confusing other operations.
Fix by explicitly calculating the result and making sure it does not
exceed 24 hours. (This calculation is redundant with what will happen
later in tm2time or tm2timetz. Maybe someday somebody will find that
annoying enough to justify refactoring to avoid the duplication; but
that seems too invasive for a back-patched bug fix, and the cost is
probably unmeasurable anyway.)
Note that this change also rejects such input as the time portion
of a timestamp(tz) value.
Back-patch to v10. The bug is far older, but to change this pre-v10
we'd need to ensure that the logic behaves sanely with float timestamps,
which is possibly nontrivial due to roundoff considerations.
Doesn't really seem worth troubling with.
Per report from Christoph Berg.
Discussion: https://postgr.es/m/20200520125807.GB296739@msg.df7cb.de
M src/backend/utils/adt/date.c
M src/backend/utils/adt/datetime.c
M src/backend/utils/adt/timestamp.c
M src/include/utils/date.h
M src/test/regress/expected/time.out
M src/test/regress/expected/timetz.out
M src/test/regress/sql/time.sql
M src/test/regress/sql/timetz.sql
psql: Clean up terminology in \dAp command
commit : f5067049cde38cd0d6333a5e3bf1bed8d99e6f44
author : Peter Eisentraut <peter@eisentraut.org>
date : Thu, 4 Jun 2020 22:09:41 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Thu, 4 Jun 2020 22:09:41 +0200
The preferred terminology has been support "function", not procedure,
for some time, so change that over. The command stays \dAp, since
\dAf is already something else.
M doc/src/sgml/ref/psql-ref.sgml
M src/bin/psql/command.c
M src/bin/psql/describe.c
M src/bin/psql/describe.h
M src/bin/psql/help.c
M src/test/regress/expected/psql.out
Fix comment in be-secure-openssl.c
commit : 3fa44a30049826bfe2fd58eec0e8acabd5757411
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Jun 2020 13:02:59 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Jun 2020 13:02:59 +0900
Since 573bd08, hardcoded DH parameters have been moved to a different
file, making the comment on top of load_dh_buffer() incorrect.
Author: Daniel Gustafsson
Discussion: https://postgr.es/m/D9492CCB-9A91-4181-A847-1779630BE2A7@yesql.se
M src/backend/libpq/be-secure-openssl.c
Fix instance of elog() called while holding a spinlock
commit : c1669fd5812a02daac58778e2708ede11edd36a3
author : Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Jun 2020 10:17:49 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Thu, 4 Jun 2020 10:17:49 +0900
This broke the project rule to not call any complex code while a
spinlock is held. Issue introduced by b89e151.
Discussion: https://postgr.es/m/20200602.161518.1399689010416646074.horikyota.ntt@gmail.com
Backpatch-through: 9.5
M src/backend/replication/logical/logical.c
Don't call palloc() while holding a spinlock, either.
commit : f88bd3139f3e2a557c086215c6b15d7f66bee845
author : Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Jun 2020 12:36:00 -0400
committer: Tom Lane <tgl@sss.pgh.pa.us>
date : Wed, 3 Jun 2020 12:36:00 -0400
Fix some more violations of the "only straight-line code inside a
spinlock" rule. These are hazardous not only because they risk
holding the lock for an excessively long time, but because it's
possible for palloc to throw elog(ERROR), leaving a stuck spinlock
behind.
copy_replication_slot() had two separate places that did pallocs
while holding a spinlock. We can make the code simpler and safer
by copying the whole ReplicationSlot struct into a local variable
while holding the spinlock, and then referencing that copy.
(While that's arguably more cycles than we really need to spend
holding the lock, the struct isn't all that big, and this way seems
far more maintainable than copying fields piecemeal. Anyway this
is surely much cheaper than a palloc.) That bug goes back to v12.
InvalidateObsoleteReplicationSlots() not only did a palloc while
holding a spinlock, but for extra sloppiness then leaked the memory
--- probably for the lifetime of the checkpointer process, though
I didn't try to verify that. Fortunately that silliness is new
in HEAD.
pg_get_replication_slots() had a cosmetic violation of the rule,
in that it only assumed it's safe to call namecpy() while holding
a spinlock. Still, that's a hazard waiting to bite somebody, and
there were some other cosmetic coding-rule violations in the same
function, so clean it up. I back-patched this as far as v10; the
code exists before that but it looks different, and this didn't
seem important enough to adapt the patch further back.
Discussion: https://postgr.es/m/20200602.161518.1399689010416646074.horikyota.ntt@gmail.com
M src/backend/replication/slot.c
M src/backend/replication/slotfuncs.c
M src/include/replication/slot.h
doc: PG 13 relnotes: fix link for grouping sets hash overflow
commit : 4d685f6d7b65fa1ca5afb5138e610cd08a3c1e12
author : Bruce Momjian <bruce@momjian.us>
date : Tue, 2 Jun 2020 22:11:47 -0400
committer: Bruce Momjian <bruce@momjian.us>
date : Tue, 2 Jun 2020 22:11:47 -0400
Use "guc-enable-groupingsets-hash-disk".
Reported-by: TAKATSUKA Haruka
Discussion: https://postgr.es/m/16468-7939d39f1786516c@postgresql.org
Backpatch-through: master
M doc/src/sgml/release-13.sgml
doc: Move wal_init_zero and wal_recycle descriptions to proper section.
commit : 43e592c706f8ce073d166f541687ad8f02dc22c0
author : Fujii Masao <fujii@postgresql.org>
date : Wed, 3 Jun 2020 09:59:43 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Wed, 3 Jun 2020 09:59:43 +0900
The group of wal_init_zero and wal_recycle is WAL_SETTINGS in guc.c,
but previously their documents were located in
"Replication"/"Sending Servers" section. This commit moves them to
the proper section "Write Ahead Log"/"Settings".
Back-patch to v12 where wal_init_zero and wal_recycle parameters
were introduced.
Author: Fujii Masao
Discussion: https://postgr.es/m/b5190ab4-a169-6a42-0e49-aed0807c8976@oss.nttdata.com
M doc/src/sgml/config.sgml
Don't call elog() while holding spinlock.
commit : caa3c4242cf86322e2ed0c86199e6462a2c41565
author : Fujii Masao <fujii@postgresql.org>
date : Tue, 2 Jun 2020 19:18:13 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Tue, 2 Jun 2020 19:18:13 +0900
Previously UpdateSpillStats() called elog(DEBUG2) while holding
the spinlock even though the local variables that the elog() accesses
don't need to be protected by the lock. Since spinlocks are intended
for very short-term locks, they should not be used when calling
elog(DEBUG2). So this commit moves that elog() out of spinlock period.
Author: Kyotaro Horiguchi
Reviewed-by: Amit Kapila and Fujii Masao
Discussion: https://postgr.es/m/20200602.161518.1399689010416646074.horikyota.ntt@gmail.com
M src/backend/replication/walsender.c
Doc: Update the documentation for spilled transaction statistics.
commit : e641b2a995abfa0dd7039863e2597feb3abf2b1e
author : Amit Kapila <akapila@postgresql.org>
date : Tue, 2 Jun 2020 11:11:25 +0530
committer: Amit Kapila <akapila@postgresql.org>
date : Tue, 2 Jun 2020 11:11:25 +0530
Reported-by: Sawada Masahiko
Author: Sawada Masahiko, Amit Kapila
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CA+fd4k4vNg7dRO5ECHdtQXXf1=Q4M98pfLW0dU7BKD8h79pkqA@mail.gmail.com
M doc/src/sgml/monitoring.sgml
Make ssl certificate for ssl_passphrase_callback test via Makefile
commit : b846091fd0a7a747933232016f0a52aa764398b8
author : Andrew Dunstan <andrew@dunslane.net>
date : Mon, 1 Jun 2020 17:32:32 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Mon, 1 Jun 2020 17:32:32 -0400
The recipe was previously given in comments in the module's test
script, but now we have an explicit recipe in the Makefile. The now
redundant comments in the script are removed.
This recipe shouldn't be needed in normal use, as the certificate and
key are in git and don't need to be regenerated.
Discussion: https://postgr.es/m/ae8f21fc-95cb-c98a-f241-1936133f466f@2ndQuadrant.com
M src/test/modules/ssl_passphrase_callback/Makefile
M src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
Use correct and consistent unit abbreviation
commit : 42181b1015b18e877e65be66ac5a2e90b731ac8b
author : Peter Eisentraut <peter@eisentraut.org>
date : Mon, 1 Jun 2020 21:18:36 +0200
committer: Peter Eisentraut <peter@eisentraut.org>
date : Mon, 1 Jun 2020 21:18:36 +0200
M src/backend/storage/sync/sync.c
Fix use-after-release mistake in currtid() and currtid2() for views
commit : ce1c5b9ae87b6153d3f40a4f7806f2effef12363
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Jun 2020 14:41:18 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Jun 2020 14:41:18 +0900
This issue has been present since the introduction of this code as of
a3519a2 from 2002, and has been found by buildfarm member prion that
uses RELCACHE_FORCE_RELEASE via the tests introduced recently in
e786be5.
Discussion: https://postgr.es/m/20200601022055.GB4121@paquier.xyz
Backpatch-through: 9.5
M src/backend/utils/adt/tid.c
Fix crashes with currtid() and currtid2()
commit : e786be5fcb257a09b05bd8e509c8d1b82e626352
author : Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Jun 2020 10:32:06 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Mon, 1 Jun 2020 10:32:06 +0900
A relation that has no storage initializes rd_tableam to NULL, which
caused those two functions to crash because of a pointer dereference.
Note that in 11 and older versions, this has always failed with a
confusing error "could not open file".
These two functions are used by the Postgres ODBC driver, which requires
them only when connecting to a backend strictly older than 8.1. When
connected to 8.2 or a newer version, the driver uses a RETURNING clause
instead whose support has been added in 8.2, so it should be possible to
just remove both functions in the future. This is left as an issue to
address later.
While on it, add more regression tests for those functions as we never
really had coverage for them, and for aggregates of TIDs.
Reported-by: Jaime Casanova, via sqlsmith
Author: Michael Paquier
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/CAJGNTeO93u-5APMga6WH41eTZ3Uee9f3s8dCpA-GSSqNs1b=Ug@mail.gmail.com
Backpatch-through: 12
M src/backend/utils/adt/tid.c
A src/test/regress/expected/tid.out
M src/test/regress/parallel_schedule
M src/test/regress/serial_schedule
A src/test/regress/sql/tid.sql
Make install-tests target work with vpath builds
commit : af4ea507c3d9217579a8d75fc17f4796a9bab0bb
author : Andrew Dunstan <andrew@dunslane.net>
date : Sun, 31 May 2020 18:33:00 -0400
committer: Andrew Dunstan <andrew@dunslane.net>
date : Sun, 31 May 2020 18:33:00 -0400
Also add a top-level install-tests target.
Backpatch to all live branches.
Craig Ringer, tweaked by me.
M GNUmakefile.in
M src/test/regress/GNUmakefile
Use CP_SMALL_TLIST for hash aggregate
commit : 4cad2534da6d17067d98cf04be2dfc1bda8f2cd0
author : Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 31 May 2020 14:43:13 +0200
committer: Tomas Vondra <tomas.vondra@postgresql.org>
date : Sun, 31 May 2020 14:43:13 +0200
Commit 1f39bce021 added disk-based hash aggregation, which may spill
incoming tuples to disk. It however did not request projection to make
the tuples as narrow as possible, which may mean having to spill much
more data than necessary (increasing I/O, pushing other stuff from page
cache, etc.).
This adds CP_SMALL_TLIST in places that may use hash aggregation - we do
that only for AGG_HASHED. It's unnecessary for AGG_SORTED, because that
either uses explicit Sort (which already does projection) or pre-sorted
input (which does not need spilling to disk).
Author: Tomas Vondra
Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/20200519151202.u2p2gpiawoaznsv2%40development
M contrib/postgres_fdw/expected/postgres_fdw.out
M src/backend/optimizer/plan/createplan.c
Doc: Mention about caveats of --concurrently on reindexdb page
commit : 9b60c4b979bce060495e2b05ba01d1cc6bffdd2d
author : Michael Paquier <michael@paquier.xyz>
date : Sun, 31 May 2020 10:48:21 +0900
committer: Michael Paquier <michael@paquier.xyz>
date : Sun, 31 May 2020 10:48:21 +0900
The documentation of REINDEX includes a complete description of
CONCURRENTLY and its advantages as well as its disadvantages, but
reindexdb was not really clear about all that.
From discussion with Tom Lane, based on a report from Andrey Klychkov.
Discussion: https://postgr.es/m/1590486572.205117372@f500.i.mail.ru
Backpatch-through: 12
M doc/src/sgml/ref/reindexdb.sgml
doc: Update the layout of "Viewing Statistics" section.
commit : 92f9468657f0916ce8589e13d5ebda60c7973c31
author : Fujii Masao <fujii@postgresql.org>
date : Fri, 29 May 2020 17:14:33 +0900
committer: Fujii Masao <fujii@postgresql.org>
date : Fri, 29 May 2020 17:14:33 +0900
This commit updates the "Viewing Statistics" section more like
the existing catalogs chapter.
- Change its layout so that an introductory paragrap is put above
the table for each statistics view. Previously the explanations
were below the tables.
- Separate each view to different section and add index terms for them.
Author: Fujii Masao
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/6f8a482c-b3fa-4ed9-21c3-6d222a2cb87d@oss.nttdata.com
M doc/src/sgml/catalogs.sgml
M doc/src/sgml/config.sgml
M doc/src/sgml/high-availability.sgml
M doc/src/sgml/logical-replication.sgml
M doc/src/sgml/logicaldecoding.sgml
M doc/src/sgml/monitoring.sgml
M doc/src/sgml/ref/initdb.sgml
M doc/src/sgml/ref/pg_basebackup.sgml
M doc/src/sgml/sslinfo.sgml
llvmjit: Fix building against LLVM 11 by removing unnecessary include.
commit : 6a4a335b841520739b7b2f0e608acdf3b814daad
author : Andres Freund <andres@anarazel.de>
date : Thu, 28 May 2020 15:08:12 -0700
committer: Andres Freund <andres@anarazel.de>
date : Thu, 28 May 2020 15:08:12 -0700
LLVM has removed this header, in the branch that will become llvm
11. But as it turns out we didn't actually need it, so just remove it.
Author: Jesse Zhang <sbjesse@gmail.com>
Discussion: https://postgr.es/m/CAGf+fX7bvtP0YXMu7pOsu_NwhxW6dArTkxb=jt7M2-UJkyJ_3g@mail.gmail.com
Backpatch: 11, where JIT support using llvm was introduced.
M src/backend/jit/llvm/llvmjit_inline.cpp
Initialize dblink remoteConn struct in all cases
commit : 9003b76e169e8524f8d7c7547aded4749b9c39a1
author : Joe Conway <mail@joeconway.com>
date : Thu, 28 May 2020 13:44:54 -0400
committer: Joe Conway <mail@joeconway.com>
date : Thu, 28 May 2020 13:44:54 -0400
Two of the members of rconn were left uninitialized. When
dblink_open() is called without an outer transaction it
handles the initialization for us, but with an outer
transaction it does not. Arrange for initialization
in all cases. Backpatch to all supported versions.
Reported-by: Alexander Lakhin
Discussion: https://www.postgresql.org/message-id/flat/9bd0744f-5f04-c778-c5b3-809efe9c30c7%40joeconway.com#c545909a41664991aca60c4d70a10ce7
M contrib/dblink/dblink.c
Add CHECK_FOR_INTERRUPTS() to the repeat() function
commit : 887cdff4dcbdfbfdbf9a29dfad0edc09c6ec3398
author : Joe Conway <mail@joeconway.com>
date : Thu, 28 May 2020 13:16:47 -0400
committer: Joe Conway <mail@joeconway.com>
date : Thu, 28 May 2020 13:16:47 -0400
The repeat() function loops for potentially a long time without
ever checking for interrupts. This prevents, for example, a query
cancel from interrupting until the work is all done. Fix by
inserting a CHECK_FOR_INTERRUPTS() into the loop.
Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/flat/8692553c-7fe8-17d9-cbc1-7cddb758f4c6%40joeconway.com
M src/backend/utils/adt/oracle_compat.c